如何修复Gitlab CE 14.1.6升级失败

4
我想更新我安装在EC2实例上的GitLab CE,按照推荐,我从13.0.x升级到13.9.2 -> 13.12.12 -> 14.0.11 -> 14.1.6。 每次跳到14.X时,只需重新启动进程即可检查服务器是否响应。但当我尝试更新到14.1.6时,出现了错误。
[...]
System Info:
    ------------
    chef_version=15.17.4
    platform=ubuntu
    platform_version=20.04
    ruby=ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client

Recipe: gitlab::gitlab-rails
  * execute[clear the gitlab-rails cache] action run
    - execute /opt/gitlab/bin/gitlab-rake cache:clear
Recipe: gitaly::enable
  * runit_service[gitaly] action restart (up to date)
  * runit_service[gitaly] action hup
    - send hup to runit_service[gitaly]

Running handlers:
There was an error running gitlab-ctl reconfigure:

rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash"  "/tmp/chef-script20211028-68712-eysjor" ----
STDOUT: rake aborted!
StandardError: An error has occurred, all later migrations canceled:

Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active':   {:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", :table_name=>"events", :column_name=>"id", :job_arguments=>[["id"], ["id_convert_to_bigint"]]}

Finalize it manualy by running

    sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']

For more information, check the documentation

    https://docs.gitlab.com/ee/user/admin_area/monitoring/background_migrations.html#database-migrations-failing-because-of-batched-background-migration-not-finished
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1109:in `ensure_batched_background_migration_is_finished'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'

Caused by:
Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active':   {:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", :table_name=>"events", :column_name=>"id", :job_arguments=>[["id"], ["id_convert_to_bigint"]]}

Finalize it manualy by running

    sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']

For more information, check the documentation

    https://docs.gitlab.com/ee/user/admin_area/monitoring/background_migrations.html#database-migrations-failing-because-of-batched-background-migration-not-finished
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1109:in `ensure_batched_background_migration_is_finished'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
== 20210611082822 AddPagesFileEntriesToPlanLimits: migrating ==================
-- add_column(:plan_limits, :pages_file_entries, :integer, {:default=>200000, :null=>false})
   -> 0.0021s
== 20210611082822 AddPagesFileEntriesToPlanLimits: migrated (0.0022s) =========

== 20210622045705 FinalizeEventsBigintConversion: migrating ===================
STDERR:
---- End output of "bash"  "/tmp/chef-script20211028-68712-eysjor" ----
Ran "bash"  "/tmp/chef-script20211028-68712-eysjor" returned 1

Running handlers complete
Chef Infra Client failed. 27 resources updated in 51 seconds
===
There was an error running gitlab-ctl reconfigure. Please check the output above for more
details.
===

dpkg: error processing package gitlab-ce (--configure):
 installed gitlab-ce package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 gitlab-ce
E: Sub-process /usr/bin/dpkg returned an error code (1)

根据输出结果,我尝试执行以下命令:

sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']

但是得到了如下结果:

rake aborted!
JSON::ParserError: unexpected character () at line 1, column 1 [parse.c:714]
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/json.rb:87:in `rescue in adapter_load'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/json.rb:82:in `adapter_load'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/json.rb:25:in `parse'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/background_migrations.rake:17:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'

Caused by:
Oj::ParseError: unexpected character () at line 1, column 1 [parse.c:714]
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/json.rb:85:in `load'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/json.rb:85:in `adapter_load'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/json.rb:25:in `parse'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/background_migrations.rake:17:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:background_migrations:finalize
(See full trace by running task with --trace)

有什么想法吗?

我建议您在Gitlab支持平台上撰写一篇帖子,他们是过去唯一解决类似问题的人。 - Antonio Petricca
1
@AntonioPetricca 谢谢,我已经完成了。 - Ragnar
2个回答

2

我在这种情况下也遇到了同样的错误。为了找出问题所在,我在/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/background_migrations.rake:17中添加了以下代码:

      puts "JSON: '#{args[:job_arguments]}'"

让代码看起来像这样:

      puts "JSON: '#{args[:job_arguments]}'"

      Gitlab::Database::BackgroundMigration::BatchedMigrationRunner.finalize(
        args[:job_class_name],
        args[:table_name],
        args[:column_name],
        Gitlab::Json.parse(args[:job_arguments])
      )

我注意到命令args[:job_arguments]返回的是无效的JSON字符串。

所以最终对于我(zsh),执行以下命令:

sudo gitlab-rake 'gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,[["id"]\, ["id_convert_to_bigint"]]]'

区别在于单引号。

2
从13版本升级到14.0版本后,GitLab将在后台进行多个数据库迁移(请参见yourgitlabserver.example.com/admin/background_migrations)。
在继续之前,您必须等待这些迁移完成。
如果您已经等待完成,有几种方法可以解决此问题(请参见文档)。

最近我遇到了一个非常类似的问题,从14.1升级到14.4时,我认为版本更新之间的时间太短,没有允许后台迁移。最终,我将虚拟机回滚到先前的状态,并从13.12.12 -> 14.0.11 -> 14.1.7 -> 14.4.1进行升级,以便给后台迁移留出时间(还需要重新启动postgresql和redis服务),这样就可以正常工作了。 - Lance Barbour

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接