Ability to rollback package on GitLab-ctl reconfigure
Summary
With 16.0 it brought some breaking changes in Gitaly configuration. For GitLab.com the configuration wasn't updated so we started getting the error below
msg: dpkg --force-confdef --force-confold -i /var/cache/deploy-tooling/deb/gitlab-ee_16.0.202304240020-047ea8e1958.2361a8ef53e_amd64.deb failed
stderr: |-
dpkg: error processing archive /var/cache/deploy-tooling/deb/gitlab-ee_16.0.202304240020-047ea8e1958.2361a8ef53e_amd64.deb (--install):
new gitlab-ee package pre-installation script subprocess returned error exit status 1
Errors were encountered while processing:
/var/cache/deploy-tooling/deb/gitlab-ee_16.0.202304240020-047ea8e1958.2361a8ef53e_amd64.deb
stderr_lines: <omitted>
stdout: |-
(Reading database ... 238271 files and directories currently installed.)
Preparing to unpack .../gitlab-ee_16.0.202304240020-047ea8e1958.2361a8ef53e_amd64.deb ...
* praefect['auth_token'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['certificate_path'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['key_path'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['database_host'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['database_user'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['database_password'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['database_dbname'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['database_sslmode'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['database_sslcert'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['database_sslkey'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['database_sslrootcert'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['database_direct_host'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['database_direct_port'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* praefect['virtual_storages'] has been deprecated since 15.9 and was removed in 16.0. In GitLab 15.9, Praefect's configuration in Omnibus GitLab was changed to structurally match Praefect's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#1590
* gitaly['prometheus_listen_addr'] has been deprecated since 15.10 and was removed in 16.0. In GitLab 15.10, Gitaly's configuration in Omnibus GitLab was changed to structurally match Gitaly's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#15100
* gitaly['certificate_path'] has been deprecated since 15.10 and was removed in 16.0. In GitLab 15.10, Gitaly's configuration in Omnibus GitLab was changed to structurally match Gitaly's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#15100
* gitaly['key_path'] has been deprecated since 15.10 and was removed in 16.0. In GitLab 15.10, Gitaly's configuration in Omnibus GitLab was changed to structurally match Gitaly's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#15100
* gitaly['logging_level'] has been deprecated since 15.10 and was removed in 16.0. In GitLab 15.10, Gitaly's configuration in Omnibus GitLab was changed to structurally match Gitaly's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#15100
* gitaly['logging_format'] has been deprecated since 15.10 and was removed in 16.0. In GitLab 15.10, Gitaly's configuration in Omnibus GitLab was changed to structurally match Gitaly's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#15100
* gitaly['logging_sentry_dsn'] has been deprecated since 15.10 and was removed in 16.0. In GitLab 15.10, Gitaly's configuration in Omnibus GitLab was changed to structurally match Gitaly's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#15100
* gitaly['logging_ruby_sentry_dsn'] has been deprecated since 15.10 and was removed in 16.0. In GitLab 15.10, Gitaly's configuration in Omnibus GitLab was changed to structurally match Gitaly's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#15100
* gitaly['prometheus_grpc_latency_buckets'] has been deprecated since 15.10 and was removed in 16.0. In GitLab 15.10, Gitaly's configuration in Omnibus GitLab was changed to structurally match Gitaly's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#15100
* gitaly['auth_token'] has been deprecated since 15.10 and was removed in 16.0. In GitLab 15.10, Gitaly's configuration in Omnibus GitLab was changed to structurally match Gitaly's own configuration. Please see the migration instructions at https://docs.gitlab.com/ee/update/#15100
Deprecations found. Please correct them and try again.
stdout_lines: <omitted>
The changes weren't trivial and needed some time to refactor the configuration. We update omnibus deprecations !6841 (merged) to point to 16.1 instead of 16.0. However the deprecation check runs in the preinst
, which means it works with already existing code in the filesystem, not something from the incoming package. So, we will have to patch the file at /opt/gitlab/embedded/cookbooks/package/libraries/deprecations.rb
making the rollback quite hard.
Not having an easy way to rollback the package when we have a problem is not ideal and slows down the recovery process, and we kept seeing problems on every server that had this bad package.
Proposal
Have preinst
use the incoming package rather than the one existing in the file system so that we can easilly rollback omnibus packages.