Presence of `grafana` keys in `gitlab.rb` prevents upgrade

Summary

The removal of Grafana still cause customer incidents to this day.
I propose we change the behavior of gitlab-ctl to handle the presence of removed keys more gracefully.

Steps to reproduce

Insert a Grafana related config key, i.e. grafana["enable"] = false.
Run gitlab-ctl reconfigure OR
Upgrade your GitLab instance

What is the current bug behavior?

gitlab-ctl reconfigure fails with a long error message, that includes the following snippet:

Mixlib::Config::UnknownConfigOptionError
----------------------------------------
Reading unsupported config value grafana.

It "ends" on the following snippet, which doesn't help conveying what's going on:

Checking PostgreSQL executables: NOT OK

What is the expected correct behavior?

gitlab-ctl reconfigure laments the presence of the grafana keys as a warning, and continues.

Alternatively

The pre_install script checks for presence of the keys and refuses to continue until the keys are removed.

Relevant logs

Relevant logs / `apt` output


Setting up gitlab-ee (17.4.2-ee.0) ... Checking PostgreSQL executables:\\\[2024-10-16T15:19:25+02:00\\\] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook) Cinc Client, version 18.3.0 Patents: https://www.chef.io/patents Infra Phase starting \\\[2024-10-16T15:19:25+02:00\\\] INFO: \\\*\\\*\\\* Cinc Client 18.3.0 \\\*\\\*\\\* \\\[2024-10-16T15:19:25+02:00\\\] INFO: Platform: x86_64-linux \\\[2024-10-16T15:19:25+02:00\\\] INFO: Cinc-client pid: 97137 \\\[2024-10-16T15:19:27+02:00\\\] INFO: Setting the run_list to \\\["recipe\\\[gitlab-ee::config\\\]", "recipe\\\[postgresql::bin\\\]"\\\] from CLI options \\\[2024-10-16T15:19:27+02:00\\\] INFO: Run List is \\\[recipe\\\[gitlab-ee::config\\\], recipe\\\[postgresql::bin\\\]\\\] \\\[2024-10-16T15:19:27+02:00\\\] INFO: Run List expands to \\\[gitlab-ee::config, postgresql::bin\\\] \\\[2024-10-16T15:19:27+02:00\\\] INFO: Starting Cinc Client Run for gl1.niklas.gcp.gitlabsandbox.net \\\[2024-10-16T15:19:27+02:00\\\] INFO: Running start handlers \\\[2024-10-16T15:19:27+02:00\\\] INFO: Start handlers complete. Resolving cookbooks for run list: \\\["gitlab-ee::config", "postgresql::bin"\\\] \\\[2024-10-16T15:19:28+02:00\\\] INFO: Loading cookbooks \\\[gitlab-ee@0.0.1, postgresql@0.1.0, package@0.1.0, gitlab@0.0.1, consul@0.1.0, patroni@0.1.0, pgbouncer@0.1.0, spamcheck@0.1.0, runit@5.1.7, logrotate@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, acme@4.1.6, crond@0.1.0\\\] Synchronizing cookbooks: - gitlab-ee (0.0.1) \\\[2024-10-16T15:19:29+02:00\\\] INFO: Storing updated cookbooks/postgresql/attributes/default.rb in the cache. \\\[2024-10-16T15:19:29+02:00\\\] INFO: Storing updated cookbooks/postgresql/templates/default/postgresql-runtime.conf.erb in the cache. - postgresql (0.1.0) \\\[2024-10-16T15:19:29+02:00\\\] INFO: Storing updated cookbooks/package/libraries/config/roles/application.rb in the cache. \\\[2024-10-16T15:19:29+02:00\\\] INFO: Storing updated cookbooks/package/libraries/deprecations.rb in the cache. \\\[2024-10-16T15:19:29+02:00\\\] INFO: Storing updated cookbooks/package/libraries/omnibus_helper.rb in the cache. - package (0.1.0) - gitlab (0.0.1) - consul (0.1.0) - patroni (0.1.0) - pgbouncer (0.1.0) - spamcheck (0.1.0) - runit (5.1.7) - logrotate (0.1.0) - redis (0.1.0) - monitoring (0.1.0) - registry (0.1.0) - mattermost (0.1.0) - gitaly (0.1.0) - praefect (0.1.0) - gitlab-kas (0.1.0) - gitlab-pages (0.1.0) - letsencrypt (0.1.0) - nginx (0.1.0) - acme (4.1.6) - crond (0.1.0) Installing cookbook gem dependencies: Compiling cookbooks... Loading Cinc Auditor profile files: Loading Cinc Auditor input files: Loading Cinc Auditor waiver files: # ================================================================================ Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab-ee/recipes/config.rb ## Mixlib::Config::UnknownConfigOptionError Reading unsupported config value grafana. ## Cookbook Trace: (most recent call first) /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:125:in \`internal_get' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:105:in \`block in from_file' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in \`auto_vivify' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:105:in \`from_file' /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:28:in \`from_file' /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab-ee/recipes/config.rb:24:in \`from_file' ## Relevant File Content: /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb: 118: end 119: 120: # Enhance get so strict mode errors aren't thrown as long as the setting is witin our defined config 121: def internal_get(symbol) 122: if configuration.key?(symbol) 123: configuration\\\[symbol\\\] 124: else 125\\\>\\\> super 126: end 127: end 128: 129: def sanitized_config 130: results = { "gitlab" =\\\> {}, "roles" =\\\> {}, "monitoring" =\\\> {} } 131: 132: # Add the settings to the results 133: sorted_settings.each do |key, value| 134: raise "Attribute parent value invalid for key: #{key} (#{value})" if value\\\[:parent\\\] && !results.key?(value\\\[:parent\\\]) ## System Info: chef_version=18.3.0 platform=ubuntu platform_version=24.04 ruby=ruby 3.1.5p253 (2024-04-023 revision 1945f8dc0e) \\\[x86_64-linux\\\] program_name=/opt/gitlab/embedded/bin/cinc-client executable=/opt/gitlab/embedded/bin/cinc-client Running handlers: \\\[2024-10-16T15:19:30+02:00\\\] ERROR: Running exception handlers Running handlers complete \\\[2024-10-16T15:19:30+02:00\\\] ERROR: Exception handlers complete Infra Phase failed. 0 resources updated in 05 seconds \\\[2024-10-16T15:19:30+02:00\\\] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out \\\[2024-10-16T15:19:30+02:00\\\] FATAL: --------------------------------------------------------------------------------------- \\\[2024-10-16T15:19:30+02:00\\\] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT \\\[2024-10-16T15:19:30+02:00\\\] FATAL: --------------------------------------------------------------------------------------- \\\[2024-10-16T15:19:30+02:00\\\] FATAL: Mixlib::Config::UnknownConfigOptionError: Reading unsupported config value grafana. Checking PostgreSQL executables: NOT OK Could not update PostgreSQL executables. Shutting down all GitLab services except those needed for migrations

Details of package version

Provide the package version installation details

Current workaround

As a workaround customers have to completely remove all occurrences of grafana in the gitlab.rb configuration file.

Setting grafana['enable'] = false does not resolve the issue.

Edited Mar 16, 2025 by Brenda Nyaringita