## Summary `gitlab-ctl check-config --version 19.0.1` does not report `mattermost[...]` keys in `gitlab.rb` as a blocking removal, even though `reconfigure` after upgrading to 19.0 immediately aborts with: ``` RuntimeError: Removed configurations found in gitlab.rb. Aborting reconfigure. ``` This means customers who run the recommended pre-upgrade check get a false clean bill of health, then hit a hard failure mid-upgrade. ## Steps to reproduce 1. On GitLab 18.11.x with `mattermost[...]` keys present in `/etc/gitlab/gitlab.rb` (for example `mattermost['enable'] = true`). 2. Run `sudo gitlab-ctl check-config --version 19.0.1`. 3. Observe: no removal warning is printed for `mattermost`. 4. Upgrade to 19.0.x and run `sudo gitlab-ctl reconfigure`. 5. Observe: reconfigure aborts with `Removed configurations found in gitlab.rb`. ## Root cause In `deprecations.rb`, the `mattermost` entry has `deprecation: '19.0'` and `removal: '19.0'` — deprecated and removed in the same version with no grace period. The `check-config` command calls `Gitlab::Deprecations.check_config(version, config, :removal)`, which correctly identifies the removal. However, the detection relies on the `mattermost` key being present and non-nil/non-empty in the embedded nodes JSON (`/opt/gitlab/embedded/nodes/<fqdn>.json`). The `check-config` command reads config from the **current** installed package's node JSON, not from a fresh parse of `gitlab.rb` against the target version. If the node JSON was generated by a pre-19.0 reconfigure, the `mattermost` key may be serialised differently (e.g. as an empty hash or with default values only) than what `applicable_deprecations` expects, causing the removal check to silently pass. Additionally, `check-config` is documented as the recommended pre-upgrade validation step, but the mattermost removal is a **same-version** deprecation+removal (`19.0`/`19.0`), which is an unusual pattern that may not be exercised by the check-config test suite against a pre-19.0 node JSON. ## Impact Customers with `mattermost[...]` in `gitlab.rb` who follow the documented upgrade procedure (`check-config` → upgrade) are not warned before the upgrade. The failure only surfaces during `reconfigure` after the package has already been replaced, potentially mid-maintenance-window. Reported via Zendesk ticket 725485 (Geo customer, SMB Ultimate, upgrade 18.11.4 → 19.0.1). ## Expected behaviour `gitlab-ctl check-config --version 19.0.1` should print a clear removal warning for any `mattermost[...]` key present in `gitlab.rb`, matching the error that `reconfigure` produces. ## Workaround Comment out or remove all `mattermost[...]` lines from `/etc/gitlab/gitlab.rb` before upgrading to 19.0. ## Environment - Installation type: Linux package (Omnibus) - Affected versions: 18.x → 19.0 upgrade path - Related: `deprecations.rb` entry `config_keys: %w(mattermost), deprecation: '19.0', removal: '19.0'`