## Context In the scope of the [logging field standardisation](https://gitlab.com/groups/gitlab-com/gl-infra/-/work_items/1432), ~"group::development tooling" identified logging fields that need to be renamed or removed. In order to avoid breaking changes for internal tools, but more importantly, self-managed and dedicated customers, we decided to create an admin panel setting to control the usage of the new logging field naming. This setting allows customers to adapt relevant observability instrumentation at their own pace, and GitLab.com to move forward with the logging field standardisation at the same time. ## Proposal: Explicit Version Pinning ### How it works The admin panel exposes a versioned setting that controls which set of logging field names the application emits. Each version is immutable — its field set is frozen at the time of release and never changes. Customers stay on a version until they explicitly decide to move. ``` logging_field_schema_version: v0 (default) | v1 | v2 | ... logging_field_dual_emit_target: none (default) | v1 | v2 | ... ``` `dual_emit_target` specifies a version strictly greater than `schema_version` to emit alongside the current version. The application emits the union of both field sets simultaneously. Setting `dual_emit_target` to `none` disables dual-emit. The UI enforces that `dual_emit_target` must be greater than `schema_version`, and disables or warns when `schema_version` is already on the latest available version. This replaces a simple boolean toggle with an explicit target, which gives customers full control over which version they are validating against — including the ability to skip intermediate versions entirely. ### Typical migration path A customer moving from v1 to v2 follows three steps at their own pace: 1. `v1, dual_emit_target: none` — stable, nothing changes 2. `v1, dual_emit_target: v2` — emitting v1 + v2 fields, customer adapts their observability pipelines 3. `v2, dual_emit_target: none` — committed to v2, dual-emit window closed A customer wanting to skip v2 and move directly from v1 to v3 follows the same pattern, just targeting v3 directly: 1. `v1, dual_emit_target: none` — stable on v1 2. `v1, dual_emit_target: v3` — emitting v1 + v3 fields (the union), customer adapts pipelines for all v3 changes at once 3. `v3, dual_emit_target: none` — committed to v3, v2 is a subset of v3 and therefore was already included ### Lifecycle and deprecation Older versions are retired over time and communicated as breaking changes through standard GitLab deprecation notices. The deprecation window gives customers a known horizon — they choose *when* to migrate within it, but not *whether* to migrate. This is the honest contract: migrate at your own pace, not indefinitely. ### Default behaviour The default is `v0, dual_emit: false`. An upgrade with no setting change produces identical log output to before. No customer is affected without taking explicit action. ### SSOT requirement The single source of truth (labkit-spec) must define, per version: - The exact set of field name mappings included in that version - The latest available version (so the UI can warn or disable `dual_emit_target` when `schema_version` is already on the latest) - A validation rule that `dual_emit_target` must always be strictly greater than `schema_version` ### Advantages - **Upgrading GitLab never silently changes log output.** A customer who has not touched their setting sees no change, ever. - **Full customer control.** Migration happens when the customer is ready, not when GitLab ships. - **Skipping versions is supported.** Because field sets are cumulative, a customer can jump from v1 to v3 without touching v2. - **Dual-emit is always meaningful.** It targets an explicit version chosen by the customer, from wherever they are pinned, not from v0. - **Auditable.** Any combination of version + dual-emit produces a fully predictable, documented field set. - **Deprecation is straightforward.** Old versions are retired with standard notice, the same as any other breaking change. ### Drawbacks - **Version numbers are customer-visible.** Customers need to understand that incrementing the version means accepting new field names in that version. - **Communication burden per release.** Every new version requires a clear changelog entry listing exactly which fields changed, so customers can assess whether it affects their setup before upgrading. - **SSOT maintenance.** Each frozen version must be permanently maintained until it is deprecated. This is manageable but not free. --- ## Alternative: Semantic Labels (`legacy` / `latest`) ### How it works Instead of version numbers, the setting exposes two semantic states: ``` logging_field_schema: legacy (default) | latest logging_field_dual_emit: false | true ``` `latest` always reflects the most current complete field set, regardless of how many internal iterations produced it. ### Advantages - Simplest possible API surface — no version numbers for customers to track. - Customers make one migration decision and are done. - Internal iterations are invisible to customers. ### Drawbacks - **`latest` is a moving target.** When a new iteration ships, any customer on `latest` silently receives new field name changes they may not have been aware of or tested for. This can break observability pipelines without the customer taking any action. - **No safe resting point between iterations.** A customer who has migrated to `latest` has no way to stay stable while a new iteration is in progress. - **False sense of control.** The setting implies stability it cannot deliver. ### Verdict The mutable nature of `latest` makes this approach unsuitable for customers with strict observability requirements. The risk of silent breakage on GitLab upgrades outweighs the simplicity benefit. --- ## Alternative: Semantic Labels with Per-Release Manifests ### How it works Keep `legacy` / `dual` / `latest` as labels, but define their meaning relative to the GitLab version installed. Each release ships a manifest of what fields `latest` and `dual` include at that version. ### Advantages - Friendly label names instead of version numbers. - Behaviour is at least documented per GitLab release. ### Drawbacks - **Customers don't upgrade GitLab lockstep with logging iterations.** A customer upgrading from 17.8 to 18.1 could skip multiple iterations — their `latest` jumps silently across all of them. - **Upgrade notes must cover every iteration in the range.** This creates a compounding communication burden that is easy to miss. - **More complexity, worse outcome than explicit versioning.** The manifest approach adds implementation overhead without solving the core problem of silent behaviour changes. ### Verdict More complex than both other approaches and still subject to silent breakage on upgrade. Not recommended. --- ## Summary Comparison | | Version Pinning (recommended) | Semantic `legacy`/`latest` | Semantic with Manifests | |---|---|---|---| | Silent breakage on upgrade | Never | Possible | Possible | | Customer migration control | Full | Limited | Partial | | Version skipping | Supported | N/A | N/A | | API surface complexity | Medium | Low | Medium | | SSOT maintenance | Per version | Per release | Per release | | Deprecation path | Standard breaking change | Unclear | Unclear | --- ## Questions - What is the planned deprecation window for old versions (e.g., two major releases)? - Should `dual_emit_target` set to the latest available version be a UI warning or a hard validation error when `schema_version` is already on the latest? - How will the changelog communicate which specific fields change in each version, and which teams are responsible for that documentation? - The change of the fields should be available in the changelogs of GitLab, but also as part of the UI, so that making a change to the setting is very clear for customers. - For nested logging field names, do we want to adapt the help overview to contain that information? - Example: | Standard field name | Also emitted as (deprecated) | |-----------------------|---------------------------------------------------------------------| | (**meta.**)gl_user_id | user_id, userid, extra.user_id, extra.current_user_id, meta.user_id | ## Related - [Logging field standardisation](https://gitlab.com/groups/gitlab-com/gl-infra/-/work_items/1432) - [Decision log](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/work_items/4349#note_3239668711) - [Logging fields to include in the setting]()