## Objective Investigate and implement a dynamic linting approach for enforcing field logging standards in the GitLab codebase, using runtime warnings and a custom TODO list instead of static RuboCop rules. ## Background As part of defining field logging standards for developers, we need a mechanism to identify and track unknown/non-standard logging fields across the codebase. Initial investigation ruled out RuboCop (static linting) due to the dynamic nature of logging field construction (see [example](https://gitlab.com/gitlab-org/gitlab/-/blob/1ad5ba489b574e33d4e7a57f211ad78e0cf94432/lib/bulk_imports/logger.rb#L46-52)). ## Approach **Dynamic linting via runtime inspection:** - Hook into LabKit JSON logger's `format_data` method to inspect logging field keys at runtime - Compare emitted fields against an allow list (standard fields + TODO list) - Emit warnings for unknown fields during development and CI **Configuration-based management:** - Maintain a YAML configuration file (`field_standard_config.yml`) with three categories: - `allow_list`: Approved standard fields (e.g., `correlation_id`, `gl_user_id`, `gl_user_name`) - `ignore`: System fields to ignore (e.g., `time`, `severity`, `message`) - `todo`: Fields currently in use but not yet standardized (temporary allowlist) ## Linting process ### Principles - Linting logic is part of labkit Ruby code along with the configuration of allowed fields - Configuration of the TODO list is part of each Ruby project - Linting is activated explicitly per project - Linting is only activated in "development" or "test" environments, never in "production" - Linting in "development" provides developer early feedback (shift-left) - Linting in CI prevents deprecated fields from sneaking back again - Linting is performed dynamically by intercepting logging code in `Labkit::Logging::JsonLogger` to validate field names - Static linting (via RuboCop) is tough because of the dynamic nature of constructing logged fields in Ruby ([example](https://gitlab.com/gitlab-org/gitlab/-/blob/7739b205715ea23c23bd4f04ae6a745f855eda6f/lib/bulk_imports/logger.rb#L46)) - Deprecated fields are fixed gradually across projects - Kibana acts as source for checking the use of deprecated keys - Linting will fail in CI (MR pipelines) if - Deprecated keys are detected - TODO entry for this very key is missing - Linting should be permissive in non-MR pipelines to accommodate linting gaps - For example, we run different set of jobs in master pipelines vs MR pipelines - Use Kibana [field aliasing](https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/field-alias) to route deprecated fields to their new form - Allow easy transition when renaming fields gradually ### Developer workflow Assuming that linting is enabled in a project, this is how the workflow to consolidate a field could look like: 1. Pick a field to consolidate - Manually, by impact, [for example](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/4067) - Automatically, by [field usage volume](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/4033#note_2916419307) 2. Deprecate fields and its variants in labkit repository (see [issue](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/4094)) - Provide the replacement. - `user_id` -> `gl_user_id` - Document common alternative field names that should also be deprecated - `userid`, `current_user_id`, ... 3. Create an MR in the project - Bump labkit version (which contains the deprecated definitions) - Extract TODO list from MR pipelines failing CI jobs - Add this TODO list to the project repo to make the CI pass again - The TODO list tracks the logger name, deprecated field, and call site (Ruby file) - Merge the MR - Any unexpected TODO list changes (additions or removals) result in an error and must be handled - If a call site is fixed, the corresponding TODO entry must be removed - If a call site emits a deprecated field, it will error, preventing new violations from sneaking in 4. Create a Kibana [field aliases](https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/field-alias) to alias all deprecated variants to the desired version 5. Gradually fix the TODO list - Rename deprecated field to use the new field - Verify in Kibana the impact 6. Cleanup, once all deprecated fields were renamed - Verify the TODO list is empty - Verify that Kibana logs only contain the new field - After the Kibana retention period (typically 28 days), remove the field alias - Adjust any Kibana links that filter using the deprecated field ## Implementation Details Reference MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/214212 The implementation: * [ ] Prepends a `Checker` module to `Labkit::Logging::JsonLogger` * [ ] Intercepts `format_message` to parse JSON output and extract field keys * [ ] Compares fields against the combined allow list * [ ] Warns once per unique set of unknown fields (deduplication via `Set`) * [ ] Only runs in non-production environments Example warning output: ```shell Logger Gitlab::Database::LoadBalancing::Logger emitted unknown fields: ["event", "model", "start_service_discovery"] Logger Gitlab::AuthLogger emitted unknown fields: ["gitlab_throttle_user_allowlist"] ``` ## Next Steps 1. **Compile comprehensive TODO list**: Run CI pipelines and parse warning output to identify all existing non-standard fields 2. **Categorize fields**: Review discovered fields and determine which should be: - Standardized and moved to `allow_list` - Deprecated and removed from code - Temporarily kept in `todo` list 3. **Create remediation plan**: For each TODO field, create issues/MRs to either standardize or remove 4. **Monitor and iterate**: Track progress on reducing TODO list over time ## Related Resources - Parent Epic: https://gitlab.com/groups/gitlab-org/quality/-/work_items/235 - Implementation MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/214212 - Labkit JSON Logger: https://gitlab.com/gitlab-org/ruby/gems/labkit-ruby/-/blob/master/lib/labkit/logging/json_logger.rb - Labkit Fields: https://gitlab.com/gitlab-org/ruby/gems/labkit-ruby/-/blob/master/lib/labkit/fields.rb