Draft: Implement field validator for logging standardization

Doug Barrettrequested to merge
dbarrett/field_validator into master

Summary

Adds runtime field validation to detect deprecated logging field names during RSpec tests. Supports the Observability Field Standardisation initiative.

How It Works

  • Auto-injects during RSpec - No application or rspec code changes needed.
  • Frozen baseline approach - Existing violations tracked in labkit.yml are ignored.
  • Fails on new violations - Prevents regression while allowing incremental cleanup.
  • Migration-aware - Highlights when logs contain both old and new fields.

Key Components

  • Labkit::Fields::Deprecated - Maps deprecated fields to standard replacements
  • Labkit::Logging::FieldValidator - Runtime validator that prepends to JsonLogger
  • labkit.yml - Version-controlled baseline of known violations

Screenshot_2025-12-11_at_23.27.14

How to validate locally

  1. Checkout this branch in your local labkit-ruby project.
  2. Modify monolith Gemfile to use your local labkit-ruby: gem 'gitlab-labkit', path: '../labkit-ruby', feature_category: :shared
  3. Run a spec that calls loggers with deprecated fields.
  4. See the generated report.
Example spec used for development 
# frozen_string_literal: true

require 'spec_helper'

RSpec.describe 'LabKit Field Validator Comprehensive Test', feature_category: :shared do
  describe 'user authentication logging with Gitlab::AppLogger' do
    it 'logs user login with deprecated user_id' do
      Gitlab::AppLogger.info(
        message: 'User logged in',
        user_id: 123,
        action: 'login',
        ip_address: '127.0.0.1'
      )
    end

    it 'logs user details with deprecated username' do
      Gitlab::AppLogger.info(
        message: 'User details accessed',
        username: 'john_doe',
        email: 'john@example.com'
      )
    end

    it 'logs with both old and new user fields' do
      Gitlab::AppLogger.info(
        message: 'Migrating user data',
        user_id: 456,
        gl_user_id: 456,
        action: 'migrate'
      )
    end
  end

  describe 'nested field violations with Gitlab::AppLogger' do
    it 'logs with extra.user_id' do
      Gitlab::AppLogger.info(
        message: 'Extra user context',
        'extra.user_id' => 789,
        action: 'context'
      )
    end

    it 'logs with meta.user_id' do
      Gitlab::AppLogger.info(
        message: 'Meta information',
        'meta.user_id' => 321,
        request_id: 'abc123'
      )
    end

    it 'logs with extra.current_user_id' do
      Gitlab::AppLogger.info(
        message: 'Current user tracking',
        'extra.current_user_id' => 654,
        controller: 'UsersController'
      )
    end

    it 'logs with extra.user' do
      Gitlab::AppLogger.info(
        message: 'User object reference',
        'extra.user' => 111,
        method: 'GET'
      )
    end

    it 'logs with meta.user' do
      Gitlab::AppLogger.info(
        message: 'User metadata',
        'meta.user' => 222,
        path: '/api/users'
      )
    end
  end

  describe 'correlation id variations with Gitlab::AppLogger' do
    it 'logs with deprecated tags.correlation_id' do
      Gitlab::AppLogger.info(
        message: 'Request tracking',
        'tags.correlation_id' => 'old-correlation-123',
        duration_ms: 150
      )
    end

    it 'logs with standard correlation_id' do
      Gitlab::AppLogger.info(
        message: 'Standard tracking',
        Labkit::Fields::CORRELATION_ID => 'new-correlation-456',
        duration_ms: 200
      )
    end
  end

  describe 'mixed scenarios with Gitlab::AppLogger' do
    it 'logs multiple deprecated fields together' do
      Gitlab::AppLogger.info(
        message: 'Complex operation',
        user_id: 111,
        username: 'jane_doe',
        'extra.user_id' => 111,
        action: 'complex'
      )
    end

    it 'logs deprecated fields across multiple calls from same location' do
      3.times do |i|
        Gitlab::AppLogger.info(
          message: "Operation #{i}",
          user_id: 100 + i,
          operation_number: i
        )
      end
    end

    it 'logs with deprecated and standard fields mixed' do
      Gitlab::AppLogger.info(
        message: 'Transition state',
        user_id: 999,
        gl_user_id: 999,
        username: 'transition_user',
        gl_user_name: 'transition_user',
        status: 'migrating'
      )
    end
  end

  describe 'different logger types' do
    context 'with Gitlab::JsonLogger' do
      let(:gitlab_logger) { Gitlab::JsonLogger.new('/dev/null') }

      it 'logs with deprecated username' do
        gitlab_logger.info(
          message: 'JSON log event',
          username: 'json_user',
          severity: 'info'
        )
      end

      it 'logs with deprecated user_id' do
        gitlab_logger.info(
          message: 'User action',
          user_id: 333,
          action: 'test'
        )
      end
    end

    context 'with Gitlab::AuthLogger' do
      it 'logs authentication with deprecated fields' do
        Gitlab::AuthLogger.info(
          message: 'Authentication attempt',
          username: 'auth_user',
          user_id: 444,
          ip: '192.168.1.1'
        )
      end
    end
  end

  describe 'edge cases' do
    it 'logs with only deprecated fields, no standard fields' do
      Gitlab::AppLogger.info(
        user_id: 888,
        username: 'old_style_user'
      )
    end

    it 'logs with deprecated field as symbol key' do
      Gitlab::AppLogger.info(
        message: 'Symbol key test',
        user_id: 555
      )
    end

    it 'logs with deprecated field as string key' do
      Gitlab::AppLogger.info(
        'message' => 'String key test',
        'user_id' => 666
      )
    end

    it 'logs with all user-related deprecated fields' do
      Gitlab::AppLogger.info(
        message: 'All deprecated user fields',
        user_id: 100,
        'extra.user_id' => 101,
        'extra.user' => 102,
        'meta.user_id' => 103,
        'meta.user' => 104,
        'extra.current_user_id' => 105,
        username: 'all_fields_user'
      )
    end
  end

  describe 'real-world scenarios' do
    it 'logs authentication event' do
      Gitlab::AuthLogger.info(
        message: 'Authentication successful',
        user_id: 1,
        username: 'admin',
        method: 'standard',
        ip: '127.0.0.1'
      )
    end

    it 'logs with correlation tracking' do
      Gitlab::AppLogger.info(
        message: 'Request processed',
        'tags.correlation_id' => 'old-style-correlation',
        user_id: 4,
        duration_ms: 250
      )
    end
  end
end

To generate a labkit.yml in the project repository, pass the environment variable:

LABKIT_FIELD_VALIDATOR_REGENERATE=true bundle exec rspec <spec file>

Note: this will remove all violations and replace with violations found in the suite that was run. During development, this should only be used for a full test suite, ideally in CI.

`labkit.yml` generated from the above RSpec 
# LabKit Logging Field Violations Baseline
# This file tracks known technical debt. New logs must use standard fields from Labkit::Fields.
# AUTO-GENERATED FILE. DO NOT EDIT MANUALLY.
#
# === HOW TO FIX RSPEC FAILURES ===
#
# Option A: Fix the code (Recommended)
#   Replace the deprecated field in your log call with the standard equivalent.
#   Ensure relevant dashboards and runbooks are updated accordingly.
#   Note: You must remove the deprecated field entirely. Merely adding the new field is insufficient.
#
# Option B: Catalogue the violation (If fixing is not immediately possible)
#   Run: LABKIT_FIELD_VALIDATOR_UPDATE_YAML=true bundle exec rspec <your_spec_file>
#   Note: You must justify this technical debt in your Merge Request.
#
# === MAINTENANCE ===
#
# To regenerate the entire baseline (e.g., in CI or full suite runs):
#   Run: LABKIT_FIELD_VALIDATOR_REGENERATE=true bundle exec rspec
#
# === DEFINITION OF A VIOLATION ===
#
# A "violation" is a unique combination of: [Logger Class] + [File Path] + [Deprecated Field]
#   - Multiple log calls in the same file using the same field count as 1 violation.
#   - The violation exists until the deprecated field is entirely removed from the file.

---
field_violations:
- standard_field: correlation_id
  total_violations: 1
  violations:
  - logger_class: Gitlab::AppJsonLogger
    callsite: lib/gitlab/multi_destination_logger.rb
    deprecated_field: tags.correlation_id
- standard_field: gl_user_id
  total_violations: 7
  violations:
  - logger_class: Gitlab::AuthLogger
    callsite: lib/gitlab/json_logger.rb
    deprecated_field: user_id
  - logger_class: Gitlab::AppJsonLogger
    callsite: lib/gitlab/multi_destination_logger.rb
    deprecated_field: extra.current_user_id
  - logger_class: Gitlab::AppJsonLogger
    callsite: lib/gitlab/multi_destination_logger.rb
    deprecated_field: extra.user
  - logger_class: Gitlab::AppJsonLogger
    callsite: lib/gitlab/multi_destination_logger.rb
    deprecated_field: extra.user_id
  - logger_class: Gitlab::AppJsonLogger
    callsite: lib/gitlab/multi_destination_logger.rb
    deprecated_field: meta.user
  - logger_class: Gitlab::AppJsonLogger
    callsite: lib/gitlab/multi_destination_logger.rb
    deprecated_field: meta.user_id
  - logger_class: Gitlab::AppJsonLogger
    callsite: lib/gitlab/multi_destination_logger.rb
    deprecated_field: user_id
- standard_field: gl_user_name
  total_violations: 2
  violations:
  - logger_class: Gitlab::AuthLogger
    callsite: lib/gitlab/json_logger.rb
    deprecated_field: username
  - logger_class: Gitlab::AppJsonLogger
    callsite: lib/gitlab/multi_destination_logger.rb
    deprecated_field: username

After this, running bundle exec rspec <spec file> should not raise any violations, as they have been recorded in the baseline.

If a developer needs to add a violation to the baseline, they can do so during a partial spec run by passing the environment variable:

LABKIT_FIELD_VALIDATOR_UPDATE_YAML=true bundle exec rspec <rspec file>

This will add any new found violations to the baseline and increment the totals by the number of added violations. Developers must justify the added tech debt in any Merge Requests.

Closes: gitlab-org/quality/quality-engineering/team-tasks#4094

Related:

Edited by Doug Barrett

Merge request reports