RuboCop rules with external dependencies can cause master broken

Problem

Some RuboCop cops have external dependencies like db/docs/background_migrations, spec/migrations/*_spec.rb, or factory definitions. When these dependencies change but the offending file is not modified, the offense is missed in MR pipelines because RuboCop only runs on changed files. However, in master pipelines, RuboCop runs on every Ruby file, which then triggers the offense and breaks the build.

Example Scenario

  1. A migration file db/migrate/20221014034338_populate_releases_access_level_from_repository.rb has a RuboCop offense that requires a spec file
  2. The spec file spec/migrations/populate_releases_access_level_from_repository_spec.rb is deleted in an MR
  3. In the MR pipeline: RuboCop only checks changed files (the spec deletion), so it doesn't check the migration file → offense is missed ✓ MR passes
  4. In the master pipeline: RuboCop checks all Ruby files including the migration → offense is detected → master breaks ✗

Related Issues

  • #413965 (closed) - "Deleting migration specs cause RuboCop offenses and broken master" - Proposes using tff (test file finder) to map deleted spec files to their corresponding migrations
  • #521435 - "Rubocop rules that span multiple files might introduce broken master" - Documents that rules like EnsureFactoryForTable can interact across files
  • #482856 (closed) - "Don't disable RSpec/DuplicateSpecLocation inline" - Shows how inline disables mask cross-file dependency issues

Affected RuboCop Rules

Rules known to have external dependencies:

  • Migration/UpdateColumnInBatches - requires spec files at spec/migrations/*_spec.rb
  • Migration/EnsureFactoryForTable - depends on factory definitions
  • RSpec/DuplicateSpecLocation - depends on EE extension specs
  • Any rule that validates presence/absence of related files

Root Cause

The optimization to run RuboCop only on changed files in MR pipelines breaks the assumption that RuboCop rules are self-contained. When a rule's dependencies change but the file being checked doesn't, the dependency change is invisible to the linter.

Proposed Solutions

  1. Map external dependencies (from #413965 (closed)):

    • Create a migrations.yml mapping for tff to map deleted spec files to migrations
    • Extend RUBOCOP_TARGET_FILES to include related files when dependencies change

  2. Run full RuboCop in tier 3 pipelines (from #521435):

    • For critical pipelines, run RuboCop on all files instead of just changed files
    • Ensures cross-file rule violations are caught before master

  3. Document cross-file dependencies:

    • Maintain a registry of which RuboCop rules have external dependencies
    • Provide guidance for rule authors to avoid cross-file dependencies

Impact

This is an active, recurring problem. The linked master-broken incident (#19949 (closed)) shows 30+ related incidents triggered on 2025-12-10 to 2025-12-11, all caused by RuboCop failures cascading through the pipeline.

See gitlab-org/quality/engineering-productivity/master-broken-incidents#19949 (comment 2946410381)

Edited by 🤖 GitLab Bot 🤖