Run spec/dot_gitlab_ci/rules_spec.rb when new files are added

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Problem Statement

Business Impact: Master branch failures block all development teams from merging changes, causing deployment delays and productivity loss across the entire engineering organization.

Root Cause: New files that don't trigger CI jobs bypass validation by spec/dot_gitlab_ci/rules_spec.rb, leading to post-merge master failures when these files aren't included in the no_matching_needed_files exception list.

Metrics & Evidence

  • Confirmed Incidents: At least 1 master breakage (incident #14611)
  • Detection Gap: MR !195468 added .devfiles/ that passed CI but broke master post-merge
  • Blast Radius: All ~2,000 GitLab engineers blocked from merging during master breakage
  • Resolution Time: Manual investigation required to identify missing file patterns and update exception lists
  • Frequency Risk: Any MR adding new file types at project root or in config directories can trigger this failure mode

Success Metrics

  1. Zero post-merge failures due to missing rules_spec.rb exceptions
  2. 100% pre-merge detection of files requiring exception list updates
  3. Reduced MTTR from manual discovery (hours) to automated detection (minutes)
  4. Improved developer velocity by eliminating master branch blocks

Solution

Automatically trigger spec/dot_gitlab_ci/rules_spec.rb when specific file patterns are added:

  • Root-level YAML files (*.yml, *.yaml)
  • Files in .devfiles/ directory
  • Other configuration files that historically bypass CI rules

This shifts detection from post-merge (blocking all teams) to pre-merge (blocking only the contributing MR).

Implementation Plan

Based on team discussion:

  1. Update tests.yml mapping to include patterns:

    # Root YAML files that often don't trigger CI
"*.yml": ["spec/dot_gitlab_ci/rules_spec.rb"]
"*.yaml": ["spec/dot_gitlab_ci/rules_spec.rb"]

# DevContainer and development files
".devfiles/**/*": ["spec/dot_gitlab_ci/rules_spec.rb"]

  2. Verify fail-fast job integration ensures the spec runs when these patterns match

  3. Fallback option: Run spec by default if pattern-based approach proves insufficient (trade-off: increased pipeline duration vs. reliability)

Technical Context

  • File: spec/dot_gitlab_ci/rules_spec.rb validates CI rule coverage
  • Current gap: New files bypass this validation, causing post-merge failures
  • Pipeline integration: Uses rspec fail-fast job with test-file-finder gem mapping
Edited by Mohga Gamea