ci: require tests with feat and fix Go changes

Summary

A new Danger rule, tests_with_code, fails a feat or fix merge request that changes a production Go package without changing a _test.go file in that package's directory. It reads the type from the title's Conventional Commits prefix, after removing the draft prefixes that lint:mr-title also strips. Every other type is exempt, refactor and perf included.

These are not production, so the rule skips them:

  • Generated files: Go's // Code generated ... DO NOT EDIT. marker before the package clause, read from the checkout.
  • Test code: testdata/, e2e/, internal/testutil/, and the testsuites, mocks and <name>test packages under internal/. That is the line the coverage gate in !3341 (merged) draws.
  • internal/storage/stub/, an in-memory BlobStore fake that only tests import. !3341 (merged) can leave it out because the stub's own test covers its lines, but this rule would ask a feat MR that updates the fake for a _test.go change beside it.

The rule calls fail, and danger-review is no longer allowed to fail, so a failure blocks the merge. The other rules still call warn, and a warning never fails the job: Danger exits non-zero only when a rule called fail (exit(1) if fail_on_errors && ran_status in danger 9.6.1).

Caveat: anything that fails danger-review now blocks the merge

Until this MR, danger-review was allow_failure. From now on, more than this rule can block a merge:

  • A rule that raises. Danger re-raises an exception from any rule, so a bug in stream_label or runway_workload_secret blocks every MR it hits.
  • A job that cannot run. Causes include the Vault token, the GitLab API, rubygems during bundle install, or a stale Gemfile.lock. The component's retry: covers runner and API failures, not a failing script.
  • Merge trains. A red danger-review drops the MR from the train.

These have been rare:

  • danger-review is not among the last 800 failed jobs in this project, which run from 2026-10-11 to 2026-10-15.
  • It passed in every finished MR pipeline sampled: 12 here and 14 on the security mirror.
  • None of the last 200 merged MRs came from a fork, where the job would have no Vault token.

Other changes

  • job_allow_failure: false is set explicitly rather than left to the component's default, so a component bump cannot quietly turn the gate off.
  • conventions.md, the Gemfile header, the renovate.json git-pin rule and the interruptible comment in .gitlab-ci.yml said danger-review is allow_failure. They now say it blocks, or no longer make the claim.
  • The harness now stubs fail and the MR title, and a new check_fail_cases helper counts failures. Every case also fails if the rule reports the other kind, so an advisory rule that starts calling fail turns its own suite red.

Noted, not changed

  • !1667 (closed) (draft) also edits danger/test/helper.rb, the root Dangerfile and conventions.md, so whichever MR lands second needs a rebase. Its db_review_evidence rule only warns, so this MR does not change its behaviour.
  • !3359 (merged) bumps the same danger component include, on the line next to this MR's change.
  • The milestone lookup in conventions.md (## MR assignee and milestone) returns null today. It reads only the first page of milestones, which no longer includes the current one, so it needs --paginate.

Governing ADRs

None. ADR-023 puts shared test helpers in internal/testutil/, which the rule treats as test code. No ADR covers Danger or CI gating. scripts/adr-freshness.sh reports the mirror current.

Testing

  • sh scripts/danger-rules-test.sh: all 60 cases pass, 30 of them for the new rule.
  • I broke the rule 24 ways, one change at a time: each exemption, the title parse, the generated check, the same-directory check, and fail → warn. I also changed stream_label's warn to fail. Each change turns the suite red.
  • The pre-commit hooks pass on the changed files, and the CI config lints valid through the GitLab API.
  • This MR's own pipeline passes with danger-review blocking.
  • Replay: I ran the rule through the harness over the last 105 merged feat and fix MRs, merged since 2026-09-29. It would have blocked 9 of them (!3316 (merged), !3315 (merged), !3187 (merged), !3048 (merged), !3047 (merged), !3150 (merged), !2332 (merged), !3075 (merged), !3078 (merged)). Each changed a package's production Go without changing that package's tests. The first replay also flagged internal/storage/stub on !3315 (merged) and !3316 (merged), which is why the stub now counts as test code.

End to end on !3361 (closed)

!3361 (closed) is stacked on this branch and stays open as evidence; it will not be merged. It changes internal/pagebound/pagebound.go and nothing else, and was run three ways:

Run Title Change danger-review Pipeline
1 feat: pagebound.go only failed, not allowed to fail failed
2 refactor: pagebound.go only passed passed
3 feat: pagebound.go and pagebound_test.go passed passed

Run 2 is a new pipeline rather than a retry of run 1, so run 1 stays failed. docs:links fails in all three on four dead links that fail it on main too; it is allowed to fail.

Danger's comment on !3361 (closed) after run 1 (Danger edits its comment on each run, so this one now shows no error):

danger-tests-with-code-error

Closes https://gitlab.com/gitlab-org/ops/artifact-registry/-/work_items/1613

🤖 Generated with Claude Code

Edited by Fiona McCawley

Merge request reports

Loading
Loading