Log rules errors dropped with fully excluded stages

Why

Issue 606330. When a job's rules: raises a Clause::ParseError (bad rules:exists:project, bad rules:exists:ref, bad rules:changes:compare_to, or a regexp over its ReDoS timeout), the job is excluded and the error is recorded at the same time. If every job in a stage is excluded, the stage is filtered out before Seed::Pipeline#errors reads it, so the error is dropped and the pipeline is created green with a job and a whole stage missing. This has been broken since 15.3 (July 2022).

Surfacing these errors converts pipeline creations that succeed today into config_error failures. Some projects may have been living with a broken rule for three years, so this MR measures the blast radius before any behaviour changes.

The would_newly_fail field

It is true when the pipeline currently succeeds and would start failing once the errors surface. It cannot be derived from the other logged fields: a pipeline can carry a dropped error while already failing on an unrelated included stage, where surfacing the error changes the message but not the outcome. This is the number that decides how the behaviour change gets rolled out.

In Kibana, filter json.message: "rule errors dropped with fully excluded stage" and count json.extra.would_newly_fail: true.

Notes

What gets created is unchanged. Cost on the creation path is negligible because the relevant results are already memoised. The logged messages are already masked and already shown to users when the stage survives.

Reproduction on GitLab.com covering three non-regexp variants: https://gitlab.com/oyakovenko-ext/ci-rules-parse-error-swallowed-606330

Edited by Oleg Yakovenko

Merge request reports

Loading
Loading