CI linter: Extend errors messaging to include warnings
Problem to solve
As we start deprecating CI features like extending only/except in shared templates gitlab-org/release/tasks#1315 (closed) it would be helpful to have a linter that helps users update their CI before it breaks. In order for this to be the most useful it should run on every pipeline and display them somewhere prominently.
Intended users
Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/
User experience goal
Proposal
For this MVC:
- Extend the existing red banner for errors to handle errors and also warning.
- Add warnings for these scenarios:
- Using
rules
withoutworkflow: rules
; link to the workflow:rules docs Usingrules
andonly/except
in the same job is not supported
(expand here for part of original proposal that is scoped to issue #219431 (closed))
Store each offenses, one per line, as part of YAML errors
, each identified with a severity, such as:
- error:
- warning:
- deprecation:
The format of a single line message could be:
<severity>: <job-name>: <message>
We could use this yaml_errors
to also further refine the Build View and show this information there
Today, we would do these things:
- we just refine this box to present it differently on frontend based on severity
- we would need to add a small change to
Ci::Config
to allow easy way to generatewarning
anddeprecations
, join them together and persist as part ofci_pipelines.yaml_errors
, including a context in above format
Execution plan
- Replace
has_yaml_errors?
withconfig_error?
so that it won't useyaml_errors
which will hold a different type of messages with this feature - Add mechanism inside
YamlProcessor
that allows to append warning messages as we do today with errors. Warning messages could be used to also describe deprecation warnings. We could distinguish them later.- Maybe we could extend
Gitlab::Config::Entry::Configurable
to have helper methods such asadd_warning
- Maybe we could extend
- Store multiple messages in
Ci::Pipeline#yaml_errors
. Ensure thatpipeline.errors
don't include warnings or deprecations. - Expose warnings in the frontend and API response for the Linter / Pipeline view - this marks the 1st iteration, from user perspective
In preparation to work on this maybe we need to first open a MR about replacing the use of has_yaml_errors?
with config_error?
. Then, we can make yaml_errors
orthogonal to failure_reason
as we may not have any failure reasons but warnings in yaml_errors
.
Further details
Permissions and Security
Documentation
Availability & Testing
- Unit test changes - Yes
- Ensure renaming does not affect existing behaviors - Error messages should still be displayed
- Warnings message should show up where there is violation even if pipeline did not fail
- Integration test changes - Yes
- Integration test API ~> UI:
- Warnings and deprecation messages are recognized and displayed accordingly
- In case pipeline did fail with a config error and there are also warning violation, show both error and warning in the banner.
- Integration test API ~> UI:
- End-to-end test change - not required
See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning