Schedule master pipeline notifications aren't sent when `package-and-qa` fails
Problem
We get notifications in Slack when there's a failure in scheduled master tests, and don't send notifications when the tests pass. We assume that no notification means that everything is fine.
This is a problem when the pipeline itself fails.
Currently, the pipeline is failing to post a comment. E.g., https://gitlab.com/gitlab-org/gitlab/-/jobs/455436734
Triggered downstream pipeline: https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/pipelines/122345225
Waiting for downstream pipeline status
Ignoring the following error: Server responded with code 404, message: 404 Not Found. Request URI: https://gitlab.com:443https://gitlab.com/api/v4/projects/gitlab%2Dorg%2Fgitlab/repository/commits/comments
(There's a possibly related issue with building images (e.g., Trigger:gitlab-docker
), which should be resolved by !26272 (merged))
This is serious because we had a ~P1 ~S1 regression that the tests would probably have caught, if they had been running. #208501 (closed)
Proposal
- Fix the request URI so that comments can be made
- Enable notifications for passing pipelines so that we at least get an indirect indicator of a problem when the Slack notifications suddenly go quiet.
/cc @zeffmorgan @tmslvnkc re. pipeline triage
/cc @gl-quality/eng-prod