notify-pipeline-failure can silently fail
As seen in https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283094:
$ if [[ "${CREATE_INCIDENT_FOR_PIPELINE_FAILURE}" == "true" ]]; then # collapsed multi-line command
Created incident "https://gitlab.com/gitlab-org/quality/engineering-productivity/master-broken-incidents/-/issues/282"
$ scripts/generate-failed-pipeline-slack-message.rb -i ${BROKEN_MASTER_INCIDENT_JSON} -f ${FAILED_PIPELINE_SLACK_MESSAGE_FILE}; # collapsed multi-line command
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6483 0 14 100 6469 112 51752 --:--:-- --:--:-- --:--:-- 51864
invalid_blocksI opened !105534 (merged) to work around the issue but we should solve the invalid_blocks in the first place:
{
"channel": "master-broken",
"username": "Failed pipeline reporter",
"icon_emoji": ":boom:",
"text": "*<https://gitlab.com/gitlab-org/gitlab|gitlab-org/gitlab> pipeline <https://gitlab.com/gitlab-org/gitlab/-/pipelines/708709380|#708709380> failed*",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*<https://gitlab.com/gitlab-org/gitlab|gitlab-org/gitlab> pipeline <https://gitlab.com/gitlab-org/gitlab/-/pipelines/708709380|#708709380> failed*"
},
"accessory": {
"type": "button",
"text": {
"type": "plain_text",
"text": "View incident #282"
},
"url": "https://gitlab.com/gitlab-org/quality/engineering-productivity/master-broken-incidents/-/issues/282"
}
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*Branch*: <https://gitlab.com/gitlab-org/gitlab/-/commits/master|`master`>"
}
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*Commit*: <https://gitlab.com/gitlab-org/gitlab/-/commit/b254f573d248a845083da71f993e675ef0c154ad|Merge branch 'cngo-organise-sidebar-code' into 'master'>"
}
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*Triggered by* <https://gitlab.com/ekigbo|Ezekiel Kigbo> • *Source:* `push` • *Duration:* 42.93 minutes"
}
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*Failed jobs (59):* <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283087|rspec-ee system pg12 10/10>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283086|rspec-ee system pg12 9/10>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283085|rspec-ee system pg12 8/10>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283084|rspec-ee system pg12 7/10>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283083|rspec-ee system pg12 6/10>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283080|rspec-ee system pg12 4/10>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283079|rspec-ee system pg12 3/10>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283078|rspec-ee system pg12 2/10>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283073|rspec-ee integration pg12 6/6>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283071|rspec-ee integration pg12 5/6>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283069|rspec-ee integration pg12 4/6>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283068|rspec-ee integration pg12 3/6>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283067|rspec-ee integration pg12 2/6>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283065|rspec-ee integration pg12 1/6>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283064|rspec-ee unit pg12 18/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283063|rspec-ee unit pg12 17/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283062|rspec-ee unit pg12 16/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283061|rspec-ee unit pg12 15/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283060|rspec-ee unit pg12 14/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283059|rspec-ee unit pg12 13/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283058|rspec-ee unit pg12 12/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283057|rspec-ee unit pg12 11/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283056|rspec-ee unit pg12 10/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283055|rspec-ee unit pg12 9/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283054|rspec-ee unit pg12 8/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283053|rspec-ee unit pg12 7/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283052|rspec-ee unit pg12 6/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283051|rspec-ee unit pg12 5/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283050|rspec-ee unit pg12 4/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283049|rspec-ee unit pg12 3/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283048|rspec-ee unit pg12 2/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283047|rspec-ee unit pg12 1/18>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283045|rspec system pg12 28/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283042|rspec system pg12 25/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283041|rspec system pg12 24/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283038|rspec system pg12 22/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283030|rspec system pg12 17/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283028|rspec system pg12 15/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283027|rspec system pg12 14/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283025|rspec system pg12 12/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283023|rspec system pg12 10/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283022|rspec system pg12 9/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283021|rspec system pg12 8/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283015|rspec system pg12 2/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283013|rspec integration pg12 12/12>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283009|rspec integration pg12 8/12>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283005|rspec integration pg12 4/12>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283004|rspec integration pg12 3/12>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398283001|rspec unit pg12 28/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398282977|rspec unit pg12 4/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398282976|rspec unit pg12 3/28>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398282972|memory-on-boot>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398282940|rspec-all frontend_fixture 5/5>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398282939|rspec-all frontend_fixture 4/5>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398282938|rspec-all frontend_fixture 3/5>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398282937|rspec-all frontend_fixture 2/5>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398282936|rspec-all frontend_fixture 1/5>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398282927|compile-production-assets>, <https://gitlab.com/gitlab-org/gitlab/-/jobs/3398282926|cache-assets:production>"
}
}
]
}I think the problem is that the block containing the list of failed jobs has too many characters:
Edited by Rémy Coutable