Allow Badges to Reference Failed Pipelines
Problem to solve
The URL used to reference job artifacts only retrieves artifacts from the latest successful pipeline.
The results in a Catch-22 when the job is creating a artifact meant to be a badge indicating job failure. The "failure" badge can be created and uploaded as an artifact, but there is no mechanism in the Badge URL to reference it.
Further details
In my gitlab-ci.yml, I create a "pass/fail" badge using the following logic:
- Create a "failed" badge in the before_script - this is the default, assuming the rest of the job doesn't succeed
- Create a "passed" badge in the main script, after a successful check is completed
- Upload artifacts using the when:always option so the badge is uploaded regardless of the job's success
Sample:
Dependencies:
stage: test
image: python:3.7.3
before_script:
- pip install anybadge
- anybadge --label=dependencies --value=failed --file=badge.svg --color=red
script:
- pip install safety
- safety check -r requirements.txt
- anybadge --label=dependencies --value=passed --file=badge.svg --color=green --overwrite
tags:
- docker
only:
- master
artifacts:
when: always
paths:
- badge.svg
Then, in the Project's Badge Image URL, I reference the badge/artifact using https://gitlab.lfg.com/%{project_path}/-/jobs/artifacts/master/raw/badge.svg?job=Dependencies
This works great when the safety module succeeds (i.e. there are no vulnerabilities): the "failed" badge is created in the before_script, that badge is then overwritten with a "passed" badge after the safety module is finished. The pipeline succeeds, and the Badge URL grabs the uploaded artifact (badge.svg).
When the safety module finds an issue, the "fail" badge is created in the before_step, but job execution is halted when a problem is discovered so the "passed" badge is never created. Since there is an artifacts:when:always clause, the ("failed") badge.svg is uploaded even though the job failed.
That is all good!
But... the URL for the badge only references the last successful pipeline.
When the Dependencies job succeeds, the Badge URL references the badge.svg from that successful pipeline.
When the Dependencies job fails, and hence the pipeline as a whole fails, the Badge URL ignores that failed pipeline and references the badge.svg from the last succcessful pipeline instead. That effectively means nothing except the "passed" badge can ever be displayed.
Interestingly, I can reference the "failed" badge by explicitly using the job number in the artifact's URL, like so:
https://gitlab.lfg.com/%{project_path}/-/jobs/449737/artifacts/raw/badge.svg
But, there's no way to construct such a URL in the Badge URL where the job number would automatically be populated with the latest job number.
Proposal
Proposal 1. Change the behavior of the artifact URL to either a) automatically reference the latest pipeline regardless of its status, or b) add a optional parameter to the artifact URL that would allow for such behavior - for example, a status parameter that allowed values of "passed", "failed", or "all"
Proposal 2. Add support for a %{latest_job} variable in badge URLs, although I'm not sure to specify which job that job number would be for. Maybe something like %{latest_job} variable in combination with the job parameter, like:
https://gitlab.lfg.com/%{project_path}/-/jobs/%{latest_job}/artifacts/raw/badge.svg?job=Dependencies
or maybe a REST-style job name as part of the URL, like:
https://gitlab.lfg.com/%{project_path}/-/jobs/Dependencies/%{latest_job}/artifacts/raw/badge.svg
Documentation
https://docs.gitlab.com/ee/user/project/pipelines/job_artifacts.html
Testing
Proposal 1.A would change how the current artifacts URL works. Whether that's good or bad can be debated.
Proposal 1.B would mean current functionality doesn't change/break, while offering new functionality via new parms. Testing would just need to prove new functionality works.
Proposal 2 would also mean current functionality remains unchanged. Any change to the job artifact URL would be a new pattern, so testing would be limited to proving the new functionality.
What does success look like, and how can we measure that?
Success would be that a Badge URL referencing job artifacts would automatically display badges created for failed pipelines (but still uploaded due to the artifacts:when:always option).