Add an API endpoint to download artifacts with CI_PIPELINE_ID
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
Problem to solve
While a pipeline is currently running, it is impossible to get an URL for an artifact created by a previous job (of this pipeline) without the corresponding job id.
Further details
According to the documentation, the artifacts you can download with the following routes :
https://example.com/<namespace>/<project>/-/jobs/artifacts/<ref>/download?job=<job_name>https://example.com/<namespace>/<project>/-/jobs/artifacts/<ref>/raw/<path_to_file>?job=<job_name>
are only updated once the pipeline is finished, so they are not usable for a subsequent job inside the same pipeline.
Now, having a look at the API documentation, there is a way to download an artifact with the corresponding job id, but there is apparently nothing using anything constant and designing uniquely this pipeline (ie pipeline id or commit sha)
The example use case I have is here, basically I have a 3-step CI process :
- A first CI job builds a static website, and makes an artifact containing it
- A second CI job
sshs into the server hosting this website, and executes there a script (with the first job's job_id as argument) - The (remote) script downloads the previously built artifact, and replaces the current site with its content
The issue is that there is no easy way for the second job to get the job_id of the first one. The only solution I found was to store it in the artifact, download the artifact during the second job, and read it. Quite cumbersome, and if the artifact were to get big, well, that would be pretty ineffective.
Obviously, there are many ways to work around this specific case (like directly scping the new content to the remote server, etc…), but this is just an example, I suppose we can easily find similar use cases that make more sense.
Proposal
My proposal is to implement endpoints similar to these and/or these, but at the pipeline scale. They would look like that :
GET /projects/:id/pipelines/:pipeline_id/artifacts?job=name
I suppose the same could be done with commits and something such as :
GET /projects/:id/repository/commits/:sha/artifacts?job=name
I'm not sure however about how strong is the link between commits and pipelines, so I'd stick with the first proposal.
A completely different approach might be to allow transmitting data (like variables) between jobs (as proposed in https://gitlab.com/gitlab-org/gitlab-ce/issues/47517)
What does success look like, and how can we measure that?
From a CI job, there is an easy and efficient way to get an URL for artifacts of the same pipeline.