Skip to content

Support multiple artifacts generation per job

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Release notes

TBD

Problem to solve

A CI job definition with multiple "artifacts" definition generates no error or warning, even from ci/lint, but only the last artifact is taken into account. This causes issues not only with the available artifact paths, but also with the artifacts that appear in the download list. The user cannot download all artifacts from that job.

  • How should existing "Download" button on Job page behave? Should this download a zip containing all of the archives?
  • How should existing "Browse" button on Job page behave? I'm aware there is follow up to browse archives, but until then, what should users expect to see when they click "Browse"?
  • How should the existing Job Artifacts APIs change?
    • GET /projects/:id/jobs/:job_id/artifacts
    • GET /projects/:id/jobs/artifacts/:ref_name/download?job=name
    • DELETE /projects/:id/jobs/:job_id/artifacts
    • If we change any of the content of the zip files sent from these APIs, we need to be careful in order not to break existing API clients.
  • Will additional archives be served as a dependencies to Runner? Currently, we serve a single artifacts archive
  • Downstream jobs need to be able to download specific archives from a job, instead of having to download all of the artifacts.

Intended users

User experience goal

Improve the experience of downloading multiple artifacts from a job, and make the experience consistent in the API.

Proposal

TBD

Further details

This description used to be using the bug template, so I copied over the expected behavior and proposal that was added.

  • At least a warning should be displayed, CI/Lint should notify of duplicate keys. But better would be to allow for multiple artifacts definitions since keywords like expose_as are at the artifacts: level, and you may want to allow for creating multiple artifact sets.

Proposal The core of the problem here is that we currently do support only one artifacts archive per build. It means that we can have only one definition, and that GitLab Runner will create only one artifacts archive that it is going to upload to GitLab. Note that despite that we support only one archive, it is possible to have multiple artifacts reports:

artifacts:
  untracked: true
  paths:
    - some/path
    - another/path
  reports:
    junit: rspec.xml
    terraform: tfplan.json

This will create three artifacts - one ZIP archive with (some/path/** + another/path/** + untracked files) and two artifacts with reports.

We could add support for multiple archives, for example you might want to upload coverage HTML report and some build objects or .exe files. In order to change this we can use, archives:

artifacts:
  archives:
    - name: coverage
      untracked: true
      paths: # ...
      expose_as: ...
    - name: binaries
      untracked: true
      paths: # ...
      expose_as: ...
  reports:
    junit: rspec.xml
    terraform: tfplan.json

In this format, a hash of archives would be provided that each would support all the artifact parameters (https://docs.gitlab.com/ee/ci/yaml/#artifacts) except reports, since that's just another way to define a pre-set archive. This requires though that we redesign how we expose artifacts definition to a runner and how we handle uploads. This might be easy or not (hard to tell without investigation), but this is not merely a syntax problem.

Permissions and Security

Documentation

Availability & Testing

Available Tier

Feature Usage Metrics

What does success look like, and how can we measure that?

What is the type of buyer?

Is this a cross-stage feature?

Links / references

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.

Edited by 🤖 GitLab Bot 🤖