Show arbitrary build results
Problem to Solve
It is unrealistic for GitLab CI to natively support every such build result. What is a "build result"? Examples might include code coverage results, test results, output from static analysis tools, generated logs, pages with shortcut links, and more that provide context to contributors about the state of their contribution and/or the overall project.
- Delaney (Development Team Lead)
- Sasha (Software Developer)
- Devon (DevOps Engineer)
- Sam (Security Analyst)
This is a request to allow arbitrary build results / links to be included in the merge request page. To achieve this we will add a new option to the
artifacts: element that will track these items as being ones to highlight:
custom_output: script: generator.sh >output.txt artifacts: expose_as: "Generator Output" paths: - output.txt
Then, for any jobs that produced outputs where this new element is provided, a new section is added to the latest pipeline section in the MR widget here:
If there are multiple files in the artifacts it links you to the job folder in the artifact browser.
This new section will contain a direct link (using the provided name as the name of the link) to the artifact browser. There, any formats supported by the artifact browser (as implemented via https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14399) would be displayed in a new tab, or downloaded if not supported. In this way generated .html or other artifacts can be coordinated and easily linked to. The generated .html could contain important content itself, or be a further list of manually generated links to any other resources.
One complaint here has been having to manually download .html artifacts, but using the artifact browser in combination with Pages as described at https://docs.gitlab.com/ee/user/project/pipelines/job_artifacts.html makes viewing .html in a new tab but without a download possible.
test: script: [ 'echo 1' ] artifacts: expose_as: 'see file 1' paths: ['path/to/file1.html']
This will show a link "see file 1" pointing directly to the file content inside the artifacts browser.
test: script: [ 'echo 1' ] artifacts: expose_as: 'see artifacts' paths: ['path/to/file1.txt', 'path/to/file2.txt'] # equivalent: # paths: ['path/to/directory/'] # paths: ['path/to/directory/*.png']
This will show a link "see artifacts" pointing to the job folder of the artifacts browser (all artifacts for the given job).
Artifacts - Allow artifacts to be linked to as build results, contolled by
- Example with HTML files:
- Tools like Robot Framework can generate nicely-formatted HTML files; this happens during the build
- The HTML output is collected as a build artifact
"Test Results"link appears next to the rest of the build results, which links directly to the generated HTML file
- Example with snippets:
- A custom tool would output a small snippet of HTML
In the future, we can improve this by supporting external things:
There are a few different kinds of external or artifact results that could be implemented here:
External - Allow external services to report a result to GitLab CI via the web API. Example:
- A build pushes his code coverage files to a service like codecov.io
- Codecov.io processes the coverage files and calculates a single coverage metric
- Codecov.io pushes this result to GitLab CI, via the web API
- GitLab CI shows this result in the side-bar of the build page, including a link back to the codecov.io page for that build
The solution adds an expandable section to the pipeline section in the merge request widget. It can be featured together with the deployment information but will, in that case, be positioned above it.
It features a counter for the number of exposed artifacts and if expanded will display a table view of exposed artifacts. This table view currently displays the artifact string name, the URL towards the source file, and the job name + link which generated it.
- Artifact links will open up a new tab
- Job links will link directly
All configured exposed artifacts will be visible in a single list. The maximum will be
10 and if more artifacts are exposed the .GitLab-ci.yml will be invalidated.
Permissions and Security
We will need to document usage of this feature.
What does success look like, and how can we measure that?
We will measure success of this feature by capturing clicks on the arbitrary items from the list.