Only generate CI artifact metadata for ZIP files
In the beginning, CI artifacts were resticted to ZIP files, but they have been expanded to be JSON files, tar.gz files, and more. Prior to this change, these gitlab-zip-metadata would be run for each of these non-ZIP artifacts, generating lots of noise in the logs.
We now inspect the artifact_format query string to determine whether
to generate metadata. If the format is ZIP, which is the case for CI
archives and LSIF files, we generate metadata. Blank formats are treated as ZIP as well.
Relates to #272 (closed)
Testing
To test this, I used a privileged Docker runner and enabled a Code Quality (https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html) job:
image: busybox:latest
include:
- template: Code-Quality.gitlab-ci.yml
try:
stage: test
script:
- echo hello > test.txt
artifacts:
paths:
- test.txt
Before
{"correlation_id":"PBFmV1KYeI5","level":"info","msg":"gitlab-zip-metadata error: zip archive format invalid, code: 10","time":"2020-10-10T19:43:21+05:30"}
{"client_mode":"local","copied_bytes":0,"correlation_id":"PBFmV1KYeI5","is_local":true,"is_multipart":false,"is_remote":false,"level":"info","local_temp_path":"/var/opt/gitlab/gitlab-rails/shared/artifacts/tmp/uploads","msg":"saved file","remote_id":"","temp_file_prefix":"metadata.gz","time":"2020-10-10T19:43:21+05:30"}
{"content_type":"application/json","correlation_id":"PBFmV1KYeI5","duration_ms":1465,"host":"host.example.com","level":"info","method":"POST","msg":"access","proto":"HTTP /1.1","referrer":"","remote_addr":"1.2.3.4:45352","remote_ip":"1.2.3.4","status":201,"system":"http","time":"2020-10-10T19:43:21+05:30","uri":"/gitlab/api/v4/jobs/86943/artifacts?artifact_format=raw\u0026artifact_type=codequality\u0026expire_in=1+week","user_agent":"gitlab-runner 13.0.0~beta.169.gdccb3f31 (master; go1.13.8; linux/amd64)","written_bytes":3}
After
{"client_mode":"local","copied_bytes":3,"correlation_id":"vreiVn4jU16","is_local":true,"is_multipart":false,"is_remote":false,"level":"info","local_temp_path":"/var/opt/gitlab/gitlab-rails/shared/artifacts/tmp/uploads","msg":"saved file","remote_id":"","temp_file_prefix":"gl-code-quality-report.json","time":"2020-10-10T20:09:52+05:30"}
{"content_type":"application/json","correlation_id":"vreiVn4jU16","duration_ms":792,"host":"host.example.com","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"1.2.3.4:53226","remote_ip":"1.2.3.4","status":201,"system":"http","time":"2020-10-10T20:09:53+05:30","uri":"/gitlab/api/v4/jobs/86944/artifacts?artifact_format=raw\u0026artifact_type=codequality\u0026expire_in=1+week","user_agent":"gitlab-runner 13.0.0~beta.169.gdccb3f31 (master; go1.13.8; linux/amd64)","written_bytes":3}
I verified the try job had the right trace and metadata:
There was no log noise after this change for the code_quality job.

