Only generate CI artifact metadata for ZIP files

In the beginning, CI artifacts were resticted to ZIP files, but they have been expanded to be JSON files, tar.gz files, and more. Prior to this change, these gitlab-zip-metadata would be run for each of these non-ZIP artifacts, generating lots of noise in the logs.

We now inspect the artifact_format query string to determine whether to generate metadata. If the format is ZIP, which is the case for CI archives and LSIF files, we generate metadata. Blank formats are treated as ZIP as well.

Relates to #272 (closed)

Testing

To test this, I used a privileged Docker runner and enabled a Code Quality (https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html) job:

image: busybox:latest

include:
  - template: Code-Quality.gitlab-ci.yml

try:
  stage: test
  script: 
    - echo hello > test.txt
  artifacts:
    paths:
      - test.txt

Before

{"correlation_id":"PBFmV1KYeI5","level":"info","msg":"gitlab-zip-metadata error: zip archive format invalid, code: 10","time":"2020-10-10T19:43:21+05:30"}
{"client_mode":"local","copied_bytes":0,"correlation_id":"PBFmV1KYeI5","is_local":true,"is_multipart":false,"is_remote":false,"level":"info","local_temp_path":"/var/opt/gitlab/gitlab-rails/shared/artifacts/tmp/uploads","msg":"saved file","remote_id":"","temp_file_prefix":"metadata.gz","time":"2020-10-10T19:43:21+05:30"}
{"content_type":"application/json","correlation_id":"PBFmV1KYeI5","duration_ms":1465,"host":"host.example.com","level":"info","method":"POST","msg":"access","proto":"HTTP /1.1","referrer":"","remote_addr":"1.2.3.4:45352","remote_ip":"1.2.3.4","status":201,"system":"http","time":"2020-10-10T19:43:21+05:30","uri":"/gitlab/api/v4/jobs/86943/artifacts?artifact_format=raw\u0026artifact_type=codequality\u0026expire_in=1+week","user_agent":"gitlab-runner 13.0.0~beta.169.gdccb3f31 (master; go1.13.8; linux/amd64)","written_bytes":3}

After

{"client_mode":"local","copied_bytes":3,"correlation_id":"vreiVn4jU16","is_local":true,"is_multipart":false,"is_remote":false,"level":"info","local_temp_path":"/var/opt/gitlab/gitlab-rails/shared/artifacts/tmp/uploads","msg":"saved file","remote_id":"","temp_file_prefix":"gl-code-quality-report.json","time":"2020-10-10T20:09:52+05:30"}
{"content_type":"application/json","correlation_id":"vreiVn4jU16","duration_ms":792,"host":"host.example.com","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"1.2.3.4:53226","remote_ip":"1.2.3.4","status":201,"system":"http","time":"2020-10-10T20:09:53+05:30","uri":"/gitlab/api/v4/jobs/86944/artifacts?artifact_format=raw\u0026artifact_type=codequality\u0026expire_in=1+week","user_agent":"gitlab-runner 13.0.0~beta.169.gdccb3f31 (master; go1.13.8; linux/amd64)","written_bytes":3}

I verified the try job had the right trace and metadata:

image

There was no log noise after this change for the code_quality job.

image

Edited by Stan Hu

Merge request reports

Loading