Job Artifact API endpoints do not work for report artifacts

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

This could probably be labeled a duplicate of #29118. I decided to create a separate issue (and narrow the scope) to specifically report artifacts (ex. sast).

Workaround

See https://docs.gitlab.com/api/job_artifacts/#downloading-artifactsreports-files

Problem to solve

Report artifacts are stored separately (not within the artifacts archive). This causes problems with any of the Job Artifacts API Endpoints that assume the artifacts archive.

Example Report Artifact (and example project here)

artifacts:
  reports:
    dotenv: env

There is an issue (and MR) for making the report artifacts visible via the UI, but the changes were never represented in the API.

The Job API Endpoint uses the build.artifacts_file

desc 'Download a specific file from artifacts archive' do
  detail 'This feature was introduced in GitLab 10.0'
end
...
route_setting :authentication, job_token_allowed: true
get ':id/jobs/:job_id/artifacts/*artifact_path', format: false do
...
  not_found! unless build.available_artifacts?
...
  send_artifacts_entry(build.artifacts_file, path)
end

But this artifacts file will never include the report objects.

# From build.rb
def available_artifacts?
  (!artifacts_expired? || pipeline.artifacts_locked?) && job_artifacts_archive&.exists?
end

def artifacts_file
  job_artifacts_archive&.file
end

User experience goal

I think we either need to mirror UI behavior with what is accessible via the API or provide a more generic endpoint to allow downloading of all of the artifact files.

Proposal

I don't see an obvious great way to change the current endpoints as they all look to the archive file (which I don't think we do at all(?) with the reports). So I'm suggesting another endpoint that allows general access to any of the job artifacts

# lib/api/ci/job_artifacts.rb
desc 'Download specific artifact from a job'
params do
  requires :job_id, type: Integer, desc: 'The ID of a job'
  requires :artifact_name, type: String, desc: 'Artifact Filename'
end
route_setting :authentication, job_token_allowed: true
get ':id/jobs/:job_id/artifact', urgency: :low do
  authorize_download_artifacts!

  build = find_build!(params[:job_id])
  authorize_read_job_artifacts!(build)

  artifact = build.job_artifacts.find_by file: params[:artifact_name]

  present_carrierwave_file!(artifact&.file)
end

This would allow access to both the archive and the report artifacts without causing breaking changes to the existing archive endpoints. (Probably need a similar endpoint for the pipeline endpoint). Example curl command to grab a sast artifact:

curl -X GET  -H "Private-Token: $TOKEN" "https://gitlab/api/v4/projects/2/jobs/31/artifact?artifact_name=gl-sast-report.json"

Internal Customer Ticket

Edited by Rutvik Shah