We have an API for downloading artifacts on a **per job** basis. See: https://docs.gitlab.com/ee/api/job_artifacts.html ## Need There is an increasing need to allow for artifact downloads of a particular job type when it pertains to ~"devops::secure" jobs. In particular Fuzzing and SAST. Several issues have requested the need to have a single URL to hit that contains a file download for the entire pipeline's artifacts aggregated from all jobs that contain a certain `report_type`. ## Requirements - Pipeline API endpoint that allows for downloading a single file of all artifacts in a pipeline for a specific set of job that contain a user provided list of `report_types`. We have been using the `stage: 'fuzz'` field from `GET /projects/:id/jobs/:job_id/artifacts`, ``` stage: 'fuzz' ``` This is fragile. We should move over to using `report_type` in our new implementation. ## Endpoints to implement in graphQL: 1. Get list of all jobs in a pipeline that contain at least one of the report types defined in user provided "report_types" (Should only need pipeline ID and report_types and scope it by project id?) 1. GraphQL endpoint for downloading artifacts for individual jobs. (We only need a job ID to find this, can be scoped by project ID as well) **Not in scope for this issue** 1. GraphQl endpoint for downloading aggregated job artifacts in a pipeline that contain at least one of the report types defined in user provided "report_types" (Should only need pipeline ID and report_types and scope it by project id?) ## File Naming - For the job artifact and pipeline artifact downloads, we should improve the file naming. Right now its just `artifacts.zip` - For job artifacts maybe something like `<JOB_NAME>-<JOB_ID>.zip` - For pipeline artifacts maybe something like `<PIPELINE_NAME>-<PIPELINE_ID>.zip` with sub folders inside of the format `<JOB_NAME>-<JOB_ID>.zip` ## Support other report types other than just Secure - Attempt to make this work for all report types not just Security related report types. ---------- Example Screenshots: # Fuzzing Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/210343  Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/217151/  # SAST Epic: https://gitlab.com/groups/gitlab-org/-/epics/4388  # Technical details ## Backend The frontend needs the following GraphQL query: ```graphql { project(fullPath: "project-full-path") { pipeline(iid: "the-iid") { jobs(securityReportTypes: [FUZZING, SAST]) { nodes { artifacts(types: [ARCHIVE]) { nodes { downloadPath } } } } # reportsArchive(reportTypes: [FUZZING, SAST]) { <-- deferred to a future issue # downloadPath # } } } } ``` The following new GraphQL fields are needed: 1. `EE::Types::Ci::PipelineType.jobs` that takes a `securityReportTypes` argument and returns all jobs with a report key that matches. Valid report types are any value from `Security::SecurityJobsFinder.allowed_job_types`, which should be created as an enum type `JobArtifactSecurityReportTypesEnum` so the frontend knows what values are valid 2. `EE::Types::Ci::JobType.artifacts` that takes a `types` argument and returns artifacts of the given types. Valid values for the `types` argument are any value from `Ci::JobArtifact::TYPE_AND_FORMAT_PAIRS`, which should be created as an enum type `JobArtifactTypesEnum` so the frontend knows what values are valid 3. A new `EE::Types::Ci::JobArtifactType` that has a `downloadPath` field that returns the path needed to download the artifact and a `fileType` field that returns the file type of the download artifact 4. ~~`Types::Ci::PipelineType.reportsArchive` which takes a `reportTypes` filter and is a new `Types::Ci::ReportsArchive` type that has a `downloadPath`. That path will need to route to a new action that fetches the reports matching the given report types and zips them into an archive for download~~ (Deferred to a future issue) ### Notes * ~~Currently, `Security::JobsFinder` can be used to fetch jobs from a pipeline that have a given Secure report type. This should be merged into `Ci::JobsFinder` so that `Types::Ci::JobType.artifacts(reportType: [])` can be used by other stages~~ This plan has been abandoned. See https://gitlab.com/gitlab-org/gitlab/-/issues/251015#note_3ca75c06cd0c677f64caccfcfce88652026492a5 * Each of these fields should be resolved in such a way that no N+1 queries are created. [LooksAhead](https://docs.gitlab.com/ee/development/api_graphql_styleguide.html#look-ahead) might be helpful ## Frontend TBD