Limit DAST artifacts parsed for the scanned resources CSV

What does this MR do and why?

Adds limits to the scanned resources CSV download (Projects::Security::ScannedResourcesController). The CSV is built in the web request by parsing the pipeline's DAST reports, and nothing limits how much it parses.

Behind the scanned_resources_csv_limits feature flag (disabled by default):

  • Size and count limits. Before downloading anything, the controller checks the pipeline's DAST artifacts, including child pipelines. If they total more than 20 MB or number more than 100, the request returns 422 with a message pointing to the DAST report artifact, without downloading or parsing anything.
  • Rate limit. scanned_resources_csv_download: 5 downloads per minute per user.

The limits are fixed values. They're explained in https://gitlab.com/gitlab-org/gitlab/-/work_items/637250#note_4166014191.

Why fixed values, not configurable limits: the CSV is downloaded about 3 times a day, and only one known customer has DAST reports over the size limit (and hasn't used the CSV in the last 30 days). Making the limits configurable would need a plan_limits migration, an application setting for the rate limit, and admin API changes, which seems too much for this fix. The rate limit is documented as a non-configurable rate limit.

Related to https://gitlab.com/gitlab-org/gitlab/-/work_items/637250

How to set up and validate locally

  1. Enable the flag: Feature.enable(:scanned_resources_csv_limits).

  2. Use a pipeline with a DAST report. Download the scanned resources CSV from the pipeline's Security tab. It downloads as normal.

  3. The limit checks the artifact's recorded size, so make the report look larger than 20 MB:

    Ci::JobArtifact.dast.last.update_column(:size, 21.megabytes)
  4. Download the CSV again. It returns the "too large" message.

  5. Download the CSV 6 times within a minute. The 6th returns 429.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Edited by Craig Smith

Merge request reports

Loading
Loading