Pre scan verification mode for API Security

The On Demand API Scan feature allows running DAST API scans directly from the GitLab UX. When performing such a scan a user creates site profile with a number of configuration options that includes authentication, API specification document, and target URL. To improve the user experience, a new site validation feature is being added. This will allow users to quickly check if their configuration works without running a scan.

To support this work, API Security needs the ability to perform a verification process and report back success or failure to an internal API.

Proposal

Add a new mode of operation to API Security, site profile verification. This mode of operation will validate the following:

  1. Connection - API Security is able to connect to the provided target URL
  2. Authentication - API Security is able to authenticate to the provided target URL
  3. Crawling - API Security is able to load the API specification and call 3 operations

One method for performing this validation is to add a new mode of operation in which API Security will only perform a record phase, but exit record after N operations have been recorded.

Additionally, it must be possible to map errors encountered back to the three validation checks being performed.

Requirements

  1. Support both DAST API and API Fuzzing modes of operation.
  2. Perform just the record phase with exit after N operations.
    1. Do not generate security report.
  3. Identify if connection errors have occurred.
  4. Identify if authentication errors have occurred.
    1. Status code of operations indicates authentication issue.
    2. Overrides script fails in some way.
  5. Identify is crawling errors have occurred.
    1. Unable to load API specification.
      1. For URLs, non-success status code.
      2. For files, file not found.
      3. API specification failed validation (in relaxed mode for OpenAPI).
      4. No operations identified.
  6. Map errors back to one of: Connection, Authentication, or Crawling.
    1. On error include an actionable error message.
  7. Submit validation results to new internal API (TBD).
    1. The format of the API call is being discussed in #366456
  8. A new template Security/DAST-Prescan-Validation-API-Scan.yml. This template will run DAST API in the site profile validation mode.
    1. The job name is validation.
    2. The stage name is dast.
    3. Same artifact collection rules as DAST API template.
    4. No report to generate or submit.
    5. No rules are needed
Edited by Michael Eddington