Make DAST configurable for speed, coverage, and other use cases

Problem to solve

Our DAST feature is based on ZAProxy, and especially zap-baseline, a CLI to interact easily with ZAP. One observed concern from customers is that zap-baseline provides a time based spidering function. It spiders for URLs for one minute and then scans those URLs in order to be efficient and for our purposes in order to not delay the time it takes to run a DAST scan in a Merge Request pipeline. It does mean that in some cases only a fraction of the surface area of an application is tested. Over-riding this time is possible by passing a specific parameter to zap-baseline.

However, it's currently impossible to pass parameters to zap-baseline that would adjust this and other variables with our recommended vendored template.

Users can always use a manual job definition, but it's against this recommendation.

Intended users

DAST users (Developers like Sasha) who want to customize the behavior of the DAST scan compared to what GitLab provides by default.

Further details

By default, zap-baseline will run for 1 minute. This parameter can't be set when using the template.

Edit (Dennis): Also other useful parameters cannot be passed to ZAP using the template, such as -j for instructing ZAP to use the AJAX spider. Full list of params is here https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan

Proposal

We should evaluate if we want to expose:

• all params at once (ex: DAST_ZAP_PARAMS)
  • Allow this configuration for both full scan AND passive scan
• all params one by one (https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan#usage)
• Only the number of minutes to spider (ex: DAST_SPIDER_LIMIT)

Documentation

We currently don't document that spidering will stop after 1 minute (/cc @axil). This can be very confusing for users, as the complete will succeed, but the results will be incomplete, without any clue about missing ones.

Testing

We should document how to change the spider time limit once we decide how we expose it. Note that a Full Scan doesn't have this limit, but it will achieve active scanning instead of the passive one from zap-baseline.

What does success look like, and how can we measure that?

Users can specify the spider time limit.

What is the type of buyer?

GitLab Ultimate

Implementation plan

For the following environment variables,

1. Map command line arguments to their environment variable in configuration.py using the mapping defined in the comment below.
2. Ensure they are unit tested.
3. Document their usage.

Environment variables

• DAST_ZAP_CONFIG_FILE
• DAST_ZAP_CONFIG_URL
• DAST_ZAP_GENERATE_CONFIG
• DAST_SPIDER_MINS
• DAST_HTML_REPORT
• DAST_MARKDOWN_REPORT
• DAST_XML_REPORT
• DAST_INCLUDE_ALPHA_VULNERABILITIES
• DAST_USE_AJAX_SPIDER
• DAST_ZAP_CLI_OPTIONS
• DAST_DEBUG
• DAST_ZAP_DELAY_PASSIVE_SCAN_SECONDS