DAST_EXCLUDE_URLS affects URLs from non-matching hosts in BBD
The DAST_EXCLUDE_URLS variable specifies URLs or URL patterns that should be excluded from the scan. If the crawler attempts to send a request to a matching URL, the request is blocked.
However in Browser-Based DAST, URLs that only match the path portion of an excluded URL - but not the host portion - will also be excluded. This is even true when the path is empty.
For example:
include:
- template: DAST.gitlab-ci.yml
dast:
variables:
DAST_WEBSITE: "https://my.site.com/"
DAST_EXCLUDE_URLS: "https://other.site.com/"
will result in job failure, with the following entries in the log file:
TRC SCOPE path matches excluded URL expression, URL is excluded from scope host="my.site.com" relative_path="/" matched_excluded_uri="^/"
WRN BROWS excluding URL from scope, please add host to scope allowed hosts or excluded hosts request_id="*" url="https://my.site.com/"
...
DBG CONTA blocked request to host excluded from scope index="0" requestID="*" url="https://my.site.com/"
This example shows an empty path, but the same happens if the path is non-empty, or contains a pattern. The recommendation "please add host to scope allowed hosts or excluded hosts" also does not work; adding the host to the allowed hosts still results in the URL being excluded.
The same is not true of Proxy-Based DAST, where the entire URL must match the given URL/pattern, including the host portion.