Add new variable DAST_API_EXCLUDE_URLS which mirrors DAST_EXCLUDE_URLS

Problem

DAST_API_EXCLUDE_PATHS doesn't support commas which makes the on-demand switch over to DAST API could break some users.

Discussion: #350514 (comment 886408439)

Proposal

Review legacy variable (DAST_EXCLUDE_URLS) and verify compatibility. Support , for URL separation. If the environmental variable DAST_API_EXCLUDE_PATHS cannot be used, then create a new variable DAST_API_EXCLUDE_URLS using a comma (,) as a separator.

Remark

  • In DAST documentation uses it says DAST_EXCLUDE_URLS uses regular expression and DAST_API_EXCLUDE_PATHS uses globbing based on Minimatcher. This leads to the creation of DAST_API_EXCLUDE_URLS
  • DAST_EXCLUDE_URLS seems to be using java regex which is not totally compatible with dotnet regex. Basic regex are likely to be exchanged without rewriting.

  1. Worker-Entry

    1. Add new variable DAST_API_EXCLUDE_URLS
    2. Send new parameter thru RunnerOptions
    3. Add/Update tests (TDB if new tests can be unit test)
      1. Update py test to support new environmental variable
      2. Add test: excluding one Url
      3. Add test: excluding two Urls
      4. Add test: excluding two Urls using RegEx
      5. [-] Add e2e test

  2. Scanner

    1. Receive new paramert in RunnerOptions
    2. Process new Parameter in:
      1. Update computation for isOperationExcluded
        1. [-] Update FindRoute to also filter request matching against AbsoluteUri
        2. [-] Update MoveNextOperation to also filter request matching against AbsoluteUri

  3. Update Documentation

    1. Add or extent seciont to explain how to exclude URLs
    2. Add examples on how to exclude URL (specific, one or more and using basic regex)
Edited by Michael Eddington