Allow the user to exclude a parameter for API Fuzzing

Problem

A user wants to exclude a parameter from testing. Parameter could be any supported (header, path, querystring, etc.). This can happen when a field always causes an API failure, or a finding that will not be fixed.

Proposal

Add the following two variables:

• DAST_API_EXCLUDE_PARAMETER_ENV={json} User can provide the json as a string
• DAST_API_EXCLUDE_PARAMETER_FILE=exlude_parameters.json User can provide the json as url/file
• Error if both are set

Use a json format similar to overrides to allow users to define parameters to skip during testing:

{
  "headers": [
    "header1",
    "header2"
  ],
  "cookies": [
    "cookie1",
    "cookie2"
  ],
  "query": [
    "query-string1",
    "query-string2"
  ],
  "body-form": [
    "form-param1",
    "form-param2"
  ],
  "body-json": [
    "json-path1",
    "json-path2"
  ],
  "body-xml" : [
    "xpath1",
    "xpath2"
  ]
}

Add a section to the worker-entry output to list excluded parameters.

Tasks

1. Worker-Entry
   1. Add new variables (_EXCLUDE_PARAMETER_ENV and _EXCLUDE_PARAMETER_FILE)
   2. Error when the two are variables are set at the same time
   3. If a file is given, then check it can be loaded and it JSON. If not throw proper error (file can use BasePath)
   4. If the content is given, check JSON can be loaded. If not then throw proper error
   5. Use RunnerOptions to pass over the scanner the exclude parameter config.
   6. Add a new section in the worker-entry final report. Listing the excluded parameters
2. Scanner/Runner
   1. Load exclude parameter config. If a file path is used, then it can use BasePath settings to locate the file.
   2. Exclude parameters TDB
3. Tests
   1. Add integration test in worker-entry to check error when the two variables are set
   2. Add integration test in worker-entry to check when file does not exist or cannot be downloaded
   3. Add integration test in worker-entry to check when the file content is not a valid JSON
   4. Add integration test in worker-entry excluding each type of parameter.
      1. Matching (for XML take into account custom namespaces)
      2. Non-Matching
      3. Invalid expression (e.g. incorrect JSON-Path or XPath expressions)
4. Update documentation
   1. Add new variables
   2. Provide explanation of expected use case
   3. Provide example

Links

• [internal] Dogfooding conversation https://gitlab.slack.com/archives/CUTDH1V8R/p1607348957372100