Problem running ajax crawl in DAST
Summary
When attempting to run a DAST scan with the ajax crawler the crawler fails to run with the following error:
1589525210998 geckodriver INFO Listening on 127.0.0.1:8381
1589525211651 mozrunner::runner INFO Running command: "/opt/firefox/firefox" "-marionette" "-headless" "-profile" "/tmp/rust_mozprofile.tFihgWbaR1LY"
*** You are running in headless mode.
1589525212448 Marionette INFO Enabled via --marionette
1589525213974 Marionette INFO Listening on port 43491
1589525214074 Marionette WARN TLS certificate errors will be ignored for this session
1589525233728 addons.productaddons ERROR Request failed certificate checks: [Exception... "Certificate checks failed. See previous errors for details." nsresult: "0x80070057 (NS_ERROR_ILLEGAL_VALUE)" location: "JS frame :: resource://gre/modules/CertUtils.jsm :: validateCert :: line 113" data: no]
[Child 525, Chrome_ChildThread] WARNING: pipe error (15): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 353
[Child 525, Chrome_ChildThread] WARNING: pipe error (3): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 353
The scan will still complete, but the ajax spider fails to run.
Another test project's jobs also failed:
https://gitlab.com/gitlab-org/security-products/benchmark-suite/dvwa-dast/-/jobs/553757948#L2308
https://gitlab.com/gitlab-org/security-products/benchmark-suite/dvwa-dast/-/jobs/553757946#L2295
Steps to reproduce
Create a gitlab-ci.yml with the following:
stages:
- dast
include:
- template: DAST.gitlab-ci.yml
variables:
DOCKER_DRIVER: overlay2
DAST_WEBSITE: http://vulnapp/
DAST_AUTH_URL: http://vulnapp/login.php
DAST_USERNAME: admin
DAST_PASSWORD: password
DAST_USERNAME_FIELD: username # the name of username field at the sign-in HTML form
DAST_PASSWORD_FIELD: password # the name of password field at the sign-in HTML form
DAST_AUTH_EXCLUDE_URLS: http://vulnapp/logout.php,http://vulnapp/setup.php,http://vulnapp/security.php
CI_DEBUG_TRACE: "true"
dast:
stage: dast
services:
- name: registry.gitlab.com/gitlab-org/security-products/benchmark-suite/dvwa-source-truth:latest
alias: vulnapp
variables:
DAST_ZAP_USE_AJAX_SPIDER: "true"
GIT_STRATEGY: fetch
artifacts:
name: "dast-baseline-ajax"
paths: [gl-dast-report.json]
expire_in: 1 week
Example Project
https://gitlab.com/gitlab-org/security-products/benchmark-suite/dvwa-dast-test
Also:
https://gitlab.com/gitlab-org/security-products/benchmark-suite/dvwa-dast
What is the current bug behavior?
The ajax spider extension for DAST fails to execute due to a Firefox error.
What is the expected correct behavior?
The ajax spider extension should successfully load Firefox and the geckodriver
Relevant logs and/or screenshots
See above
Output of checks
This bug happens on GitLab.com
Results of GitLab environment info
Running on GitLab.com shared-runners
Possible fixes
Not sure if these are related:
https://github.com/mozilla/geckodriver/issues/1295
https://support.mozilla.org/en-US/questions/1271306#answer-1260814