DAST passive scan rule takes significant amount of time
Summary
While scanning nextcloud scan times run excessively long. This appears to be due to a poorly performing regex (html_mailto)
nextcloud returns very large javascript bodies which this regex is run against for every response.
Steps to reproduce
docker pull nextcloud:19-fpm-alpine
docker run -e SQLITE_DATABASE=nextcloud -e NEXTCLOUD_ADMIN_USER=admin -e NEXTCLOUD_ADMIN_PASSWORD=XXX -e NEXTCLOUD_TRUSTED_DOMAINS=vulnapp -d -p 8080:80 nextcloud
...
export img=`docker ps | grep nextcloud | awk -F " " '{print $NF}'`
docker run -ti --rm --link ${img}:vulnapp -v "$PWD/session":/home/zap/.ZAP_D/session -v $(pwd)/wrk:/zap/wrk:rw -v "$PWD":/output dast /analyze -t "http://vulnapp/" -j --full-scan true --auth-url "http://vulnapp/login" --auth-username-field="user" --auth-password-field="password" --auth-submit-field="submit-form" --auth-username="admin" --auth-password="XXX" -d --auth-exclude-urls "http://vulnapp/logout"
What is the current bug behavior?
I am still tracking down the passive scan plugin(?) that loads this xml and scans the response bodies, but it's called from: https://github.com/zaproxy/zaproxy/blob/main/zap/src/main/java/org/zaproxy/zap/extension/pscan/PassiveScanThread.java#L227
with the error being reported here: https://github.com/zaproxy/zaproxy/blob/main/zap/src/main/java/org/zaproxy/zap/extension/pscan/PassiveScanThread.java#L251
What is the expected correct behavior?
It does not warn that the passive scan takes too long, because the passive scans should not take long.
Relevant logs and/or screenshots
2431457 [ZAP-PassiveScanner] WARN org.zaproxy.zap.extension.pscan.PassiveScanThread - Passive Scan rule html_mailto took 8 seconds to scan http://vulnapp/apps/workflowengine/js/workflowengine.js?v=www.google.com%3A80%2F application/javascript 1367286
2439939 [ZAP-PassiveScanner] WARN org.zaproxy.zap.extension.pscan.PassiveScanThread - Passive Scan rule html_mailto took 8 seconds to scan http://vulnapp/apps/workflowengine/js/workflowengine.js?v=www.google.com application/javascript 1367286
2448187 [ZAP-PassiveScanner] WARN org.zaproxy.zap.extension.pscan.PassiveScanThread - Passive Scan rule html_mailto took 8 seconds to scan http://vulnapp/apps/workflowengine/js/workflowengine.js?v=www.google.com%2Fsearch%3Fq%3DOWASP%2520ZAP application/javascript 1367286
2456654 [ZAP-PassiveScanner] WARN org.zaproxy.zap.extension.pscan.PassiveScanThread - Passive Scan rule html_mailto took 8 seconds to scan http://vulnapp/apps/workflowengine/js/workflowengine.js?v=www.google.com%3A80%2Fsearch%3Fq%3DOWASP%2520ZAP application/javascript 1367286
2468073 [ZAP-PassiveScanner] WARN org.zaproxy.zap.extension.pscan.PassiveScanThread - Passive Scan rule html_mailto took 8 seconds to scan http://vulnapp/core/js/dist/main.js?v=http%3A%2F%2Fwww.google.com%2F application/javascript 1443963
2477345 [ZAP-PassiveScanner] WARN org.zaproxy.zap.extension.pscan.PassiveScanThread - Passive Scan rule html_mailto took 8 seconds to scan http://vulnapp/core/js/dist/main.js?v=http%3A%2F%2Fwww.google.com%3A80%2F application/javascript 1443963
Output of checks
This bug happens on GitLab.com
Possible fixes
I've opened an upstream bug report: https://github.com/zaproxy/zaproxy/issues/6509
Work around is to disable passive scan tags:
-z"-config pscans.autoTagScanners.scanner(0).name=html_mailto -config pscans.autoTagScanners.scanner(0).type=TAG -config pscans.autoTagScanners.scanner(0).config=MailTo -config pscans.autoTagScanners.scanner(0).enabled=false -config pscans.autoTagScanners.scanner(1).name=html_tag_form -config pscans.autoTagScanners.scanner(1).type=TAG -config pscans.autoTagScanners.scanner(1).config=Form -config pscans.autoTagScanners.scanner(1).enabled=false -config pscans.autoTagScanners.scanner(2).name=html_tag_password -config pscans.autoTagScanners.scanner(2).type=TAG -config pscans.autoTagScanners.scanner(2).config=Password -config pscans.autoTagScanners.scanner(2).enabled=false -config pscans.autoTagScanners.scanner(3).name=html_type_hidden -config pscans.autoTagScanners.scanner(3).type=TAG -config pscans.autoTagScanners.scanner(3).config=Hidden -config pscans.autoTagScanners.scanner(3).enabled=false -config pscans.autoTagScanners.scanner(4).name=html_type_upload -config pscans.autoTagScanners.scanner(4).type=TAG -config pscans.autoTagScanners.scanner(4).config=Upload -config pscans.autoTagScanners.scanner(4).enabled=false -config pscans.autoTagScanners.scanner(5).name=html_tag_object -config pscans.autoTagScanners.scanner(5).type=TAG -config pscans.autoTagScanners.scanner(5).config=Object -config pscans.autoTagScanners.scanner(5).enabled=false -config pscans.autoTagScanners.scanner(6).name=html_tag_script -config pscans.autoTagScanners.scanner(6).type=TAG -config pscans.autoTagScanners.scanner(6).config=Script -config pscans.autoTagScanners.scanner(6).enabled=false -config pscans.autoTagScanners.scanner(7).name=html_setcookie -config pscans.autoTagScanners.scanner(7).type=TAG -config pscans.autoTagScanners.scanner(7).config=SetCookie -config pscans.autoTagScanners.scanner(7).enabled=false -config pscans.autoTagScanners.scanner(8).name=html_comment1 -config pscans.autoTagScanners.scanner(8).type=TAG -config pscans.autoTagScanners.scanner(8).config=Comment -config pscans.autoTagScanners.scanner(8).enabled=false -config pscans.autoTagScanners.scanner(9).name=html_comment2 -config pscans.autoTagScanners.scanner(9).type=TAG -config pscans.autoTagScanners.scanner(9).config=Comment -config pscans.autoTagScanners.scanner(9).enabled=false -config pscans.autoTagScanners.scanner(10).name=response_json -config pscans.autoTagScanners.scanner(10).type=TAG -config pscans.autoTagScanners.scanner(10).config=JSON -config pscans.autoTagScanners.scanner(10).enabled=false"