DAST AJAX Spider functionality crashes scan
Summary
Running DAST using an AJAX scan crashes while attempting to start the Firefox browser. URLs that require Javascript will not be scanned. A report is produced at the end of the scan, which contains no vulnerabilities from pages that normally would be scanned.
Steps to reproduce
Run a DAST scan using the -j
option. For example: docker run -ti --rm registry.gitlab.com/gitlab-org/security-products/dast:v1.6.0 -j -t www.yourwebsite.com
Example Project
DAST's end to end tests capture this case, and indeed this is where it was first reported on the 12th December 2019.
Failing Job, Failing test zap log file
What is the current bug behavior?
Vulnerabilities may be reported as fixed because the page is not crawled in a browser.
What is the expected correct behavior?
Pages should be crawled, hence vulnerabilities will be reported. The ZAP log file should not show an exception related to starting Firefox.
Relevant logs and/or screenshots
ZAP log containing Firefox exception:
[zap.out] 47134 [ZAP-AjaxSpiderApi] WARN org.zaproxy.zap.extension.spiderAjax.SpiderThread - Failed to start browser firefox-headless
[zap.out] com.google.inject.ProvisionException: Guice provision errors:
[zap.out]
[zap.out] 1) Error in custom provider, org.openqa.selenium.WebDriverException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:32639 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
[zap.out] Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
[zap.out] System info: host: 'adf0ae63ca2a', ip: '172.19.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '4.19.78-coreos', java.version: '1.8.0_222'
[zap.out] Driver info: driver.version: FirefoxDriver
[zap.out] at com.crawljax.di.ConfigurationModule.configure(ConfigurationModule.java:47)
[zap.out] while locating com.crawljax.browser.EmbeddedBrowser
[zap.out] for parameter 0 at com.crawljax.core.CrawlerContext.<init>(CrawlerContext.java:33)
[zap.out] while locating com.crawljax.core.CrawlerContext
[zap.out] for parameter 0 at com.crawljax.core.Crawler.<init>(Crawler.java:75)
[zap.out] while locating com.crawljax.core.Crawler
[zap.out] for parameter 2 at com.crawljax.core.CrawlTaskConsumer.<init>(CrawlTaskConsumer.java:30)
[zap.out] while locating com.crawljax.core.CrawlTaskConsumer
[zap.out]
[zap.out] 1 error
[zap.out] at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)
[zap.out] at com.crawljax.core.CrawlController.call(CrawlController.java:66)
[zap.out] at com.crawljax.core.CrawljaxRunner.call(CrawljaxRunner.java:37)
[zap.out] at org.zaproxy.zap.extension.spiderAjax.SpiderThread.run(SpiderThread.java:257)
[zap.out] at java.lang.Thread.run(Thread.java:748)
[zap.out] Caused by: org.openqa.selenium.WebDriverException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:32639 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
[zap.out] Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
[zap.out] System info: host: 'adf0ae63ca2a', ip: '172.19.0.3', os.name: 'Linux', os.arch: 'amd64', os.version: '4.19.78-coreos', java.version: '1.8.0_222'
[zap.out] Driver info: driver.version: FirefoxDriver
[zap.out] at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:92)
[zap.out] at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:600)
[zap.out] at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:219)
[zap.out] at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:142)
[zap.out] at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:120)
[zap.out] at org.zaproxy.zap.extension.selenium.ExtensionSelenium.getWebDriverImpl(ExtensionSelenium.java:777)
[zap.out] at org.zaproxy.zap.extension.selenium.ExtensionSelenium.getWebDriver(ExtensionSelenium.java:700)
[zap.out] at org.zaproxy.zap.extension.selenium.internal.BuiltInSingleWebDriverProvider.getWebDriver(BuiltInSingleWebDriverProvider.java:62)
[zap.out] at org.zaproxy.zap.extension.selenium.ExtensionSelenium.getWebDriverImpl(ExtensionSelenium.java:639)
[zap.out] at org.zaproxy.zap.extension.selenium.ExtensionSelenium.getWebDriver(ExtensionSelenium.java:509)
[zap.out] at org.zaproxy.zap.extension.spiderAjax.SpiderThread$AjaxSpiderBrowserBuilder.get(SpiderThread.java:480)
[zap.out] at org.zaproxy.zap.extension.spiderAjax.SpiderThread$AjaxSpiderBrowserBuilder.get(SpiderThread.java:448)
[zap.out] at com.google.inject.util.Providers$3.get(Providers.java:109)
[zap.out] at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
[zap.out] at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)
[zap.out] at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)
[zap.out] at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:84)
[zap.out] at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
[zap.out] at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)
[zap.out] at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)
[zap.out] at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:84)
[zap.out] at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
[zap.out] at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)
[zap.out] at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)
[zap.out] at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:84)
[zap.out] at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
[zap.out] at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
[zap.out] at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)
[zap.out] at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
[zap.out] ... 4 more
[zap.out] Caused by: org.apache.http.conn.HttpHostConnectException: Connect to localhost:32639 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
[zap.out] at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159)
[zap.out] at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
[zap.out] at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
[zap.out] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
[zap.out] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[zap.out] at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[zap.out] at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[zap.out] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[zap.out] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
[zap.out] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[zap.out] at org.openqa.selenium.remote.internal.ApacheHttpClient.fallBackExecute(ApacheHttpClient.java:138)
[zap.out] at org.openqa.selenium.remote.internal.ApacheHttpClient.execute(ApacheHttpClient.java:86)
[zap.out] at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:101)
[zap.out] at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:73)
[zap.out] at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:142)
[zap.out] at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
[zap.out] ... 32 more
Results of environment info
Verified on DAST v1.6.0
. This likely is also present in other DAST versions.
Possible fixes
This is likely related to a recent update of Firefox. Hopefully we can fix this ourselves, worst case we may require an update from the ZAP SpiderAjax plugin.