User can configure DAST to wait for a loading modal to disappear before crawling the page
Problem
Web Applications that have Loading modal dialogs in between page transitions pose a problem for DAST browser-based scans.
Proposal
When a DAST browser-based scan transitions to a new page, a series of checks are run to determine that the page is loaded. These include whether or not the document has updated, if there are any pending requests, timeouts, etc.
This MR proposes that a new CI/CD environment variable DAST_BROWSER_PAGE_LOADING_SELECTOR
be made available to the user. When the user enters a selector of the login modal, the DAST browser-based crawler should check to ensure the element is not found or is hidden before classifying a page transition as complete.
More details
The issue arises as the DAST crawler assumes the page has finished loading when it is still showing "Loading". This can lead to:
- Missed elements in the scan that will not be crawled.
- Elements that are hidden, therefore skipped by the crawler.
Implementation Plan
-
Add PageLoadingSelector
to browser-based scanner -
When checking whether a navigation is stable, a check should be made to the browser to determine if the loading selector is present. If it is, the page should not be considered stable, even if DAST has waited a domStableAfter
amount of time. -
If the page is still not stable after max(one minute, domStableAfter), abort the scan with a message saying that the loading page is still displayed. -
Upgrade DAST and add DAST_BROWSER_PAGE_LOADING_SELECTOR
-
Document DAST_BROWSER_PAGE_LOADING_SELECTOR
in GitLab DAST browser-based analyzer documentation -
Add an end-to-end test using a fixture that has a loading modal dialog. Elements should be hidden behind the loading modal until loading is complete.
Reference
This issue has come up related to a customer issue, https://gitlab.com/gitlab-org/gitlab/-/issues/382778.
This has been raised before, but was implemented using a page ready selector. Knowing when a page is ready can be useful, but it is not able to say when a loading model has disappeared.