User can configure DAST to wait for an element to be present before crawling the page
Problem
Web Applications that have Loading modal dialogs in between page transitions pose a problem for DAST browser-based scans.
Proposal
When a DAST browser-based scan transitions to a new page, a series of checks are run to determine that the page is loaded. These include whether or not the document has updated, if there are any pending requests, timeouts, etc.
This MR proposes that a new CI/CD environment variable DAST_BROWSER_PAGE_READY_SELECTOR
be made available to the user. When the user enters a selector of the login modal, the DAST browser-based crawler should check to ensure the element is not found or is hidden before classifying a page transition as complete.
More details
The issue arises as the DAST crawler assumes the page has finished loading when it is still showing "Loading". This can lead to:
- Missed elements in the scan that will not be crawled.
- DevTools will return the error
code: -32000 msg: Could not compute box model
when attempting to retrieve the position of elements that are occluded by a Loading modal (even if the modal is partially transparent). These are considered hidden elements by the scan and are not crawled.
The error is picked up in the following code. Note that when the error is found, it is not logged, and the element is assumed to be hidden.
func (e *InternalElement) IsElementHidden() bool {
x, y, err := e.GetPosition()
if err != nil || (x == 0 && y == 0) || (x < 0 || y < 0) {
return true
}
Implementation Plan
-
Add PageReadySelector
to browser-based scanner -
Upgrade DAST and add DAST_BROWSER_PAGE_READY_SELECTOR
gitlab-org/security-products/dast!537 (merged) -
Document DAST_BROWSER_PAGE_READY_SELECTOR
in https://docs.gitlab.com/ee/user/application_security/dast/browser_based.html#available-cicd-variables. Using this feature has the potential of slowing down the scan 389#note_743194516 so that should be documented too. !75456 (merged)