Skip to content

Site Availablity check should produce output when site returns content

Problem to solve

As a user who is trying to debug site availability to my target application from the DAST scanner, it is difficult to know what the root cause of communication errors are.

Intended users

User experience goal

The user should be able to get visual confirmation of what communication errors exist between the scanner and the target application to scan.

Proposal

When running a DAST scan the site availability check determines if the target application is accessible. The site validation check should give users more information on why it determines the site is unavailable. It could be due to:

  1. No route to host (general network errors)
  2. Website is not accessible on the provided target port (unable to establish a connection)
  3. Website is accessible, but returning a 404 or some other indication that the application is not able to be scanned.

Having messages such as:

2020-08-17 03:03:33,874 requesting access to http://vulnapp:8000/ (No route to host)
...
2020-08-17 03:03:33,874 requesting access to http://vulnapp:8000/ (Unable to connect to port)
...
2020-08-17 03:03:33,874 requesting access to http://vulnapp:8000/ (404 + body text)
...

Having the above log text would significantly reduce time to debug what the exact issues may be.

Further details

Currently the availability check only states:

2020-08-17 03:03:33,874 waiting for http://vulnapp:8000/ to be available
2020-08-17 03:03:33,874 requesting access to http://vulnapp:8000/
...
2020-08-17 03:04:31,040 http://vulnapp:8000/ could not be reached, attempting scan anyway
2020-08-17 03:04:31,040 domain validation failed due to: Attempting to full scan, but the site is unavailable. Permissions cannot be verified., see https://docs.gitlab.com/ee/user/application_security/dast/#domain-validation

In reality the site is somewhat accessible. By using curl from the DAST container we can access the site:

$ curl http://vulnapp:8000/
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8" />
  <title>Action Controller: Exception caught</title>
...
<header>
  <h1>Blocked host: vulnapp</h1>
</header>
<div id="container">
  <h2>To allow requests to vulnapp, add the following to your environment configuration:</h2>
  <pre>config.hosts &lt;&lt; "vulnapp"</pre>
</div>

Documentation

Troubleshooting documentation should be updated to assist users with these types of communication errors.

Availability & Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Gold/Ultimate