DAST finding evidence is missing many response headers

Problem

The evidence used for a DAST finding contains HTTP response information, including headers. A scan of a customer site revealed that some response headers are not being shown as evidence. Customers rely on this information to determine whether a finding is a true or false positive.

Proposal

Show all response headers as evidence in a DAST vulnerability finding.

Reference

https://gitlab.com/gitlab-com/sec-sub-department/section-sec-request-for-help/-/issues/265#note_1951010745

More details

DAST listens to events sent by Chromium when crawling the site. This example was found with the following information in the logs:

TRC CHROM event received    {"method":"Fetch.requestPaused"...
TRC CHROM event received    {"method":"Network.requestWillBeSent"...
TRC CHROM event received    {"method":"Network.requestWillBeSentExtraInfo"...
TRC CHROM event received    {"method":"Network.responseReceivedExtraInfo","params":{"requestId":"30331.55","blockedCookies":[],"headers":{"Set-Cookie":"sessionID=12345678; Path=/;...
TRC CHROM event received    {"method":"Fetch.requestPaused"...
TRC CHROM event received    {"method":"Network.responseReceived","params":{"requestId":"30331.55",...,"response":{"headers":{"Pragma":"no-cache","Date":"Mon, 17 Jun 2024 12:04:27 GMT",...
TRC CHROM event received    {"method":"Network.loadingFinished"...

The actual cause has not been determined, however, the following may be of relevance:

  • There was no response body (Content-Length: 0)
  • The Network.responseReceivedExtraInfo was out-of-order, being received before Network.responseReceived.

Also worth noting:

  • The DatabaseFindingStore marshals the entire response to save it to the database. Ideally, this should use the HTTPResponseStore or HTTPMessageStore. It's unlikely that this is the cause of the error.
  • HTTPMessageService.Finalize only uses headers from the Network.responseReceived event. Ideally, this would also incorporate headers from the Network.responseReceivedExtraInfo event. HTTPMessageFactory.getResponseHeaders shows how both events can combine to get the actual headers sent.
Edited by Cameron Swords