Write browser-based configuration to log
Problem
DAST engineers often ask for and receive browserker logs to assist with customer support requests. If the customer does not also submit the CI job log, the DAST engineer cannot determine how DAST was configured, hindering the engineer's ability to diagnose the problem.
The workaround is to go back to the customer and ask for more information about their scan. Replies from the customer include incomplete configuration settings (haven't included CI/CD variables set by the GitLab UI), incorrect configuration settings (as determined by what DAST is doing in the log), or no reply at all.
Proposal
DAST python prints Browserker settings to the log file when run. This should be implemented in Browserker, so that when Browserker is released as the stand-alone DAST solution configuration is available in log files.
Log all *config.Config
values to the browserker log using a new log module, LogConfig: "CONFG"
. This should log at DEBUG
level.
The value of fields CustomHeaders
, CustomCookies
, ClientCertificatePassword
, AuthDetails.Username
, and AuthDetails.Password
should be obfuscated with *******
in the log.
Implementation Plan
- At the start of
BrowserkRunner#Run
, ask the configuration to log itself. Pass in the logger, e.g.r.cfg.LogTo(log)
. - In general, log every configuration name and value.
- Exception: obfuscate
CustomHeaders
,CustomCookies
,ClientCertificatePassword
,AuthDetails.Username
, andAuthDetails.Password
with*******
. E.g.Authentication.Username = "*******"
. - Don't log the following fields:
FormData
,ConsoleLogIncludeGoRoutineID
,FileLogIncludeGoRoutineID
,
- Exception: obfuscate
- Use a new log module. (e.g.
LogConfig: "CONFG"
) - Log at DEBUG level.
- Alphabetically order configuration values.
- Log inner structs. Handle when they may be nil. (e.g.
AuthDetails
,DatabaseDetails
,Timeouts
,DevToolsLoggingConfig
,FeatureFlags
) - Log pointers, careful of when they may be nil. (e.g.
DisableCache *bool
) - Suggested example log output:
2023-12-14 12:32:43,076 DBG CONFG DAST settings 2023-12-14 12:32:43,076 DBG CONFG Auth.Cookies = ["*******"] 2023-12-14 12:32:43,076 DBG CONFG Auth.LoginURL = "https://site.com/login" 2023-12-14 12:32:43,076 DBG CONFG Auth.UserNameField = "css:.username input" 2023-12-14 12:32:43,076 DBG CONFG Auth.UserName = "*******" 2023-12-14 12:32:43,076 DBG CONFG Auth.VerificationURL = "https://site.com/welcome" 2023-12-14 12:32:43,076 DBG CONFG DataPath = "/data/browserker" 2023-12-14 12:32:43,076 DBG CONFG DatabaseDetails.MemoryTableSize = 67108864 2023-12-14 12:32:43,076 DBG CONFG ExcludedElements = [] 2023-12-14 12:32:43,076 DBG CONFG MaxActions = 10000 2023-12-14 12:32:43,076 DBG CONFG MaxDepth = 10 2023-12-14 12:32:43,076 DBG CONFG NumBrowsers = 3 2023-12-14 12:32:43,076 DBG CONFG PluginResourcePath = "/browserker/resources/" 2023-12-14 12:32:43,076 DBG CONFG ReportCookiesPath = "/output/cookies.json" 2023-12-14 12:32:43,076 DBG CONFG ScanMode = "passive" 2023-12-14 12:32:43,076 DBG CONFG ShowBrowser = false
- Out of scope: Ideally, logged variable names would use the CI/CD name as the logged key. This is out of scope for now, until we find a good way to create relationships between TOML/CLI/ENV variable names and configuration settings.