Non-JSON output being logged to JSON formatted logs
Summary
Logs are being written to with non-JSON formatted text. This impacts log consumers expecting JSON formatted input.
Steps to reproduce
-
Ensure logging is configured to use JSON (the default)
-
Access instance URL using:
https://<instance URL>/users/sign_in?test=;
-
Observe the following logged in
/var/log/gitlab/gitlab-workhorse/current
yyyy/mm/dd HH:MM:ss http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192
What is the current bug behavior?
This has been observed in the following log files:
- /var/log/gitlab/praefect/current
- /var/log/gitlab/gitalycurrent
- /var/log/gitlab/gitlab-workhorse/current
The message is not in JSON format.
What is the expected correct behavior?
I'm not sure whether there is an underlying issue that results in this message, if so this may need to be addressed. At the very least the message should be handled correctly and wrapped in JSON to allow correct processing.
Relevant logs
Relevant logs (gitlab-workhorse/current)
{"content_type":"","correlation_id":"01HBW4DQ3A1QKPF7WMJ5AWF18H","duration_ms":0,"host":"gitlab-s1.lab.example.com","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-10-04T01:21:48Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 16.0.1 (16-0-stable; go1.19.9; linux/amd64)","written_bytes":0} {"content_type":"text/html; charset=utf-8","correlation_id":"01HBW4DSBSZN7PGERFT0AV149E","duration_ms":62,"host":"gitlab-s1.lab.example.com","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":200,"system":"http","time":"2023-10-04T01:21:50Z","ttfb_ms":62,"uri":"/users/sign_in?filter=;","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36","written_bytes":12342} 2023/10/04 01:21:50 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192 {"content_type":"","correlation_id":"01HBW4DSJQ6JDMG7QD8A0V59TA","duration_ms":37,"host":"gitlab-s1.lab.example.com","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"https://gitlab-s1.lab.example.com/users/sign_in?filter=;","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/-/","status":304,"system":"http","time":"2023-10-04T01:21:50Z","ttfb_ms":37,"uri":"/-/manifest.json","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36","written_bytes":0}
Details of package version
Provide the package version installation details
Reported on 15.11.11 Reproduced on: gitlab-ee 16.2.5-ee.0 amd64 GitLab Enterprise Edition (including NGINX, Postgres, Redis)
Environment details
- Operating System:
Ubuntu 22.04.2 LTS
- Installation Target, remove incorrect values:
- Bare Metal Machine
- Installation Type, remove incorrect values:
- New Installation