Make `remote_ip` and `path` log fields in `auth.log` messages more consistent with other logs
Problem to solve
The json formatted auth.log
entries have inconsistent field names with other log entries included in the same log for shared fields for the client IP and path of the request. They should be consistent.
Intended users
- Sidney (Systems Administrator)
- Sam (Security Analyst)
Further details
The fields for request path and client IP are inconsistent between different tag
. This prevents easily correlating or querying across multiple logs for known values. This also requires mental overhead in analysts working with the data to remember the different field names.
{
"tag": "rails.production",
"path": "mygroup/myproject/raw/master/things.png",
"remote_ip": "192.168.1.1"
...
}
{
"tag": "rails.auth",
"json.env": "raw_blob_request_limit",
"fullpath": "mygroup/myproject/raw/master/things.png",
"ip": "192.168.1.1"
...
Proposal
Standardize to common set of field names:
remote_ip
path
These can be any that make sense and would be best to follow any de facto industry standards; the goal here is just to make common fields consistent across the multiple logs generated by the single application.
For example "client_ip" is often used in other logs in production architectures where requests may be forwarded through multiple proxies or components before getting serviced.
Permissions and Security
This applies to log files as written to disk, so maintains the same access required of system administrators.
Documentation
The details of the various tag
s are not present in: https://docs.gitlab.com/ee/administration/logs.html#production_jsonlog
Testing
This will mostly break downstream usage of the logs in our own processes and tooling.
What does success look like, and how can we measure that?
Implementing this change should reduce the time spent by support, security, and infrastructure teams when investigating reports of blocked access we frequently receive when rate limits are exceeded in the application.