Always log remote_ip and client_id for API routes
Some API requests are handled by Grape directly without ever going
through before
blocks. We can still log remote_ip
(and hence
client_id
) for those requests, as that doesn't depend on anything
application-specific.
We handle this by putting it in the logger itself as well as the
before
block. This is because client_id
is computed based on an
ApplicationContext instance, but this is backed by a Labkit::Context. A
Labkit::Context isn't aware of our models, and so user
is only
inherited as a string, not an object - which means that client_id
could end up incorrectly using the IP rather than the user ID.
For gitlab-com/gl-infra/scalability#384 (closed).
Testing
These tests manually hit each case I found in the production logs, and show the values of (in order):
meta.caller_id
meta.feature_category
meta.remote_ip
route
We expect some cases where meta.caller_id
is not set but route
is:
that means that it's a route that doesn't match our application's
endpoints, usually because we don't handle that method. Even if
meta.caller_id
is not present, we may be able to fill in
meta.feature_category
if that is defined on the class level.
405 Method Not Allowed
$ curl -i -X PUT 'http://localhost:3000/api/scim/v2/groups/1/Users/foo'
HTTP/1.1 405 Method Not Allowed
Allow: OPTIONS, GET, PATCH, DELETE, HEAD
Cache-Control: no-cache
Content-Type: application/scim+json
Vary: Origin
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: 01FHTYF4KTT18ZYTR7048JVMXG
X-Runtime: 0.174975
Date: Tue, 12 Oct 2021 19:08:03 GMT
Content-Length: 27
{"error":"405 Not Allowed"}
$ tail log/api_json.log | grep 405 | jq '[."meta.caller_id", ."meta.feature_category", ."meta.remote_ip", .route]'
[
null,
"authentication_and_authorization",
"127.0.0.1",
"/api/scim/:version/groups/:group/Users/:id"
]
415 Unsupported Media Type
$ curl -i -X PUT -H 'Content-Type: image/png' -H "Private-Token: $GITLAB_API_TOKEN_LOCAL" 'http://localhost:3000/api/v4/users/1' -d '{"name": "Administrator"}'
HTTP/1.1 415 Unsupported Media Type
Cache-Control: no-cache
Content-Type: application/json
Vary: Origin
X-Request-Id: 01FHTYDDKN0JBFD1P0AS3SKJRD
X-Runtime: 4.001786
Date: Tue, 12 Oct 2021 19:07:10 GMT
Content-Length: 67
{"error":"The provided content-type 'image/png' is not supported."}
$ tail log/api_json.log | grep 415 | jq '[."meta.caller_id", ."meta.feature_category", ."meta.remote_ip", .route]'
[
null,
null,
"127.0.0.1",
"/api/:version/users/:id"
]
204 No Content (from OPTIONS)
$ curl -i -X OPTIONS -H "Private-Token: $GITLAB_API_TOKEN_LOCAL" 'http://localhost:3000/api/v4/users/1'
HTTP/1.1 204 No Content
Allow: OPTIONS, GET, PUT, DELETE, HEAD
Cache-Control: no-cache
Vary: Origin
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: 01FHTYBT65EWZDS59QJV8057NH
X-Runtime: 0.364321
Date: Tue, 12 Oct 2021 19:06:14 GMT
$ tail log/api_json.log | grep 204 | jq '[."meta.caller_id", ."meta.feature_category", ."meta.remote_ip", .route]'
[
null,
"users",
"127.0.0.1",
"/api/:version/users/:id"
]
404 Not Found (from bad API version)
$ curl -i 'http://localhost:3000/api/v4_what'
HTTP/1.1 404 Not Found
Cache-Control: no-cache
Content-Type: application/json
Vary: Origin
X-Request-Id: 01FHTY9Y89E0ZW6CYHSADF6F05
X-Runtime: 3.856277
Date: Tue, 12 Oct 2021 19:05:16 GMT
Content-Length: 37
{"error":"404 API Version Not Found"}
$ tail log/api_json.log | grep 404 | jq '[."meta.caller_id", ."meta.feature_category", ."meta.remote_ip", .route]'
[
null,
null,
"127.0.0.1",
"/api/:version/*path"
]