Allow read_api scope in rate limiter
What does this MR do?
Previously, requests using a token that only had read_api access would be treated as unauthenticated by the rate limiter. This ensures that they are (correctly) treated as authenticated.
This does point to a general problem with the rate limiting code: because it happens in a middleware before any application code, it's hard to keep in sync with the different authentication methods the application may allow. This applies to things like:
- How the credentials are sent (Private-Token header, Authorization header, basic auth, etc.).
- What credentials we accept (a personal access token, a personal access token with read_api scope, a job token, a deploy token, etc.).
- Which endpoints those apply to.
The last one is mostly a concern for performance: we might pre-emptively try authenticating using a method that isn't valid for a particular endpoint. From a rate limiting perspective it doesn't matter so much, as we're treating a request with valid credentials for a different endpoint as authenticated for the current endpoint, which is wrong but not a significant problem.
We're not solving the first two in this MR, just continuing to play whack-a-mole.
Testing this locally
- Create a token with
read_api
scope and put it in aGITLAB_API_TOKEN_LOCAL_READ
environment variable. - Enable rate limits for unauthenticated requests at a rate of 1 request per 60 seconds.
-
Note: do this second, and undo it afterwards. Otherwise things like logging in become tricky
😃
-
Note: do this second, and undo it afterwards. Otherwise things like logging in become tricky
On master, you'll see this:
$ curl -s -i -H "Private-Token: $GITLAB_API_TOKEN_LOCAL_READ" http://localhost:3000/api/v4/users/1 | head -n 1
HTTP/1.1 200 OK
$ curl -s -i -H "Private-Token: $GITLAB_API_TOKEN_LOCAL_READ" http://localhost:3000/api/v4/users/1 | head -n 1
HTTP/1.1 429 Too Many Requests
Along with an entry in log/auth.log
for the second request with throttle_unauthenticated
:
{"severity":"ERROR","time":"2020-12-30T13:58:43.863Z","correlation_id":"01ETSZ365Z6FTAYED0K8SDY2G1","message":"Rack_Attack","env":"throttle","remote_ip":"127.0.0.1","request_method":"GET","path":"/api/v4/users/1","matched":"throttle_unauthenticated"}
And on this branch, you'll see:
$ curl -s -i -H "Private-Token: $GITLAB_API_TOKEN_LOCAL_READ" http://localhost:3000/api/v4/users/1 | head -n 1
HTTP/1.1 200 OK
$ curl -s -i -H "Private-Token: $GITLAB_API_TOKEN_LOCAL_READ" http://localhost:3000/api/v4/users/1 | head -n 1
HTTP/1.1 200 OK