Skip to content

Record rate limiting gates in logs

Sean McGivern requested to merge record-rate-limiters-in-logs into master

This will allow us to do things like:

  1. See which requests are the most expensive for a particular gate.
  2. Find expensive requests that have no rate limiting applied.

etc.

Local testing

Easy mode: look at development_json.log and api_json.log. Make a comment with the web UI:

$ tail -f log/development_json.log | grep -vF '"rate_limiting_gates":[]' | jq '[.controller, .action, .rate_limiting_gates]'
[
  "Projects::NotesController",
  "create",
  [
    "notes_create"
  ]
]

Or through the API:

# In one shell
$ tail -f log/api_json.log | grep -vF '"rate_limiting_gates":[]' | jq '[.route, .rate_limiting_gates]'
[
  "/api/:version/projects/:id/issues/:noteable_id/notes",
  [
    "notes_create"
  ]
]
# In another
$ curl -H "Private-Token: $GITLAB_API_TOKEN_LOCAL" -X POST 'http://gdk.test:3000/api/v4/projects/24/issues/21/notes?body=Hello'

To test what happens when multiple gates are checked (I don't know if this happens anywhere in practice), try this diff and then commenting:

diff --git a/app/controllers/concerns/notes_actions.rb b/app/controllers/concerns/notes_actions.rb
index 8410a8779f6..65b2310420f 100644
--- a/app/controllers/concerns/notes_actions.rb
+++ b/app/controllers/concerns/notes_actions.rb
@@ -19,6 +19,9 @@ module NotesActions
       check_rate_limit!(:notes_create,
         scope: current_user,
         users_allowlist: Gitlab::CurrentSettings.current_application_settings.notes_create_limit_allowlist)
+      check_rate_limit!(:issues_create,
+        scope: current_user,
+        users_allowlist: Gitlab::CurrentSettings.current_application_settings.notes_create_limit_allowlist)
     }, only: [:create]
   end
$ tail -f log/development_json.log | grep -vF '"rate_limiting_gates":[]' | jq '[.controller, .action, .rate_limiting_gates]'
[
  "Projects::NotesController",
  "create",
  [
    "notes_create",
    "issues_create"
  ]
]

For gitlab-com/gl-infra/scalability#1662 (closed).

Edited by Sean McGivern

Merge request reports

Loading