Loading
feat(rate_limit): add BlockSummary to log block counts per endpoint and rule
Description
Adds Labkit::RateLimit::BlockSummary. It counts blocks per (endpoint_id, rule) in one process and logs one line per pair every interval.
A line per block grows with an attack. These lines grow only with the number of distinct pairs, and never past max_pairs + 1 per process per interval.
summary = Labkit::RateLimit::BlockSummary.new
Thread.new { summary.call }
summary.add(endpoint_id: "GET /api/:version/projects", rule: "unauthenticated_api", feature_category: "source_code_management")
# on shutdown
summary.stop{"message": "rate_limit_block_summary", "endpoint_id": "GET /api/:version/projects", "rate_limit_rule": "unauthenticated_api", "block_count": 412, "feature_category": "source_code_management"}- The key is the pair. The first non-nil
feature_categorypassed for a pair is the one logged, so a category never adds a line. - Caps at
max_pairspairs per interval (default 10). The rest fold into oneotherline with nofeature_category, so memory and lines stay bounded during an attack. - The first flush waits a random part of the interval, so workers booted together do not flush at once.
stopsignals the thread running#call, which runs a final flush. With no thread in#call, nothing is flushed. A process that exits withoutstoploses up to one interval of counts.- A logging failure is logged at
WARNwithmessage: "rate_limit_error"anddropped_pairs, and never raises. newraisesArgumentErroron a non-positiveintervalormax_pairs.- The caller owns the thread and decides what
endpoint_idmeans. LabKit does not resolve it.
The log line is pinned in labkit-spec first: gitlab-org/quality/tooling/labkit-spec!50 (merged). This MR is set to merge after that one.
Testing
spec/labkit/rate_limit: 469 examples, 0 failures. RuboCop clean.- A gitlab-rails version of the same tally ran end to end on a local Caproni cluster: 5 blocked requests to
/explore/projectsgave one line withcount: 5(the field was namedcountbefore the rename toblock_count).
Related Issues
Checklist
- labkit-spec MR merged
- Teammate review
- Maintainer review
Edited by Nidhey Indurkar