Limit ClickHouse select queries for guest web requests
What does this MR do and why?
For guest web requests (no user_id), this MR caps ClickHouse select queries with max_rows_to_read, max_bytes_to_read and
max_execution_time, behind the click_house_guest_query_limits derisk FF, and over a limit the result is truncated (break) instead of
raising an error (this behavior will be documented later).
Additionally, for extra security measure, for requests originating from web, the readonly=2 flag will be added which prevents non-SELECT queries to be invoked. We don't have any queries that are invoked from web context and this setting ensures that.
- Note 1: More rules will be added later for authenticated users.
- Note 2: I added a prometheus metric (
gitlab_click_house_select_queries_total) to track applied rate limits. - Note 3: When time limit is reached CH returns HTTP 200 with empty body -> we transform this into an error.
flowchart TD
A[ClickHouse::Client.select] -->|ReadOnlyMarker sets flag| B[HttpClient.with_query_settings]
X[execute / insert_csv] --> B
B --> C{select?}
C -->|no| D[URL + log_comment]
C -->|yes| E[RateLimiting.rule]
E --> F{guest + web + FF on?}
F -->|yes| G[GuestWebRequest: add limits, break overflow mode]
F -->|no| H[NoLimit]
G --> I[Increment counter, rate_limit_type=guest_web_request]
H --> J[Increment counter, rate_limit_type=none]
I --> K[URL + log_comment + limits]
J --> DReferences
https://gitlab.com/gitlab-org/gitlab/-/work_items/635119+
How to set up and validate locally
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.