Limit ClickHouse select queries for guest web requests

What does this MR do and why?

For guest web requests (no user_id), this MR caps ClickHouse select queries with max_rows_to_read, max_bytes_to_read and max_execution_time, behind the click_house_guest_query_limits derisk FF, and over a limit the result is truncated (break) instead of raising an error (this behavior will be documented later).

Additionally, for extra security measure, for requests originating from web, the readonly=2 flag will be added which prevents non-SELECT queries to be invoked. We don't have any queries that are invoked from web context and this setting ensures that.

  • Note 1: More rules will be added later for authenticated users.
  • Note 2: I added a prometheus metric (gitlab_click_house_select_queries_total) to track applied rate limits.
  • Note 3: When time limit is reached CH returns HTTP 200 with empty body -> we transform this into an error.
  flowchart TD
      A[ClickHouse::Client.select] -->|ReadOnlyMarker sets flag| B[HttpClient.with_query_settings]
      X[execute / insert_csv] --> B
      B --> C{select?}
      C -->|no| D[URL + log_comment]
      C -->|yes| E[RateLimiting.rule]
      E --> F{guest + web + FF on?}
      F -->|yes| G[GuestWebRequest: add limits, break overflow mode]
      F -->|no| H[NoLimit]
      G --> I[Increment counter, rate_limit_type=guest_web_request]
      H --> J[Increment counter, rate_limit_type=none]
      I --> K[URL + log_comment + limits]
      J --> D

References

https://gitlab.com/gitlab-org/gitlab/-/work_items/635119+

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Adam Hegyi

Merge request reports

Loading
Loading