Enforce ci_lint rate limit on lint endpoints behind feature flag
What does this MR do and why?
Enables CI Lint rate limits on various lint entry points, gated behind a feature flag so we can roll it out safely. For the time being adds logging when rate limit is throttled.
- Adds
Gitlab::Ci::Lint::RateLimitErrorand raises it from bothGitlab::Ci::Lint#legacy_validateand#validatewhen a request is throttled. This is the single chokepoint all lint paths flow through. - Adds
#ci_lint_rate_limited?which increments the:ci_lintcounter (scoped to the current user), logs every over-limit request, but only returns true (i.e. actually rejects) when theci_enforce_ci_lint_rate_limitflag is enabled. - Translates the error into the right response at each entry point:
- REST API (
lib/api/lint.rb) and the controller → 429 Too Many Requests with a Retry-After header and a JSON error message. - GraphQL mutation and resolver →
raise_resource_not_available_error!.
- REST API (
- Excludes lint requests from the pipeline-creation rate limiter (
Chain::Limit::RateLimitnow returns early whencommand.linting?) - This MR doesnt cover blob viewer updates, which have been removed here
References
Relates to https://gitlab.com/gitlab-org/gitlab/-/work_items/599486
How to set up and validate locally
- Adjust the CI Lint rate limit (Admin → Settings UI (Pipeline limits section)). In rails console check
Gitlab::ApplicationRateLimiter.threshold(:ci_lint)andGitlab::ApplicationRateLimiter.interval(:ci_lint) - Either using a test or manually-
- Validate log-only mode (flag is disabled:
Feature.disabled?(:ci_enforce_ci_lint_rate_limit)) - Hit the REST endpoint repeatedly past the limit:
- Validate log-only mode (flag is disabled:
for i in $(seq 1 50); do
curl -s -o /dev/null -w "%{http_code}\n"
--header "PRIVATE-TOKEN: <token>"
--header "Content-Type: application/json"
--data '{"content": "build:\n script: echo hello"}'
"http://127.0.0.1:3000/api/v4/projects/<project_id>/ci/lint"
done- Verify every request returns 200, and we can see logs for rate limit
- Enable the flag and reset the counter:
Feature.enable(:ci_enforce_ci_lint_rate_limit)gdk redis-cli flushdb(or clear therate_limitingredis to reset counters) - Re-run the curl loop from step 2. Expected: the first requests return 200, then once the limit is exceeded you get:
HTTP 429 Too Many Requestswith aRetry-Afterheader- JSON body
{"error": "This endpoint has been requested too many times. Try again later."}
- Validate the GraphQL path. With the flag enabled and the limit exceeded, run the
ciLintmutation (orciConfigquery):
mutation {
ciLint(input: { projectPath: "<group/project>", content: "build:\n script: echo" }) {
errors
config { mergedYaml }
}
}- Verify once throttled, the response contains a "resource not available" error instead of a lint result.
- Confirm pipeline-creation limit is unaffected.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #599486
Edited by Tarun Raja