Add per IP/domain rate-limits to GitLab Pages
This epic groups issues related to rate-limits in GitLab Pages, other resiliency improvements will be in the parent epic ## Current status(2021-10-27) There are 2 major iterations ### IP-based rate-limits There are 2 main blockers: 1. [Running pages with `-listen-proxy` in production](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5706), it's blocked by: 1. [doing than in pre-prod](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2083), it's blocked by 1. [migrating pages to k8s in production](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/273): it's blocked because most likely the same people will work on k8s migration and other pages changes in production, and we want to minimize redoing the work(if we do `-listen-proxy` first, then we'll need to do it both in omnibus now and in charts later) 1. [Adding IP-rate-limit config options to omnibus/charts](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/631) - without it we won't be able to enable anything in production ### Domain-based rate-limits Currently on hold, we will re-evaluate/prioritize it once the previous iteration will be closer to the finish line ### Next things to work on 1. Wait for https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5706 1. While we do that, we should work on https://gitlab.com/gitlab-org/gitlab-pages/-/issues/631 to allow us actually enable rate-limit ### Release notes We have added rate limiting capabilities to our Pages feature. Unlimited or undesired traffic (such as a Denial of Service attack) to hosted pages can cause unexpected availability issues or even downtime for users. With this update, rate limiting can be enforced per specific client IP addresses and per specific hosted pages domain. Limits can be configured for each independently. When enabled and traffic exceeds these limits, requests will be reported and rejected.
epic