## Goal Make GitLab more stable and robust against unintentional or intentional abuses and scenarios where "noisy neighbours" (users, projects, groups) are able to disrupt the whole instance by just doing one action (for example push XX.000 tags, post 10000 comments to an issue through a bot, etc.). > **Note**: The primary goal is the stability of GitLab. Utilizing limits as an opportunity for upsell could be a longer term goal, but we should not gate the implementation of these on a larger pricing/packaging effort. It is recommended to set limits that will allow normal behavior, but prevent intentional or unintentional abuse of the system. ## Solution By putting in different limit layers we should be able to stop the effect on the whole system by rather penalising a single user/project/group which is doing things outside the norm. * These can be either hard limits (higher priority as this can be a safety net) or soft limits (sending warnings to log files to have a better understanding when we getting close and how many executions are near the limit). * Limits should be configurable by Infra so we can adapt on the spot if needed * Hard Limits should fail and raise log errors, should be monitorable and should fail as nicely as possible for the user (lower priority) * Soft Limits should raise log warnings and should be monitorable * Prioritisation input should be in close collaboration with Infrastructure ### Type of limits * Timeout limits * Size/Length limits (e.g. not able to add more than 2MB per Redis entry) * Count limits (e.g. not more than X comments per issue) * Call/time limits (e.g. User can only do 1000 API calls per 10 minutes) * Complexity limits (e.g. 1000 calls to endpoint X compared to 1000 calls to endpoint Y can be totally different) * Max result limit (e.g. Only return max 1000 results and say refine your search) ### Short term solutions **Everybody** Let's collaborate on good ideas, quick wins on what types of limits, where, how. #### Engineering managers - Lead the research Please start an assessment with your team for your speciality by creating a confidential issue and going through this list of tasks: 1. Check Existing limits (are they documented properly? Does infra know about them?) 1. Collect new possibilities for reasonable limits that would make sense in a research issue: * Highlight risk that we currently have without this limit? * Super rough effort estimation to see if we have quick wins * Discuss them with product management and see that for future implementations we also cover such scenarios more during implementation * It's not about the actual value for the limit, the major priority is to find what limits we could introduce Please make them confidential to prevent DoS information from leaking. #### Research issues per area | Area | EM | Research Issue | Implementation Epic | | ------ | ------ | ------ | ------ | | Manage | @lmcandrew | https://gitlab.com/gitlab-org/manage/issues/62 | https://gitlab.com/groups/gitlab-org/-/epics/1748 | | Plan: Team Planning | @johnhope | https://gitlab.com/gitlab-org/plan/issues/9 |https://gitlab.com/groups/gitlab-org/-/epics/1832 | | Plan: Portfolio Management | @johnhope | https://gitlab.com/gitlab-org/plan/issues/9| https://gitlab.com/groups/gitlab-org/-/epics/1833 | | Plan: Certify | @johnhope | https://gitlab.com/gitlab-org/plan/issues/9 | https://gitlab.com/groups/gitlab-org/-/epics/1834 | | Create: Source Code | @m_gill | https://gitlab.com/gitlab-org/create-stage/issues/41 | https://gitlab.com/groups/gitlab-org/-/epics/1787 | | Create: Editor | @dsatcher | https://gitlab.com/gitlab-org/create-stage/issues/40 | https://gitlab.com/groups/gitlab-org/-/epics/1806 | | Create: Knowledge | @dsatcher | https://gitlab.com/gitlab-org/create-stage/issues/39 | https://gitlab.com/groups/gitlab-org/-/epics/1805 | Gitaly | @zj-gitlab | https://gitlab.com/gitlab-org/gitaly/issues/1866 | https://gitlab.com/groups/gitlab-org/-/epics/1882 | | Verify | @erushton | https://gitlab.com/gitlab-org/verify-stage/issues/1 | https://gitlab.com/groups/gitlab-org/-/epics/2037 | | Package | @dcroft | https://gitlab.com/gitlab-org/gitlab/-/issues/213724 | https://gitlab.com/groups/gitlab-org/-/epics/1422 | | Release:Release Management | @sean_carroll & @nicolewilliams | https://gitlab.com/gitlab-org/release-stage/issues/1 | https://gitlab.com/groups/gitlab-org/-/epics/2182 | | Release:Progressive Delivery | @csouthard & @nicolewilliams | https://gitlab.com/gitlab-org/release-stage/issues/1 | https://gitlab.com/groups/gitlab-org/-/epics/2182 | | Configure | @nicholasklick | https://gitlab.com/gitlab-org/configure/general/issues/4 | https://gitlab.com/groups/gitlab-org/-/epics/1842 | | Configure, Serverless | @nicholasklick | https://gitlab.com/gitlab-org/gitlab-ce/issues/66014 | None | | Secure:Static analysis | @twoodham | https://gitlab.com/gitlab-org/gitlab/issues/119359| | | Secure:Composition analysis | @gonzoyumo | https://gitlab.com/gitlab-org/gitlab/issues/119360| | | Secure:Dynamic Analysis | @sethgitlab | https://gitlab.com/gitlab-org/gitlab/issues/119360| | | Defend | @thiagocsf | https://gitlab.com/gitlab-org/gitlab/-/issues/215460 | | | Monitor | @ClemMakesApps @crystalpoole @mnohr | https://gitlab.com/gitlab-org/gitlab-ee/issues/13558 | https://gitlab.com/groups/gitlab-org/-/epics/1862 | | Search | @DylanGriffith | - | https://gitlab.com/groups/gitlab-org/-/epics/2071 | ### Midterm solutions * Tackle those limit implementations prioritized based on impact, safety factor, implementation cost * Can we have global limits (e.g. API level, libraries that we use, etc.)? * Blocking tools for infrastructure * Training for engineers? * Do we need global middleware for some areas (Redis for example) so engineers are warned if they send weird stuff down the infrastructure? * Gorilla testing for weird outside the box inputs