Accelerate runners using Geo secondary sites
<!-- The first section "Release notes" is required if you want to have your release post blog MR auto generated. Currently in BETA, details on the **release post item generator** can be found in the handbook: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator and this video: https://www.youtube.com/watch?v=rfn9ebgTwKg. The next four sections: "Problem to solve", "Intended users", "User experience goal", and "Proposal", are strongly recommended in your first draft, while the rest of the sections can be filled out during the problem validation or breakdown phase. However, keep in mind that providing complete and relevant information early helps our product team validate the problem and start working on a solution. --> ### Release notes <!-- What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). " --> ### Problem to solve Accelerating runners via Geo secondary sites will help to reduce the load on the primary site by distributing the load to the secondary sites. We are seeing increasing interest in accelerating runners by registering with secondary Geo sites. We know this works for the most part and there are few customers who are already using Geo secondary sites in this way. We want to formally announce support for this by testing all common uses and addressing any gaps found. ### Intended users * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) ### User experience goal System administrators specify the Unified URL as the clone URLs for runners. The runners will be directed to the closest or most appropriate Geo secondary site when resolving the Unified URL. This behaviour should be transparent to the end users. ### Proposal The runners should automatically clone from the closest secondary site based on the unified URL identifying the closest site. For separate secondary URLs additional configuration maybe necessary such as `clone_url`. If so, we should document the needed configuration. If the secondary site does not have the required data to complete the job due to the replicated data not being available in time, the secondary site should proxy the request to the primary site. Secondary sites can be tailored/sized to meet workloads from the runners and will not necessarily need to mirror the resources needed by the primary site. ### Permissions and Security The secondary sites should honour the permission and security policies being enforced by the primary site. ### Availability & Testing <!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the Quality Engineering quad planning and test planning processes and reach out to your counterpart Software Engineer in Test for assistance. Quad Planning: https://about.gitlab.com/handbook/engineering/quality/quality-engineering/quad-planning Test Planning: https://about.gitlab.com/handbook/engineering/quality/quality-engineering/test-engineering/#test-planning --> ### Available Tier * Premium/Silver * Ultimate/Gold ### Feature Usage Metrics We want to track the number of jobs accelerated by Geo secondary sites via usage ping. This is discussed here https://gitlab.com/gitlab-org/gitlab/-/issues/389278. ### What does success look like, and how can we measure that? We expect to see Geo increasingly being used to accelerate runners in addition to accelerating remote users and for DR. We will measure this by analysing the usage ping data and qualitative interviewing of customers using this feature. ### What is the type of buyer? Self-manage customers. ### Is this a cross-stage feature? No ### What is the competitive advantage or differentiation for this feature? GitHub has a [repository cache](https://github.blog/2022-03-22-introducing-the-ghes-repository-cache/) feature that reduces the load on the main instance by serving Git requests from the cache. By accelerating CI runners via Geo, we can not only accelerate repository data by positions us to accelerate other data such as pipeline artifacts, job artifacts and container registry. While Geo doesn't behave like a cache it does provide acceleration that of certain data types by serving them from secondary sites. ### Links / references <!-- Label reminders - you should have one of each of the following labels. Use the following resources to find the appropriate labels: - https://gitlab.com/gitlab-org/gitlab/-/labels - https://about.gitlab.com/handbook/product/categories/features/ --> <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> *This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.* <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
epic