### Problem to solve <!-- What problem do we solve? --> Geo supports two distinct use cases: Disaster Recovery (DR) and Geo Replication. These two use cases have different requirements and support different user stories. For DR purposes it is important that all data is replicated and that a working GitLab installation can be recovered in a failover event. For Geo replication, the emphasis is on user experience improvements. Data required in remote locations should be fast to work with. Currently, Geo nodes can generally support both use cases but this has a few downsides for Geo replication: - For Geo replication only a subset of data may need to be replicated. Systems administrators can select a subset via selective sync, but they may be wrong. - Geo nodes require spinning up the entire GitLab stack, less may be sufficient - We don't manage data replication based on user demand - The secondary web interface is read-only ### Intended users * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) (and all other personas using GitLab really) ### Further details https://gitlab.com/grouperenault/git_cdn Is a project that is already achieving part of this behavior. <!-- Include use cases, benefits, and/or goals (contributes to our vision?) --> ### Proposal <!-- How are we going to solve the problem? Try to include the user journey! https://about.gitlab.com/handbook/journeys/#user-journey --> I propose a Geo "caching mode" with the following properties: - A Geo node that dynamically determines which data needs to be kept close to the remote location - Minimal hardware requirements - Fully hidden behind geo-location aware URL The user journey would be as follows: CLI: - Sasha clones a project - Geo node checks if project is cached - If yes: Clone will use cache and download from Geo node - Else: Proxy to primary; first load is slow - Geo node now starts caching the project locally (default expiration 30 days for example) - Sasha pulls from the project - Geo node now provides fast access If the project is no longer used by the location, we drop the replication after a while UI: Sasha accesses `gitlab.example.com` - Local Geo node provides all assets that require read access via something like or example through Varnish or some such. - The Geo node only acts as a local "CDN" that should speed up rendering the webpage. We can also provide all uploads etc. as well - The Web page is read-write (writes are still slow) ### Permissions and Security <!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)?--> * Testing required ### Documentation <!-- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html Add all known Documentation Requirements here, per https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html#documentation-requirements If this feature requires changing permissions, this document https://docs.gitlab.com/ee/user/permissions.html must be updated accordingly. --> * Extensive documentation necessary ### Availability & Testing * Unit test changes * Integration test changes * End-to-end test change ### What does success look like, and how can we measure that? <!-- Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this. --> - Geo caching nodes provide on-demand cache for users that is fully transparent - Vastly reduced infrastructure cost because not everything needs to be replicated - Read-write* interface with caches assets ### What is the type of buyer? <!-- Which leads to: in which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#four-tiers --> * Ultimate ### Links / references