Geo 15.9 outlook
This is the current outlook for the 15.9 milestone. The Geo team schedules issues and works in a continuous Kanban manner, while still aligning with milestones. See our process page for more detail.
Last Updated: 16 January 2023
Move existing data types into the Self Service Framework
Who: Douglas, Javiera, Zack
Geo's Self Service Framework significantly simplifies Geo's codebase and ensures that all data types are replicated and verified. It also empowers other developers to contribute new data types. We've already moved over most data types but a few are still implemented using our legacy system. The remaining data types are
In %15.9 and the following milestones we should focus on finishing the migration of these data types so that we can increase the velocity of future feature develop and reduce the complexity of Geo.
Move Projects and Wiki to Self-Service Framework priority1
Who: Douglas, Javiera, Zack
We will split project and wiki replication and verification logic.
- We will first extract wiki replication and verification data in its own table
- Add Graph QL endpoints for Projects and Wikis
- Revisiting and consolidate the existing UI functionality for Geo replication views
- Implement Git repo housekeeping in SSF for Project Wikis, Group Wikis and Snippets
Will be looking to wrap up this migration in this milestone.
Move design repositories to the Self-Service Framework priority1
Who: Aakriti
Migrate the last legacy data type, design repositories, to the self-service framework.
Will be looking to wrap up this migration in this milestone.
Fix timeouts when pushing via SSH to secondary priority2
Who: Mike
Currently there is 60 second timeout on the proxied push request from the secondary to the primary. For large push requests this is insufficient time for the request to complete and times out.
The solution will likely require a architectural change to how we handle SSH proxy requests and will be a significant effort.
The ultimate goal will be to remove the restriction on size of proxied SSH push requests.
Verification of container repositories priority3
Add verification of container repositories. This is part of our effort to increase verification coverage of replicated data types.
priority3
Deprecate node in favor of siteWe've revised most of the terminology in our documentation to align to the updated glossary of Geo terms from node to site to reflect. The term node now denoting a single server and site being a collection one or more nodes running a single GitLab application. In this milestone we want update the labels for values in the gitlab.rb file and API endpoints to the updated terms.
- Deprecate geo_node_name in favor of geo_site_name in gitlab.rb
- New API endpoint for geo_sites to supersede geo_nodes
Support mulitple backend DBs in support of decomposition priority4
To improve scalability the GitLab database team split the backend database into two separate databases, main and CI. Geo is currently built to work with a single backend database. In order to support the future state of GitLab deployments, Geo needs to support the decomposed two database model. In the first phase we will support two databases on the same on a single Postgres instance.
Testing progress is awaiting completion of Decomposed database (main + ci) should work with PgBouncer
Bugs
- Secondary node exposes the "external_url" of the Primary node in the clone dialog priority2
- Secondary site redirects to primary when using Hybrid primary priority2
- Geo: Secondary fails to replicate when using Hybrid Primary priority2
- staging-ref.gitlab.com has replication and checksum failures priority2
- Geo - Improve the performance of slow count queries triggered by the Geo::MetricsUpdateWorker priority2 - @jtapiab
-
Failed
gitlab-backup create
to remote storage silently fails with no error priority3
Misc
- Add specs to catch new kinds of Geo data priority2
- Geo: replicate-geo-database error messages highlight wrong information priority3
- Git repository replication failures: Error while processing content unencoding: incorrect header check priority3
- Hashed Storage as the only storage - Clean up activities to wrap up this effort. priority4
-
Deprecate
rake backup:<resource_name>:*
subtasks - priority4