Gitaly 16.7 Planning (December 21, 2023)
This issue and linked pages contain information related to upcoming products, features, and functionality. It is important to note that the information presented is for informational purposes only. Please do not rely on this information for purchasing or planning purposes. As with all projects, the items mentioned in this video and linked pages are subject to change or delay. The development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.
Table of Contents
Boards & Links
Issue boards we use for planning
- Full list of release dates
- Gitaly Objectives and Key Results - GitLab Internal Only
Capacity Notes
Given that this release spans holidays (Thanksgiving, Christmas, etc...) for team members, we will be operating at a rather reduced capacity. As such, we will be using the 16.7 release to catch up on overflow from the %16.6 release, while attempting to ensure we make progress on our committed objectives.
Objectives and Themes
Please see our direction page section What's Next & Why for our ongoing focus. The list below are specific pieces from these themes.
Support Bundle URIs for Clones
We are close to being able to roll out a very minimal bundle URI implementation. This will allow the server to skip the expensive step of packing up all objects into a packfile to send to the client, and is designed to reduce server load significantly for cases where large repositories are cloned often.
This has some direct benefits for customers with monorepos, as this should allow for less server load during normal operation.
Specific items being worked for this release include:
- [Feature flag] Enable roll out of bundle URI (gitlab-org/gitaly#5656)
- Write playbook that shows how to use bundle-uri (gitlab-org/gitaly#5686 - closed)
Server-side Backups
As discussed in our Repository Backups blueprint, we have recently released the option to perform server side backups. The goal of this is to improve performance of backups, and eliminate complicated backups scripts that have to perform the backup on the Gitaly node, and then transfer the backups elsewhere. You can see the full list of goals in the blueprint.
Given that we've released a very minimal version of this, our desire is to continue to iterate in an effort to improve the usability of this feature, and allow it to be more accessible for our customers.
Implement write-ahead logging in Gitaly
As we migrate toward a decentralized architecture for Gitaly cluster, and in support of many other key initiatives (such as the Disaster Recovery Working Group), we need to implement a write-ahead log for Gitaly. In this release, we're looking to continue making progress toward write-ahead logging.
RefTable Support in GitLab
RefTables are a new storage structure within Git that stores refs
in a portable binary format instead of loose files or packed-refs files. This will improve Git in two main ways:
- Performance - updating
refs
in areftable
will be more performant than updating loose refs or packed-refs files - Remove race conditions - the current Git implementation causes race-conditions around deletion or modifications to refs that must be avoided.
The group::gitalygit team is actively working with the upstream Git mailing list to contribute this feature. It will be an ongoing effort, but one which will help out GitLab as well as the broader Git community and we feel honoured to be able to work on this improvement. This epic is tracking the work to upstream the reftable backend to Git.
Planning Inputs
Note: The Gitaly team currently does not have a UX component for planning inputs.
Engineering Priorities
Quality Priorities
Bug Prioritization Sisense Board - Internal Only. Filter on team_group = gitaly
to see Gitaly specific.
References
Handbook pages which are useful
/cc @eread