Plain HTTP driver for promisors
Problem to solve
Partial clone is only supported by the Git protocol, not the HTTP protocol. The Git protocol is not well suited to large blobs because the data is packed together. If support for a new simple HTTP protocol was added, this would allow better resume support and streaming objects from object storage.
Further details
This is best done after multiple promisor remotes gitaly#1735 (moved) so that a repository can use the Git protocol for small text files and HTTP for large binary files.
Proposal
Implement a simple HTTP protocol for promisors, that works like this:
- client clones from a regular remote (
git clone --filter=blob:1m) - the regular remote sends a partial clone and tells (in its text output) the client to configure an additional plain HTTP promisor remote (
Configure promisor remote for large file storage: http://gitlab.com/<namespace>/<project>/lfs.git) - client manually configures the additional remote (
git config ...)
Then when the client performs a fetch or an operation that needs objects it doesn't have:
- git first asks the helper/driver for the additional remote to perform either a regular fetch or a "simple blob fetch"
- in case of a regular fetch, the helper/driver for the additional remote does nothing, so Git asks the helper/driver for the regular remote to perform it
- in case of a "simple blob fetch", the helper/driver for the additional remote translates the fetch into a number of simple HTTP GET requests, retrieves the resulting objects and passes them to Git which is happy as it got what it wanted
Links / references
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.