Support cloning with an object pool in CreateFork
This is needed to support fast forking. When an object pool is provided, forking can cheap in terms of disk space and time since the clone only needs to fetch the references.
The git clone
can take in a --reference <repository>
flag and will output an alternates file with absolute paths. To
maintain the use of relative paths, after the fork is successful we
recreate the alternates file.
Forking a 1.1 GB copy of gitlab-org/gitlab now takes just seconds as opposed to over a minute on my local system:
{
"correlation_id":"01ES6N07PMDJTGPCW05JX2XBGQ",
"grpc.code":"OK",
"grpc.meta.auth_version":"v2",
"grpc.meta.client_name":"gitlab-sidekiq",
"grpc.meta.deadline_type":"unknown",
"grpc.method":"CreateFork",
"grpc.request.deadline":"2020-12-10T13:41:06-08:00",
"grpc.request.fullMethod":"/gitaly.RepositoryService/CreateFork",
"grpc.request.glProjectPath":"first/second/gitlab",
"grpc.request.glRepository":"project-268",
"grpc.request.repoPath":"@hashed/8b/49/8b496bf96bbcc9e5ac11c068b6cfb00c32f9d163bb8a3d5af107217499de997a.git",
"grpc.request.repoStorage":"praefect-internal-0",
"grpc.request.topLevelGroup":"@hashed",
"grpc.service":"gitaly.RepositoryService",
"grpc.start_time":"2020-12-10T07:41:06-08:00",
"grpc.time_ms":1220.468,
"level":"info",
"msg":"finished unary call with code OK",
"object_pool_path":"/Users/stanhu/gitlab/gdk-ee/repositories/@pools/2c/62/2c624232cdd221771294dfbb310aca000a0df6ac8b66b696d90ef06fdefb64a3.git",
"peer.address":"",
"pid":43517,
"source_path":"@hashed/6a/f1/6af1f692e9496c6d0b668316eccb93276ae6b6774fa728aac31ff40a38318760.git",
"source_storage":"default",
"span.kind":"server",
"system":"grpc",
"target_path":"@hashed/8b/49/8b496bf96bbcc9e5ac11c068b6cfb00c32f9d163bb8a3d5af107217499de997a.git",
"target_storage":"praefect-internal-0",
"time":"2020-12-10T07:41:07.249Z"
}
Relates to gitlab#24523
Edited by Stan Hu