Skip to content

Support cloning with an object pool in CreateFork

Stan Hu requested to merge sh-create-fork-with-object-pool into master

This is needed to support fast forking. When an object pool is provided, forking can cheap in terms of disk space and time since the clone only needs to fetch the references.

The git clone can take in a --reference <repository> flag and will output an alternates file with absolute paths. To maintain the use of relative paths, after the fork is successful we recreate the alternates file.

Forking a 1.1 GB copy of gitlab-org/gitlab now takes just seconds as opposed to over a minute on my local system:

{
   "correlation_id":"01ES6N07PMDJTGPCW05JX2XBGQ",
   "grpc.code":"OK",
   "grpc.meta.auth_version":"v2",
   "grpc.meta.client_name":"gitlab-sidekiq",
   "grpc.meta.deadline_type":"unknown",
   "grpc.method":"CreateFork",
   "grpc.request.deadline":"2020-12-10T13:41:06-08:00",
   "grpc.request.fullMethod":"/gitaly.RepositoryService/CreateFork",
   "grpc.request.glProjectPath":"first/second/gitlab",
   "grpc.request.glRepository":"project-268",
   "grpc.request.repoPath":"@hashed/8b/49/8b496bf96bbcc9e5ac11c068b6cfb00c32f9d163bb8a3d5af107217499de997a.git",
   "grpc.request.repoStorage":"praefect-internal-0",
   "grpc.request.topLevelGroup":"@hashed",
   "grpc.service":"gitaly.RepositoryService",
   "grpc.start_time":"2020-12-10T07:41:06-08:00",
   "grpc.time_ms":1220.468,
   "level":"info",
   "msg":"finished unary call with code OK",
   "object_pool_path":"/Users/stanhu/gitlab/gdk-ee/repositories/@pools/2c/62/2c624232cdd221771294dfbb310aca000a0df6ac8b66b696d90ef06fdefb64a3.git",
   "peer.address":"",
   "pid":43517,
   "source_path":"@hashed/6a/f1/6af1f692e9496c6d0b668316eccb93276ae6b6774fa728aac31ff40a38318760.git",
   "source_storage":"default",
   "span.kind":"server",
   "system":"grpc",
   "target_path":"@hashed/8b/49/8b496bf96bbcc9e5ac11c068b6cfb00c32f9d163bb8a3d5af107217499de997a.git",
   "target_storage":"praefect-internal-0",
   "time":"2020-12-10T07:41:07.249Z"
}

Relates to gitlab#24523

Edited by Stan Hu

Merge request reports