GDK-in-a-box : switch to blobless clones to make DX better on first start

What does this merge request do and why?

The various tasks in GDK don't have full awareness of shallow cloned repo's and keeping them shallow.
From the perspective of the intended usecase of GDK is this makes perfect sense, as the repos managed by GDK are used for daily development (where e.g. gdk update is a normal task to run).

Shallow or blobless clones are not realistic and will result in issues / slowdowns in the end.
This was already experienced by developers using GDK, see #1902 (closed), !3312 (merged) and !5027 (comment 2629824736) (very interesting discussion on this subject)

however, for the current use-case we have with GDK-in-a-box, not fully cloning the repos and living with the known caveats still brings value.
This results in less storage, faster download, faster onboarding for contributors, ...

This MR changes the current clone feature (shallow) to one that has enough context to be faster (blobless).
This results in faster initial gdk update executions AND in a consistent smaller gitlab repo

side note
Another clone feature that is worth trying is treeless.
I'll created a follow-up MR for this so GDK is able to clone repo's that way, and we can switch the GDK-in-a-box image to that approach in the future.

the data

BEFORE (shallow clone)

initial gdk update call

...
┌ ✔ Update your GDK
├── ✔ Install gems for GDK [186ms]
├── ✔ Platform update [11s]
├── ✔ Tool versions update [5m 10s]
├── ✔ Preflight Update checks [10ms]
├── ✔ Update GitLab repository [53s]
...

✅️ Successfully updated in 8m 19s!

repo size before gdk update

gdk@9f71225f6a54:/gitlab-gdk/gitlab-development-kit$ du -sh -BM **/.git | sort -n | tail -n5
1M      gitlab-shell/.git
1M      gitlab-topology-service/.git
3M      gitlab-http-router/.git
95M     gitaly/.git
166M    gitlab/.git

repo size after gdk update

gdk@9f71225f6a54:/gitlab-gdk/gitlab-development-kit$ du -sh -BM **/.git | sort -n | tail -n5
1M      gitlab-topology-service/.git
4M      gitlab-http-router/.git
6M      gitlab-shell/.git
95M     gitaly/.git
2641M   gitlab/.git

AFTER (blobless clone)

initial gdk update call

...
┌ ✔ Update your GDK
├── ✔ Install gems for GDK [165ms]
├── ✔ Platform update [11s]
├── ✔ Tool versions update [2s]
├── ✔ Preflight Update checks [2ms]
├── ✔ Update GitLab repository [4s]
...
✅️ Successfully updated in 1m 11s!

repo size before gdk update

gdk@7df9c21c0039:/gitlab-gdk/gitlab-development-kit$ du -sh -BM **/.git | sort -n | tail -n5
2M      gitlab-topology-service/.git
3M      gitlab-shell/.git
4M      gitlab-http-router/.git
95M     gitaly/.git
1127M   gitlab/.git

repo size after gdk update

gdk@7df9c21c0039:/gitlab-gdk/gitlab-development-kit$ du -sh -BM **/.git | sort -n | tail -n5
2M      gitlab-topology-service/.git
3M      gitlab-shell/.git
5M      gitlab-http-router/.git
95M     gitaly/.git
1133M   gitlab/.git

FUTURE (treeless clone)

initial gdk update call

...
┌ ✔ Update your GDK
├── ✔ Install gems for GDK [182ms]
├── ✔ Platform update [13s]
├── ✔ Tool versions update [5s]
├── ✔ Preflight Update checks [3ms]
├── ✔ Update GitLab repository [2s]
...
✅️ Successfully updated in 1m 19s!

repo size before gdk update

gdk@6949947cf058:/gitlab-gdk/gitlab-development-kit$ du -sh -BM **/.git | sort -n | tail -n5
2M      gitlab-shell/.git
2M      gitlab-topology-service/.git
4M      gitlab-http-router/.git
95M     gitaly/.git
430M    gitlab/.git

repo size after gdk update

gdk@6949947cf058:/gitlab-gdk/gitlab-development-kit$ du -sh -BM **/.git | sort -n | tail -n5
2M      gitlab-shell/.git
2M      gitlab-topology-service/.git
4M      gitlab-http-router/.git
95M     gitaly/.git
546M    gitlab/.git

How to set up and validate locally

Impacted categories

The following categories relate to this merge request:

Merge request checklist

  • This MR references an issue describing the change.
  • This change is backward compatible. If not, include steps to communicate to users.
  • Tests added for new functionality. If not, raise an issue to follow-up.
  • Observability added/updated (logging, metrics, tracing).
  • Documentation added/updated.
  • Announcement added for notable changes.
  • gdk doctor test added.
Edited by Mattias Michaux

Merge request reports

Loading