GDK-in-a-box : switch to blobless clones to make DX better on first start
What does this merge request do and why?
The various tasks in GDK don't have full awareness of shallow cloned repo's and keeping them shallow.
From the perspective of the intended usecase of GDK is this makes perfect sense, as the repos managed by GDK are used for daily development (where e.g. gdk update is a normal task to run).
Shallow or blobless clones are not realistic and will result in issues / slowdowns in the end.
This was already experienced by developers using GDK, see #1902 (closed), !3312 (merged) and !5027 (comment 2629824736) (very interesting discussion on this subject)
however, for the current use-case we have with GDK-in-a-box, not fully cloning the repos and living with the known caveats still brings value.
This results in less storage, faster download, faster onboarding for contributors, ...
This MR changes the current clone feature (shallow) to one that has enough context to be faster (blobless).
This results in faster initial gdk update executions AND in a consistent smaller gitlab repo
side note
Another clone feature that is worth trying is treeless.
I'll created a follow-up MR for this so GDK is able to clone repo's that way, and we can switch the GDK-in-a-box image to that approach in the future.
the data
BEFORE (shallow clone)
initial gdk update call
...
┌ ✔ Update your GDK
├── ✔ Install gems for GDK [186ms]
├── ✔ Platform update [11s]
├── ✔ Tool versions update [5m 10s]
├── ✔ Preflight Update checks [10ms]
├── ✔ Update GitLab repository [53s]
...
✅️ Successfully updated in 8m 19s!
repo size before gdk update
gdk@9f71225f6a54:/gitlab-gdk/gitlab-development-kit$ du -sh -BM **/.git | sort -n | tail -n5
1M gitlab-shell/.git
1M gitlab-topology-service/.git
3M gitlab-http-router/.git
95M gitaly/.git
166M gitlab/.git
repo size after gdk update
gdk@9f71225f6a54:/gitlab-gdk/gitlab-development-kit$ du -sh -BM **/.git | sort -n | tail -n5
1M gitlab-topology-service/.git
4M gitlab-http-router/.git
6M gitlab-shell/.git
95M gitaly/.git
2641M gitlab/.git
AFTER (blobless clone)
initial gdk update call
...
┌ ✔ Update your GDK
├── ✔ Install gems for GDK [165ms]
├── ✔ Platform update [11s]
├── ✔ Tool versions update [2s]
├── ✔ Preflight Update checks [2ms]
├── ✔ Update GitLab repository [4s]
...
✅️ Successfully updated in 1m 11s!
repo size before gdk update
gdk@7df9c21c0039:/gitlab-gdk/gitlab-development-kit$ du -sh -BM **/.git | sort -n | tail -n5
2M gitlab-topology-service/.git
3M gitlab-shell/.git
4M gitlab-http-router/.git
95M gitaly/.git
1127M gitlab/.git
repo size after gdk update
gdk@7df9c21c0039:/gitlab-gdk/gitlab-development-kit$ du -sh -BM **/.git | sort -n | tail -n5
2M gitlab-topology-service/.git
3M gitlab-shell/.git
5M gitlab-http-router/.git
95M gitaly/.git
1133M gitlab/.git
FUTURE (treeless clone)
initial gdk update call
...
┌ ✔ Update your GDK
├── ✔ Install gems for GDK [182ms]
├── ✔ Platform update [13s]
├── ✔ Tool versions update [5s]
├── ✔ Preflight Update checks [3ms]
├── ✔ Update GitLab repository [2s]
...
✅️ Successfully updated in 1m 19s!
repo size before gdk update
gdk@6949947cf058:/gitlab-gdk/gitlab-development-kit$ du -sh -BM **/.git | sort -n | tail -n5
2M gitlab-shell/.git
2M gitlab-topology-service/.git
4M gitlab-http-router/.git
95M gitaly/.git
430M gitlab/.git
repo size after gdk update
gdk@6949947cf058:/gitlab-gdk/gitlab-development-kit$ du -sh -BM **/.git | sort -n | tail -n5
2M gitlab-shell/.git
2M gitlab-topology-service/.git
4M gitlab-http-router/.git
95M gitaly/.git
546M gitlab/.git
How to set up and validate locally
Impacted categories
The following categories relate to this merge request:
-
gdk-reliability - e.g. When a GDK action fails to complete. -
gdk-usability - e.g. Improvements or suggestions around how the GDK functions. -
gdk-performance - e.g. When a GDK action is slow or times out.
Merge request checklist
-
This MR references an issue describing the change. -
This change is backward compatible. If not, include steps to communicate to users. -
Tests added for new functionality. If not, raise an issue to follow-up. -
Observability added/updated (logging, metrics, tracing). -
Documentation added/updated. -
Announcement added for notable changes. -
gdk doctortest added.