Speed up Review Apps deploy step
Right now we have a 70-80 pending jobs waiting for a Review App job to build (https://gitlab.com/gitlab-com/www-gitlab-com/-/jobs?scope=pending). This is blocking merge requests from being merged.
We only have one runner that can pick up these jobs, and it usually takes 5 minutes/job to:
- Download the artifacts.zip
- Extract the ZIP to a directory
-
rsyncit to the right place
How can we speed this up? Some ideas:
- Shrink the artifacts.zip file where possible (i.e. make images, LFS objects, etc. smaller)
- Add more parallel runners (https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/8203)
- Deploy the Review Apps to object storage directly from the build step
- Send a delta of the artifacts instead of the full blown copy?
- Can we make the Review App an optional build step (i.e. you can merge without it passing?)
@axil @alejandro What do you think?
Edited by Stan Hu