Add `deploy-cleanup-old-deleted-files` scheduled job
What does this MR do?
- Add
deploy-cleanup-old-deleted-files
scheduled job (which can also be run manually on-demand)- Supports independent per-site production deploys by removing need for
--delete
flag on master GCP deploy rsync. - Introduces a separate "
deploy-cleanup-old-deleted-files
" scheduled job which does a full build of everything and deploys with theDEPLOY_CLEANUP_OLD_DELETED_FILES
flag set. - This will use all artifacts from partial build jobs to ensure everything is still cleaned up, even if partial build jobs are added/deleted/refactored, and when top-level paths of the deploy are deleted/renamed.
- As part of this, the staging app deploy will be changed to a manual job. See discussion on slack. Also update related docs.
- This also resolves #7661 (closed)
- Supports independent per-site production deploys by removing need for
Tasks
-
Write job -
Remove DEPLOY_CLEANUP_OLD_DELETED_FILES
flag fromdeploy
-
...and add to deploy_staging
job and make it invoked manually (see slack thread)
-
-
Update docs about staging app deploy being switched to manual: https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/doc/redirects_on_about_gitlab_com.md -
Set up schedule in CI config (Daily at 2PM UTC to start, can be changed to more frequently if needed). -
Ensure it can be triggered manually too -
Test with fake config in MR -
Review cleanup job in prod after merge (Successful run via scheduled job: https://gitlab.com/gitlab-com/www-gitlab-com/-/jobs/651326382) -
Test manual staging deploy in prod after merge (https://gitlab.com/gitlab-com/www-gitlab-com/-/jobs/651326376)
Related issues
- Relates #8302 (closed)
- Resolves #7661 (closed)
- Resolves #8337 (closed)
Edited by Chad Woolley