Schedule gitlab-qa-sandbox-groups subgroups for cleanup once a month
Summary
Subgroups in gitlab-qa-sandbox-groups-* pile up over time and cause errors in e2e tests that block deployments. This has happened numerous times in the past. I propose these subgroups should be automatically deleted on a regular basis as a catch-all for any dangling test resources that aren't cleaned up.
Context
Another incident occurred where the fix was to delete excessive subgroups in gitlab-qa-sandbox-group-4 (the one in use that day). These subgroups are leftover from automated testing for various reasons and pile up over time. This can cause errors in specs that block deployments such as the one in the linked incident above.
Corrective Action --- NOT needed to be completed prior to the next upgrade scheduled for 2023-09-09
-
Update the Run Cleanup with Permanent Subgroup Deletion job to include all gitlab-qa-sandbox-groups as it currently only deletes subgroups from one gitlab-qa-sandbox-group at a time as specified by an environment variable that has to be changed for each group. -
Add delete_user_projects
script to the cleanup pipeline above -
Run these scripts monthly to prevent deployment issues due to too many subgroups.
Edited by Désirée Chevalier