Container Registry cleanup policy policy doesn't remove images
Summary
On our self hosted gitlab instance, we have enabled the container registry tag expiration policy for several of our projects, however it does not appear to be cleaning up the registries. This relates to epic &2270 (closed)
Steps to reproduce
-
Enable policy for the project with the following settings;
-
Wait for cronjob to execute
-
Check container registry to confirm images have not been removed;
What is the current bug behavior?
Gitlab does not remove any images that should meet the criteria to be removed
What is the expected correct behavior?
Based on the documentation, Gitlab should keep latest and at least 5 other tags for this image
Relevant logs and/or screenshots
SideKiq pod does not have any errors around the expire policy
2020-06-01T13:50:31.476Z 11 TID-gp9m7nws3 CleanupContainerRepositoryWorker JID-74b01a3006a281a8af66c384 INFO: start 2020-06-01T13:50:31.517Z 11 TID-gp9m7nxaf CleanupContainerRepositoryWorker JID-5d3e758e15c1d2925fee0f97 INFO: start 2020-06-01T13:50:31.531Z 11 TID-gp9m7ny4z CleanupContainerRepositoryWorker JID-eb2b9fccd3eb8a710aa7cdd9 INFO: start 2020-06-01T13:50:31.562Z 11 TID-gp9m7nynb CleanupContainerRepositoryWorker JID-8991a2deaf92206fb010b62e INFO: start 2020-06-01T13:50:31.566Z 11 TID-gp9m7nws3 CleanupContainerRepositoryWorker JID-74b01a3006a281a8af66c384 INFO: done: 0.09 sec 2020-06-01T13:50:31.578Z 11 TID-gp9m7nxt7 CleanupContainerRepositoryWorker JID-b46536c117e1584252dde87b INFO: start 2020-06-01T13:50:31.609Z 11 TID-gp9m7nxaf CleanupContainerRepositoryWorker JID-5d3e758e15c1d2925fee0f97 INFO: done: 0.092 sec 2020-06-01T13:50:31.611Z 11 TID-gp9mugujj CleanupContainerRepositoryWorker JID-0ed159558dc0e5df5b7b2b6b INFO: start 2020-06-01T13:50:31.651Z 11 TID-gp9m7nynb CleanupContainerRepositoryWorker JID-8991a2deaf92206fb010b62e INFO: done: 0.09 sec 2020-06-01T13:50:31.653Z 11 TID-gp9m7nwlz CleanupContainerRepositoryWorker JID-27f04bfe3bf20bf498ab8250 INFO: start 2020-06-01T13:50:31.684Z 11 TID-gp9mugx73 CleanupContainerRepositoryWorker JID-6724de5afa806a1432633423 INFO: start 2020-06-01T13:50:31.726Z 11 TID-gp9m7ny4z CleanupContainerRepositoryWorker JID-eb2b9fccd3eb8a710aa7cdd9 INFO: done: 0.196 sec 2020-06-01T13:50:31.727Z 11 TID-gp9mugujj CleanupContainerRepositoryWorker JID-0ed159558dc0e5df5b7b2b6b INFO: done: 0.116 sec 2020-06-01T13:50:31.730Z 11 TID-gp9mugwyb CleanupContainerRepositoryWorker JID-ea3fd4509e43ee549a51965f INFO: start 2020-06-01T13:50:31.745Z 11 TID-gp9m7nxt7 CleanupContainerRepositoryWorker JID-b46536c117e1584252dde87b INFO: done: 0.167 sec 2020-06-01T13:50:31.751Z 11 TID-gp9m7nwyn CleanupContainerRepositoryWorker JID-088e022f51ccaea2ae728d3a INFO: start 2020-06-01T13:50:31.775Z 11 TID-gp9m7nwlz CleanupContainerRepositoryWorker JID-27f04bfe3bf20bf498ab8250 INFO: done: 0.121 sec 2020-06-01T13:50:31.814Z 11 TID-gp9mugwyb CleanupContainerRepositoryWorker JID-ea3fd4509e43ee549a51965f INFO: done: 0.084 sec 2020-06-01T13:50:31.831Z 11 TID-gp9mugx73 CleanupContainerRepositoryWorker JID-6724de5afa806a1432633423 INFO: done: 0.147 sec 2020-06-01T13:50:31.839Z 11 TID-gp9m7nwyn CleanupContainerRepositoryWorker JID-088e022f51ccaea2ae728d3a INFO: done: 0.088 sec
Results of GitLab environment info
| Component | Version |
|---|---|
| Gitlab | 13.0.3 |
| Gitlab Shell | 13.2.0 |
| Gitlab Workhorse | v8.31.1 |
| Gitlab API | v4 |
| Ruby | 2.6.6p146 |
| Rails | 6.0.3 |
| PostgreSQL | 11.7 |
Possible fixes
Is there a command than can be used to manually trigger the expire policy cronjob so that users can test it without having to wait 24 hours for it to trigger automatically? Is there also any documentation on the format of the regex pattern that gitlab accepts?