R&D: Analyze and propose a solution to improve clean-up of the runner job environment - Docker Executor

Overview

In GitLab 13.9 we added the clear-docker-cache script to clean up old build images left behind by previous CI jobs when using the Docker executor.

However, there are still gaps with this feature:

1. The clear-docker-cache script only addresses images, not volumes. #27332
2. The clear-docker-cache script only does not clean up all dangling images (untagged images). #29518

As shown in this blog post, the GitLab community and customers have created their own solutions for cleaning up the build environment when using Runner + Docker executor.

R&D Tasks/Deliverable

• Determine if there is feasible technical solution that will add a smart cache clean-up feature to runner.
• If yes, then document the implementation proposal so that we can prioritize the feature development work in a future milestone.

Note: we do not expect feature delivery as the output of this issue.