Deprecate retention of build metadata older than 3 months
Problem to solve
There is a large footprint on CI on large scale systems
Further details
This MR: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21464 introduces a concept of "archived jobs".
Proposal
To reduce the footprint of CI on large scale system we should consider archiving jobs older than some interval: a good compromise seems to be 3 months. After that time the only way to run this pipeline again is to recreate it.
It has implication on:
- older jobs will no longer be allowed to be "played" nor "retired",
- rollback (on environment) functionality on older jobs will stop working.
I also consider that all import/export builds will not have retry/rollback functionality.
I consider imposing soft-archiving now, hardcoding 3 months interval in a product, implementing that behind a feature flag, enabled by default to measure the impact of that change on our customers. We could gather feedback if such change is welcomed or not.
An operational task to start actually deleting data older than 3 months (or some age) on some cadence is a task to be created later.
We might want to add some banner on Job Page with Archived information, similar to failure reason.
Likely this banner should represent a warning sign, saying:
-
This job is archived. The play and retry functionality is not available. To run this job, schedule that pipeline again.
Solution
Should be sticky when scrolling down
What does success look like, and how can we measure that?
We reduced the footprint of CI on large scale system by X
TODO: X still to be defined
Links / references
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21464