Migrate away `ci_builds.trace`
Description
We might be able to remove ci_builds.trace completely after we store this data somewhere in a safe place, if it is not stored in object storage already.
The data was already migrated by https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/3658#note_63677251 and 0d00d02e
The column removal needs to be done across three releases to avoid downtime:
Progress
- Ignore the column - done in %15.4 - !96865 (merged)
- Drop the column from the database - done in %15.5 - !98646 (merged)
- Remove the ignore rule - in review %15.6 - !101744 (merged)
Old content
I did check on GitLab.com how much data is stored in ci_builds.trace column. This was used when GitLab CI was still separate product. We then moved to storing traces on disk. According to my queries, it is:
$ Ci::Build.where.not(trace: nil).count
=> 87636
$ Ci::Build.where.not(trace: nil).pluck('sum(octet_length(trace))')
=> [5608799010]
It is over 5.5GB, on average over 60k per build.
We should prepare Background Migration to move that to disk.
Edited by Marius Bobin