Skip to content

Migrate away `ci_builds.trace`

Description

We might be able to remove ci_builds.trace completely after we store this data somewhere in a safe place, if it is not stored in object storage already.

The data was already migrated by https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/3658#note_63677251 and 0d00d02e

The column removal needs to be done across three releases to avoid downtime:

Progress

  1. Ignore the column - done in %15.4 - !96865 (merged)
  2. Drop the column from the database - done in %15.5 - !98646 (merged)
  3. Remove the ignore rule - in review %15.6 - !101744 (merged)

Old content

I did check on GitLab.com how much data is stored in ci_builds.trace column. This was used when GitLab CI was still separate product. We then moved to storing traces on disk. According to my queries, it is:

$ Ci::Build.where.not(trace: nil).count
=> 87636
$ Ci::Build.where.not(trace: nil).pluck('sum(octet_length(trace))')
=> [5608799010]

It is over 5.5GB, on average over 60k per build.

We should prepare Background Migration to move that to disk.

Edited by Marius Bobin