Trace Chunks: Implement use of new store and migration flags in GitLab
We can start writing trace chunks to the new on a per chunk basis: the Ci::BuildTraceChunk
model can keep track of which redis the data is stored in (either in a new column, or using the existing data_store
with a new store type).
When a new Ci::BuildTraceChunk
is created, it uses a feature flag to decide which Redis instance to store the data in. This allows us to roll out per project.
For all chunks started on any instance, we'll continue reading/writing from that instance, regardless of the feature flag state. This does mean that any Ci::BuildTraceChunk
started in the new redis, will need to finish there.
For all operations append
(eval
), set
, get
, strlen
, del
we should have counters identifiable as belonging to trace-chunking, and labelled with the instance that fulfilled the operation. Because we keep track of the chunks in the database, we can verify if that there are no live ones left going to redis-persistent.
All Redis operations for trace chunks live in Ci::BuildTraceChunks::Redis
.
Status 2021-06-15
The code to switch Redis instance via feature flag has been merged. An improvement to the feature flag is in review gitlab-org/gitlab!63865 (merged).