Backfill embeddings for work items

Description

Backfill embeddings for work items on gitlab.com: Work items updated within the last year from the gitlab-org/gitlab project.

The migration only runs on gitlab.com and is skipped for other instances.

This is based on !154940 (merged).

Migration running

Running the migration locally and it works well

When there are no projects with the flag enabled

Sets space required to 0B, marks as completed.

Screenshot_2024-10-31_at_10.26.14

When there is a project with the flag enabled and it spans over two batches: 40 docs, batch size 30

First run calculates size correctly, enqueues 30 docs

Screenshot_2024-10-31_at_10.38.44

Second run enqueues 10 docs

Screenshot_2024-10-31_at_10.39.48

Third run marks as completed

Screenshot_2024-10-31_at_10.41.11

If the space required is not enough

It halts the migration

Screenshot_2024-10-31_at_10.54.42

Runtime calculation

!170784 (comment 2186084088)


Related to #479777 (closed).

Edited by Arturo Herrero

Merge request reports

Loading