Skip to content

Add data migration to remove ai catalog flow records

What does this MR do and why?

The AI Catalog Flow is behind a feature flag (FF), and only some internal team members enabled this FF to test how it works. As a result, they created flows in production, which we asked them to delete in Slack channel. Almost all members have deleted their flows; only 2-3 remained from about 20.

Some flows were soft-deleted, so to remove them permanently, we need to run a data migration. This migration will:

  • Delete the item consumers records for flows
  • Remove any dependent records of flows
  • Delete the flow records themselves

Once this data migration runs, there will be no records remaining in production, GDK, or any other environment for flows.

We're introducing support for the new schema version v2 for flow execution, so we need to ensure that any existing records using v1 are removed. For more details, see this.

Query Plans

There are a total of 6 records in the system. Out of these, 4 records are soft-deleted and 2 are active.The 2 active records were created by internal team members in the production environment.

Query plan: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/45181/commands/138594

SELECT
  "ai_catalog_items".*
FROM
  "ai_catalog_items"
WHERE
  "ai_catalog_items"."item_type" = 2
  AND "ai_catalog_items"."deleted_at" IS NULL;

Similarly for ItemVersion there are 6 records. - https://console.postgres.ai/gitlab/gitlab-production-main/sessions/45181/commands/138590

And other tables ai_catalog_item_version_dependencies, duo_workflows_workflows, ai_flow_triggers, ai_catalog_item_consumers are having minimal records.

References

Screenshots or screen recordings

Before After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #578228

Edited by Jaydip Pansuriya

Merge request reports

Loading