Avoid partition scan in Geo skip-enqueue check

What does this MR do and why?

Gitlab::Geo::LogCursor::Events::Event#skip_enqueue? added in !245426 needs to check whether a specific blob record is stored locally.

That per-event lookup was measured at ~13-15ms with a warm cache, which risks becoming a bottleneck for LogCursor's single-threaded processing loop at >100 events/second. Investigation showed the cost is almost entirely Postgres planning time: p_ci_job_artifacts (partitioned by partition_id) and uploads (partitioned by model_type) can't be pruned by a bare id lookup, so the planner builds an Append across every partition before even running the (trivially fast) per-partition scan.

This change threads the relevant partition key through the Geo event payload at creation time, where it's already in memory on the replicator. skip_enqueue? can scope its lookup by it and let Postgres prune to a single partition. Non-partitioned replicable tables are unaffected, since the extra WHERE clauses are only added when the payload carries a partition key.

Database queries

SELECT 1 AS one FROM "uploads" WHERE "uploads"."id" = 1197913122 AND "uploads"."model_type" = 'Project' LIMIT 1

Query plan: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/53884/commands/156210

SELECT 1 AS one FROM "p_ci_job_artifacts" WHERE "p_ci_job_artifacts"."id" = 18028112522 AND "p_ci_job_artifacts"."partition_id" = 100

Query plan: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/53884/commands/156213

With at most 2/3ms per query, >100events per second won't become a bottleneck for LogCursor's single-threaded processing loop, and won't cause a queue buildup. This fix is safe.

References

Fixes Geo: Logcursor enqueues blob events through ful... (#593115)

How to set up and validate locally

See !245426. The behaviour should be unchanged.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Chloe Fons

Merge request reports

Loading