Skip to content

Slow query to count pending mirror updates

We are seeing slow query on GitLab.com (at about 1,500ms) which seemingly is involved in counting pending mirror updates (see https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/6678#note_175302706 for context):

SELECT
    COUNT(*)
FROM
    projects
    INNER JOIN project_mirror_data ON project_mirror_data.project_id = projects.id
    INNER JOIN namespaces ON projects.namespace_id = namespaces.id
    LEFT JOIN plans ON namespaces.plan_id = plans.id
WHERE
    projects.mirror = TRUE
    AND project_mirror_data.retry_count <= 14
    AND (projects.visibility_level = 20
        OR plans.name IN ('early_adopter', 'bronze', 'silver', 'gold'))
    AND project_mirror_data.status NOT IN ('scheduled',
        'started')
    AND project_mirror_data.last_update_at >= NOW() - '30 seconds'::interval;

Plan: https://explain.depesz.com/s/SQtT

  • It looks like we lack a proper/complete index to satisfy the last_update_at > ? condition.
  • Note we cannot answer the status <> ALL(...) from an index. We might want to explicitly enumerate the status values we're interested in here instead.

cc @DouweM

Edited by Andreas Brandl