Skip to content

Optimize usage ping queries by using batch counting

What does this MR do?

Optimizes the following queries to use batch counting. Before this fix, turning batch counting on would produce an invalid statement:

users_with_projects_added

New queries:

SELECT MIN("users_ops_dashboard_projects"."user_id") FROM "users_ops_dashboard_projects" INNER JOIN "users" ON "users"."id" = "users_ops_dashboard_projects"."user_id" WHERE ("users"."state" IN ('active')) AND (ghost IS NOT TRUE) AND ("users"."user_type" IS NULL OR "users"."user_type" NOT IN (2, 1, 3))
SELECT MAX("users_ops_dashboard_projects"."user_id") FROM "users_ops_dashboard_projects" INNER JOIN "users" ON "users"."id" = "users_ops_dashboard_projects"."user_id" WHERE ("users"."state" IN ('active')) AND (ghost IS NOT TRUE) AND ("users"."user_type" IS NULL OR "users"."user_type" NOT IN (2, 1, 3))
SELECT COUNT(DISTINCT "users_ops_dashboard_projects"."user_id") FROM "users_ops_dashboard_projects" INNER JOIN "users" ON "users"."id" = "users_ops_dashboard_projects"."user_id" WHERE ("users"."state" IN ('active')) AND (ghost IS NOT TRUE) AND ("users"."user_type" IS NULL OR "users"."user_type" NOT IN (2, 1, 3)) AND "users_ops_dashboard_projects"."user_id" BETWEEN 0 AND 9999

Plan: https://explain.depesz.com/s/sXBI

projects_with_packages

New queries:

SELECT MIN("packages_packages"."project_id") FROM "packages_packages"
SELECT MAX("packages_packages"."project_id") FROM "packages_packages"
SELECT COUNT(DISTINCT "packages_packages"."project_id") FROM "packages_packages" WHERE "packages_packages"."project_id" BETWEEN 1 AND 10000

Plan: https://explain.depesz.com/s/ngKs

operations_dashboard_users_with_projects_added

New queries:

SELECT MIN("users_ops_dashboard_projects"."user_id") FROM "users_ops_dashboard_projects" INNER JOIN "users" ON "users"."id" = "users_ops_dashboard_projects"."user_id" WHERE ("users"."state" IN ('active')) AND (ghost IS NOT TRUE) AND ("users"."user_type" IS NULL OR "users"."user_type" NOT IN (2, 1, 3)) AND "users_ops_dashboard_projects"."created_at" BETWEEN 2020-02-20 16:31:06.587165 AND 2020-03-19 16:31:06.587255
SELECT MAX("users_ops_dashboard_projects"."user_id") FROM "users_ops_dashboard_projects" INNER JOIN "users" ON "users"."id" = "users_ops_dashboard_projects"."user_id" WHERE ("users"."state" IN ('active')) AND (ghost IS NOT TRUE) AND ("users"."user_type" IS NULL OR "users"."user_type" NOT IN (2, 1, 3)) AND "users_ops_dashboard_projects"."created_at" BETWEEN 2020-02-20 16:31:06.587165 AND 2020-03-19 16:31:06.587255
SELECT COUNT(DISTINCT "users_ops_dashboard_projects"."user_id") FROM "users_ops_dashboard_projects" INNER JOIN "users" ON "users"."id" = "users_ops_dashboard_projects"."user_id" WHERE ("users"."state" IN ('active')) AND (ghost IS NOT TRUE) AND ("users"."user_type" IS NULL OR "users"."user_type" NOT IN (2, 1, 3)) AND "users_ops_dashboard_projects"."created_at" BETWEEN 2020-02-20 16:31:06.587165 AND 2020-03-19 16:31:06.587255 AND "users_ops_dashboard_projects"."user_id" BETWEEN 0 AND 9999

Plan: https://explain.depesz.com/s/yJYE

It also converts count(services, batch: false) to services.size, since we are dealing with Array objects here, not ActiveRecord::Relation objects.

Part of issue #208923 (closed)

Edited by Alex Buijs

Merge request reports