Optimize projects_service_active queries
What does this MR do?
Optimizes the projects_*_active
counter queries for batch counting in usage data
Part of issue #208923 (closed)
Query
SELECT COUNT(“services”.“id”) FROM “services” WHERE “services”.“type” = ‘JiraService’ AND “services”.“active” = true AND “services”.“template” = false AND “services”.“id” BETWEEN 0 AND 9999;
Optimization
CREATE CONCURRENT INDEX index_services_on_type_and_id_and_template ON public.services USING btree (type, id, template) WHERE (active = true);
-- The query has been executed. Duration: 10.803 s
https://explain.depesz.com/s/FsVe
Before: Bad!https://explain.depesz.com/s/CqcL
After: Good!Timing
After the index for batch counting takes 1 second pessimistic
- ~3.6M services,
- with 10_000 batch sizes
- 3.6M/10_000 = 360 loops
- Time: < 0.5 ms ( cold cache with no time constraint )
Migration output
VERBOSE=true bundle exec rake db:migrate:up VERSION=20200312125121
== 20200312125121 AddIndexOnActiveAndTemplateAndTypeAndIdToServices: migrating
-- transaction_open?()
-> 0.0000s
-- index_exists?(:services, [:type, :id, :template], {:where=>"active = TRUE", :algorithm=>:concurrently})
-> 0.0033s
-- execute("SET statement_timeout TO 0")
-> 0.0001s
-- add_index(:services, [:type, :id, :template], {:where=>"active = TRUE", :algorithm=>:concurrently})
-> 0.0032s
-- execute("RESET ALL")
-> 0.0001s
== 20200312125121 AddIndexOnActiveAndTemplateAndTypeAndIdToServices: migrated (0.0069s)
VERBOSE=true bundle exec rake db:migrate:down VERSION=20200312125121
== 20200312125121 AddIndexOnActiveAndTemplateAndTypeAndIdToServices: reverting
-- transaction_open?()
-> 0.0000s
-- index_exists?(:services, [:type, :id, :template], {:where=>"active = TRUE", :algorithm=>:concurrently})
-> 0.0035s
-- execute("SET statement_timeout TO 0")
-> 0.0001s
-- remove_index(:services, {:where=>"active = TRUE", :algorithm=>:concurrently, :column=>[:type, :id, :template]})
-> 0.0046s
-- execute("RESET ALL")
-> 0.0002s
== 20200312125121 AddIndexOnActiveAndTemplateAndTypeAndIdToServices: reverted (0.0085s)
Edited by Alex Buijs