Prepare async index for project vulnerability_reads on common filters
What does this MR do and why?
This MR creates an index for the vulnerability_reads
table on project_id
, state
, report_type
, severity
, and vulnerability_id
columns asynchronously to fix a timing out query.
The original query will be overridden by the UnnestedInFilters
abstraction automatically once we introduce this index as the index can be utilized for the specialized query.
Related to #413697 (closed)
Problemmatic query
SELECT
"vulnerability_reads".*
FROM
"vulnerability_reads"
WHERE
"vulnerability_reads"."project_id" = 278964
AND "vulnerability_reads"."report_type" IN (6, 2, 5, 3, 1, 0, 4, 99)
AND "vulnerability_reads"."severity" = 4
AND "vulnerability_reads"."state" IN (1, 4)
ORDER BY
"vulnerability_reads"."severity" DESC,
"vulnerability_reads"."vulnerability_id" DESC
LIMIT 21
New query after the index
SELECT
"vulnerability_reads".*
FROM
"vulnerability_reads"
WHERE
"vulnerability_reads"."vulnerability_id" IN (
SELECT
"vulnerability_reads"."vulnerability_id"
FROM
unnest('{6,2,5,3,1,0,4,99}'::smallint[]) AS "report_types" ("report_type"),
unnest('{1,4}'::smallint[]) AS "states" ("state"),
LATERAL (
SELECT
"vulnerability_reads"."project_id",
"vulnerability_reads"."report_type",
"vulnerability_reads"."severity",
"vulnerability_reads"."state",
"vulnerability_reads"."vulnerability_id"
FROM
"vulnerability_reads"
WHERE
"vulnerability_reads"."severity" = 4
AND "vulnerability_reads"."project_id" = 278964
AND (vulnerability_reads."report_type" = "report_types"."report_type")
AND (vulnerability_reads."state" = "states"."state")
ORDER BY
"vulnerability_reads"."severity" DESC,
"vulnerability_reads"."vulnerability_id" DESC
LIMIT 21) AS vulnerability_reads
ORDER BY
"vulnerability_reads"."severity" DESC,
"vulnerability_reads"."vulnerability_id" DESC
LIMIT 21)
ORDER BY
"vulnerability_reads"."severity" DESC,
"vulnerability_reads"."vulnerability_id" DESC
LIMIT 21
https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/19183/commands/63327
rake command outputs
up
main: == [advisory_lock_connection] object_id: 228420, pg_backend_pid: 55811
main: == 20230619134106 PrepareIndexForVulnerabilityReadsOnCommonProjectFilters: migrating
main: -- index_exists?(:vulnerability_reads, [:project_id, :state, :report_type, :severity, :vulnerability_id], {:order=>{:severity=>:desc, :vulnerability_id=>:desc}, :name=>"index_project_vulnerability_reads_common_finder_query_desc", :algorithm=>:concurrently})
main: -> 0.0156s
main: -- add_index_options(:vulnerability_reads, [:project_id, :state, :report_type, :severity, :vulnerability_id], {:order=>{:severity=>:desc, :vulnerability_id=>:desc}, :name=>"index_project_vulnerability_reads_common_finder_query_desc", :algorithm=>:concurrently})
main: -> 0.0006s
main: == 20230619134106 PrepareIndexForVulnerabilityReadsOnCommonProjectFilters: migrated (0.1872s)
main: == [advisory_lock_connection] object_id: 228420, pg_backend_pid: 55811
down
main: == [advisory_lock_connection] object_id: 228480, pg_backend_pid: 55274
main: == 20230619134106 PrepareIndexForVulnerabilityReadsOnCommonProjectFilters: reverting
main: == 20230619134106 PrepareIndexForVulnerabilityReadsOnCommonProjectFilters: reverted (0.1316s)
main: == [advisory_lock_connection] object_id: 228480, pg_backend_pid: 55274
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Thiago Figueiró