Skip to content

Optimize database query for last deployment

Shinya Maeda requested to merge optimize-query-for-last-deployment into master

What does this MR do?

This MR fixes the query statement timeout issue on Environment#last_deployment.

Currently, the executed query looks like this:

SELECT
    "deployments".*
FROM
    "deployments"
WHERE
    "deployments"."environment_id" = 693272
    AND "deployments"."status" = 2
ORDER BY
    deployments.id DESC
LIMIT
    1;

which produces the following query plan on db-lab:

 Limit  (cost=0.57..3.38 rows=1 width=139) (actual time=64952.563..64952.566 rows=1 loops=1)
   Buffers: shared hit=1789 read=70646 dirtied=482
   I/O Timings: read=64229.280
   ->  Index Scan using index_deployments_on_environment_id_and_id on public.deployments  (cost=0.57..109970.76 rows=39200 width=139) (actual time=64952.560..64952.561 rows=1 loops=1)
         Index Cond: (deployments.environment_id = 693272)
         Filter: (deployments.status = 2)
         Rows Removed by Filter: 72178
         Buffers: shared hit=1789 read=70646 dirtied=482
         I/O Timings: read=64229.280

It seems index_deployments_on_environment_id_status_and_finished_at is not used. Probably because it's sorted by deployments.id.

In this MR, we'll introduce a new index on (environment_id, status, id). With this index, the query plan will look like:

 Limit  (cost=0.57..1.68 rows=1 width=139) (actual time=0.664..0.665 rows=1 loops=1)
   Buffers: shared hit=4 read=4
   I/O Timings: read=0.522
   ->  Index Scan using dosuken_test on public.deployments  (cost=0.57..43722.10 rows=39304 width=139) (actual time=0.658..0.658 rows=1 loops=1)
         Index Cond: ((deployments.environment_id = 693272) AND (deployments.status = 2))
         Buffers: shared hit=4 read=4
         I/O Timings: read=0.522

It's fast enough with cold cacche.

Migration

INFO: This script is a predefined script in devkitkat.
== 20210331145548 AddIndexForLastDeployment: reverting ========================
-- transaction_open?()
   -> 0.0000s
-- indexes(:deployments)
   -> 0.0089s
-- remove_index(:deployments, {:algorithm=>:concurrently, :name=>"index_deployments_on_environment_id_status_and_id"})
   -> 0.0047s
== 20210331145548 AddIndexForLastDeployment: reverted (0.0144s) ===============

== 20210331145548 AddIndexForLastDeployment: migrating ========================
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:deployments, [:environment_id, :status, :id], {:name=>"index_deployments_on_environment_id_status_and_id", :algorithm=>:concurrently})
   -> 0.0059s
-- add_index(:deployments, [:environment_id, :status, :id], {:name=>"index_deployments_on_environment_id_status_and_id", :algorithm=>:concurrently})
   -> 0.0083s
== 20210331145548 AddIndexForLastDeployment: migrated (0.0152s) ===============

Closes #325552 (closed)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Mayra Cabrera

Merge request reports