Skip to content

Fix 500 for pipeline charts page

Max Orefice requested to merge mo-add-index-to-ci-pipeline into master

Ref: #31528 (closed)

What does this MR do?

This MR creates a new index to our ci_pipelines table.

It prevents our https://gitlab.com/gitlab-org/gitlab-foss/pipelines/charts page to timeout and returns a 500.

#database-lab query Execution plan
https://gitlab.slack.com/archives/CLJMDRD8C/p1595992741387000 https://explain.depesz.com/s/cqYv

As mentioned by @dosuken123, the query looks like this and takes almost 5 mins (timeout is 15 sec).

SELECT
    COUNT("ci_pipelines"."created_at") AS count_created_at,
    DATE(ci_pipelines.created_at) AS date_ci_pipelines_created_at
FROM
    "ci_pipelines"
WHERE
    "ci_pipelines"."project_id" = 278964
    AND (
        '2020-07-29 23:59:59.999999' > ci_pipelines.created_at
        AND ci_pipelines.created_at > '2020-07-22 00:00:00'
    )
GROUP BY
    DATE(ci_pipelines.created_at)

Index creation

exec CREATE INDEX index_ci_pipelines_on_project_id_and_created_at ON public.ci_pipelines USING btree (project_id, created_at DESC)

=> The query has been executed. Duration: 15.220 min (edited) 
== 20200729202222 AddIndexToCiPipelineProjectIdCreatedAt: migrating ===========
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:ci_pipelines, [:project_id, :created_at], {:order=>{:created_at=>:desc}, :algorithm=>:concurrently})
   -> 0.0077s
-- add_index(:ci_pipelines, [:project_id, :created_at], {:order=>{:created_at=>:desc}, :algorithm=>:concurrently})
   -> 0.0051s
== 20200729202222 AddIndexToCiPipelineProjectIdCreatedAt: migrated (0.0132s) ==

Index rollback

-- transaction_open?()
   -> 0.0000s
-- index_exists?(:ci_pipelines, [:project_id, :created_at], {:order=>{:created_at=>:desc}, :algorithm=>:concurrently})
   -> 0.0081s
-- remove_index(:ci_pipelines, {:order=>{:created_at=>:desc}, :algorithm=>:concurrently, :column=>[:project_id, :created_at]})
   -> 0.0076s
== 20200729202222 AddIndexToCiPipelineProjectIdCreatedAt: reverted (0.0161s) ==

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Max Orefice

Merge request reports