Backend: Query cancelled due to statement timeout when destroying a scheduled pipeline

Summary

One customer is unable to delete one specific scheduled pipeline. After they hit "Delete" button, they get 500 error after a small waiting period. We narrowed it down to the following statement timeout:

ERROR:  canceling statement due to statement timeout
CONTEXT:  SQL statement "UPDATE ONLY "public"."ci_pipelines" SET "pipeline_schedule_id" = NULL WHERE $1 OPERATOR(pg_catalog.=) "pipeline_schedule_id"" 

Reported by the customer in ZenDesk ticket https://gitlab.zendesk.com/agent/tickets/207445 [internal].

Steps to reproduce

Unknown

Example Project

NA

What is the current bug behavior?

The customer gets 500 error after hitting the delete button for this schedule.

What is the expected correct behavior?

The schedule should get deleted successfully.

Relevant logs and/or screenshots

PG::QueryCanceled

ERROR:  canceling statement due to statement timeout
CONTEXT:  SQL statement "UPDATE ONLY "public"."ci_pipelines" SET "pipeline_schedule_id" = NULL WHERE $1 OPERATOR(pg_catalog.=) "pipeline_schedule_id""

https://sentry.gitlab.net/gitlab/gitlabcom/issues/2585448/events/40553623/

Output of checks

This bug happens on GitLab.com

Possible fixes

A possible solution is to update the pipelines in batches setting pipeline_schedule_id to null prior to the ci_pipeline_schedule destruction.

Edited by Mark Nuzzo