ActiveRecord::QueryCanceled: PG::QueryCanceled: ERROR: canceling statement due to statement timeout

https://sentry.gitlab.net/gitlab/gitlabcom/issues/963781/

PG::QueryCanceled: ERROR:  canceling statement due to statement timeout

  active_record/connection_adapters/postgresql_adapter.rb:611:in `async_exec_params'
    @connection.exec_params(sql, type_casted_binds)
  active_record/connection_adapters/postgresql_adapter.rb:611:in `block (2 levels) in exec_no_cache'
    @connection.exec_params(sql, type_casted_binds)
  active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
    yield
  active_support/concurrency/share_lock.rb:187:in `yield_shares'
    yield
  active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
    @lock.yield_shares(compatible: [:load]) do
...
(171 additional frame(s) were not displayed)

ActiveRecord::QueryCanceled: PG::QueryCanceled: ERROR:  canceling statement due to statement timeout
: SELECT "merge_request_diff_files".* FROM "merge_request_diff_files" WHERE "merge_request_diff_files"."merge_request_diff_id" = 55854132 ORDER BY "merge_request_diff_files"."merge_request_diff_id" ASC, "merge_request_diff_files"."relative_order" ASC

ActiveRecord::QueryCanceled: PG::QueryCanceled: ERROR:  canceling statement due to statement timeout
: SELECT "merge_request_diff_files".* FROM "merge_request_diff_files" WHERE "merge_request_diff_files"."merge_request_diff_id" = 55854132 ORDER BY "merge_request_diff_files"."merge_request_diff_id" ASC, "merge_request_diff_files"."relative_order" ASC

added typebug label

Bronze customer hit this error when trying to export their project.

Couldn't find the exact sentry event relevant to the customer's project, but this sentry error (in description) looks very similar.

Internal links:

• ticket: https://gitlab.zendesk.com/agent/tickets/133052
• Kibana:
  • specific error: https://log.gitlab.net/app/kibana#/doc/AWOSvARQwig0Nc2UGcr2/pubsub-rails-inf-gprd-2019.09.23-000003/doc/?id=AW1cFV84n8X3P-EK4KPM
  • specific debug: https://log.gitlab.net/app/kibana#/doc/AWOSvARQwig0Nc2UGcr2/pubsub-rails-inf-gprd-2019.09.23-000003/doc/?id=AW1cFV84n8X3P-EK4KPN

error message from Kibana:

PG::QueryCanceled: ERROR: canceling statement due to statement timeout : SELECT "merge_request_diff_files".* FROM "merge_request_diff_files" WHERE "merge_request_diff_files"."merge_request_diff_id" IN (55315210, 54189762, 54179953, 54177718, 54167890, 54166914, 54015754, 54014741, 54011919, 54010461, 53999486, 53982460, 53981297, 53863804, 53563124, 53553067, 53551743, 53438111, 53434692, 53428670, 53425014, 53423223, 53422502, 53420942, 53418780, 53304176, 53304048, 53303653, 53303517, 53303445, 53303329, 53302766, 53279086, 53269184, 52918035, 52905665, 52858365, 52850950, 52801351, 52797226, 52787735, 52647742, 52642771, 52641214, 52629001, 52627484, 52481540, 52480711, 52460397, 52459634, 52458008, 52457477, 52456342, 52455983, 52450007, 52427582, 52404546, 52278087, 52273581, 52271622, 52269017, 52259835, 52116315, 52111460, 52105831, 52072827, 52056441, 52053254, 52033712, 52000296, 51997458, 51989471, 51988297, 51986681, 51983638, 51975934, 51950713, 41454635, 35484170, 35479185, 35477245, 35468583, 35457782, 35453434, 35446908, 35427977, 35421869, 34460456, 34348205, 34344322, 34343329, 34342986, 34340508, 34340265, 34299657, 34264469, 34260692, 34260405, 34260367, 34257672, 34257087, 34250710, 34233095, 34227254, 34221888, 30776712, 30775046, 30774125, 25488801, 25268880, 25267745, 25267744, 17616725, 17616724, 17616723) ORDER BY "merge_request_diff_files"."merge_request_diff_id" ASC, "merge_request_diff_files"."relative_order" ASC

@jeremy or @lmcandrew can we get someone to investigate this for the customer? suggested workaround would be great even if we don't have a fix

added DEPRECATED_project export customer devopsmanage severity2 + 1 deleted label

added needs investigation label

@lmcandrew, do you have a perspective on why we've run into so many import issues recently? Let me know if you have any thoughts on how we should handle these S2 issues.

Do you feel like #30804 (closed) would help mitigate some of these issues? We can't keep firefighting like this with only 1 engineer on Import.

Noted down some thoughts here: https://gitlab.com/gitlab-org/manage/issues/70

mentioned in issue #32974 (closed)

mentioned in issue #33500 (closed)

mentioned in issue #33958 (closed)

mentioned in issue #34465 (closed)

mentioned in issue #35029 (closed)

mentioned in issue #35482 (closed)

mentioned in issue #35976 (closed)

mentioned in issue #36567 (closed)

mentioned in issue #37182 (closed)

mentioned in issue #37860 (closed)

mentioned in issue #39002 (closed)

mentioned in issue #103403 (closed)

mentioned in issue #121480 (closed)

mentioned in issue #193027 (closed)

mentioned in issue #195099 (closed)

mentioned in issue #196463 (closed)

mentioned in issue #197797 (closed)

mentioned in issue #195981 (closed)

mentioned in issue #198979 (closed)

mentioned in issue #201628 (closed)

mentioned in issue #204651 (closed)

mentioned in issue #206814 (closed)

mentioned in issue #207751 (closed)

mentioned in issue #208613 (closed)

mentioned in issue #209697 (closed)

mentioned in issue #211273 (closed)

mentioned in issue #212097 (closed)

mentioned in issue #212790 (closed)

mentioned in issue #212803 (closed)

mentioned in issue #213478 (closed)

mentioned in issue #214170 (closed)

mentioned in issue #214940 (closed)

mentioned in issue #215806 (closed)

mentioned in issue #216401 (closed)

mentioned in issue #217261 (closed)

mentioned in issue #218179 (closed)

mentioned in issue #218922 (closed)

mentioned in issue #219675 (closed)

mentioned in issue #220709 (closed)

mentioned in issue #222209 (closed)

mentioned in issue #223628 (closed)

mentioned in issue #225109 (closed)

mentioned in issue #226936 (closed)

mentioned in issue #230635 (closed)

mentioned in issue #232356 (closed)

mentioned in issue #233311 (closed)

mentioned in issue #235066 (closed)

mentioned in issue #237804 (closed)

mentioned in issue #240835 (closed)

mentioned in issue #243449 (closed)

added triage question label

@cynthia Have we seen any recent reoccurrences of this issue? There have been significant changes to the code since this was logged and I would like to make sure that the issue is still relevant before we spend time on it. Thanks!

@hdelalic thanks for checking in. It was a slightly different error but I think this was resolved in #212355 (closed) so I'll go ahead and mark this as a duplicate

added Category:Importers label

added to epic &3054

marked this issue as a duplicate of #212355 (closed)

closed

marked this issue as related to #212355 (closed)

added Importer:Project Export/Import label

added groupimport and integrate label and removed 1 deleted label