Unable to process merge request / Error in PostgreSQL ?

Summary

Unable to process MR by a specific account on all projects. Other accounts can merge normally.

Steps to reproduce

1. Open an active MR
2. Try to valide/refuse the MR

What is the current bug behavior?

The action button is disabled and I get infinite spinner on certain fields (participants, assignees). It happens only with a specific account, who has the correct permissions to process MR. Other accounts can process the MR as usual. Only the "Close MR" button is enabled but then of course the MR is not processed.

What is the expected correct behavior?

The account should be able to process MR.

Relevant logs and/or screenshots

I don't know if it's related but the following are logged :

In sidekiq logs:

{"severity":"WARN","time":"2020-04-21T11:59:12.744Z","error_class":"ActiveRecord::StatementInvalid","error_message":"PG::UndefinedColumn: ERROR:  **column merge_requests.state does not exist**\nLINE 1: ...HERE \"merge_requests\".\"id\" BETWEEN 1 AND 4227 AND \"merge_req...\n                                                             ^\n","context":"Job raised exception","jobstr":"{\"class\":\"BackgroundMigrationWorker\",\"args\":[\"ResetMergeStatus\",[1,4227]],\"retry\":3,\"queue\":\"background_migration\",\"jid\":\"aff7bf0ca421a0da3e726aea\",\"created_at\":1587470229.345452,\"meta.caller_id\":\"ResetMergeStatus\",\"correlation_id\":\"e0d9497741c3a139de804b493c6b8e3c\",\"duplicate-of\":\"c1f2f7788694b7de1cc54c7f\",\"enqueued_at\":1587470352.7255466}","class":"BackgroundMigrationWorker","args":["ResetMergeStatus",[1,4227]],"retry":3,"queue":"background_migration","jid":"aff7bf0ca421a0da3e726aea","created_at":"2020-04-21T11:57:09.345Z","meta.caller_id":"ResetMergeStatus","correlation_id":"e0d9497741c3a139de804b493c6b8e3c","duplicate-of":"c1f2f7788694b7de1cc54c7f","enqueued_at":"2020-04-21T11:59:12.725Z","error_backtrace":["lib/gitlab/background_migration/reset_merge_status.rb:13:in `perform'","lib/gitlab/background_migration.rb:58:in `perform'","app/workers/background_migration_worker.rb:29:in `block in perform'","app/workers/concerns/worker_context.rb:63:in `block in with_context'","lib/gitlab/application_context.rb:46:in `block in use'","lib/gitlab/application_context.rb:46:in `use'","app/workers/concerns/worker_context.rb:63:in `with_context'","app/workers/background_migration_worker.rb:25:in `perform'","lib/gitlab/with_request_store.rb:7:in `with_request_store'","lib/gitlab/sidekiq_daemon/monitor.rb:49:in `within_job'"]}

In postgresql logs:

2020-04-21_11:59:12.73927 ERROR:  column merge_requests.state does not exist at character 109
2020-04-21_11:59:12.73928 STATEMENT:  UPDATE "merge_requests" SET "merge_status" = 'unchecked' WHERE "merge_requests"."id" BETWEEN 1 AND 4227 AND "merge_requests"."state" = 'opened' AND "merge_requests"."merge_status" = 'can_be_merged'

Output of checks

Results of GitLab environment info

System information
System:		Ubuntu 16.04
Current User:	git
Using RVM:	no
Ruby Version:	2.6.5p114
Gem Version:	2.7.10
Bundler Version:1.17.3
Rake Version:	12.3.3
Redis Version:	5.0.7
Git Version:	2.24.1
Sidekiq Version:5.2.7
Go Version:	unknown
GitLab information
Version:	12.9.2
Revision:	ac5568eb5d8
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	10.12
URL:		https://gitlab.mycompany.com
HTTP Clone URL:	https://gitlab.mycompany.com/some-group/some-project.git
SSH Clone URL:	git@gitlab.mycompany.com:some-group/some-project.git
Using LDAP:	yes
Using Omniauth:	yes
Omniauth Providers:
GitLab Shell
Version:	12.0.0
Repository storage paths:

• default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 12.0.0 ? ... OK (12.0.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... Server: ldapmain
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
User output sanitized. Found 39 users of 100 limit.
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
43/5 ... yes
7/8 ... yes
10/9 ... yes
10/10 ... yes
10/11 ... yes
15/12 ... yes
15/13 ... yes
15/14 ... yes
17/16 ... yes
33/17 ... yes
12/18 ... yes
12/19 ... yes
12/20 ... yes
12/21 ... yes
46/22 ... yes
46/23 ... yes
48/24 ... yes
24/25 ... yes
26/28 ... yes
61/31 ... yes
60/33 ... yes
1/37 ... yes
1/38 ... yes
1/40 ... yes
1/41 ... yes
1/42 ... yes
57/49 ... yes
57/50 ... yes
1/51 ... yes
1/52 ... yes
1/53 ... yes
1/54 ... yes
1/55 ... yes
56/57 ... yes
56/58 ... yes
55/59 ... yes
55/60 ... yes
1/61 ... yes
55/62 ... yes
54/63 ... yes
54/64 ... yes
54/65 ... yes
1/68 ... yes
1/69 ... yes
1/71 ... yes
1/78 ... yes
1/82 ... yes
1/83 ... yes
1/85 ... yes
1/86 ... yes
1/87 ... yes
39/89 ... yes
1/95 ... yes
1/96 ... yes
52/97 ... yes
52/98 ... yes
53/99 ... yes
17/100 ... yes
17/101 ... yes
39/104 ... yes
1/105 ... yes
1/106 ... yes
1/107 ... yes
1/108 ... yes
1/109 ... yes
1/110 ... yes
74/112 ... yes
51/113 ... yes
1/114 ... yes
1/115 ... yes
1/116 ... yes
1/117 ... yes
1/118 ... yes
1/120 ... yes
1/121 ... yes
50/122 ... yes
1/123 ... yes
1/124 ... yes
132/125 ... yes
132/127 ... yes
132/128 ... yes
50/129 ... yes
7/142 ... yes
7/143 ... yes
7/144 ... yes
7/145 ... yes
1/149 ... yes
1/150 ... yes
1/151 ... yes
1/152 ... yes
1/153 ... yes
1/154 ... yes
1/155 ... yes
1/157 ... yes
1/158 ... yes
1/163 ... yes
1/165 ... yes
1/166 ... yes
1/168 ... yes
17/169 ... yes
132/172 ... yes
132/176 ... yes
132/177 ... yes
132/178 ... yes
132/179 ... yes
32/181 ... yes
43/182 ... yes
169/184 ... yes
40/186 ... yes
39/188 ... yes
38/189 ... yes
132/190 ... yes
34/191 ... yes
29/192 ... yes
29/193 ... yes
29/194 ... yes
35/195 ... yes
29/196 ... yes
29/197 ... yes
37/198 ... yes
29/199 ... yes
29/200 ... yes
32/201 ... yes
39/202 ... yes
52/203 ... yes
62/204 ... yes
12/205 ... yes
32/206 ... yes
7/208 ... yes
7/209 ... yes
7/210 ... yes
62/211 ... yes
12/212 ... yes
172/215 ... yes
44/216 ... yes
170/217 ... yes
1/218 ... yes
12/219 ... yes
171/220 ... yes
67/221 ... yes
67/223 ... yes
68/224 ... yes
69/227 ... yes
32/228 ... yes
62/229 ... yes
32/230 ... yes
6/232 ... yes
44/234 ... yes
44/235 ... yes
37/236 ... yes
37/237 ... yes
69/239 ... yes
17/241 ... yes
32/242 ... yes
35/243 ... yes
44/246 ... yes
29/247 ... yes
62/248 ... yes
62/251 ... yes
1/252 ... yes
74/254 ... yes
75/256 ... yes
75/257 ... yes
37/260 ... yes
51/263 ... yes
1/264 ... yes
12/265 ... yes
79/267 ... yes
1/269 ... yes
79/271 ... yes
32/272 ... yes
12/273 ... yes
62/274 ... yes
112/275 ... yes
112/276 ... yes
32/278 ... yes
172/279 ... yes
62/281 ... yes
82/283 ... yes
82/284 ... yes
51/285 ... yes
1/286 ... yes
83/287 ... yes
83/288 ... yes
83/289 ... yes
83/290 ... yes
83/291 ... yes
87/292 ... yes
88/293 ... yes
12/295 ... yes
88/296 ... yes
89/297 ... yes
12/298 ... yes
12/299 ... yes
44/300 ... yes
83/301 ... yes
46/302 ... yes
6/303 ... yes
43/306 ... yes
43/307 ... yes
83/308 ... yes
44/309 ... yes
167/311 ... yes
83/312 ... yes
45/313 ... yes
92/314 ... yes
92/315 ... yes
1/316 ... yes
71/317 ... yes
6/318 ... yes
45/319 ... yes
83/320 ... yes
12/321 ... yes
46/322 ... yes
56/323 ... yes
46/324 ... yes
83/325 ... yes
83/326 ... yes
71/327 ... yes
12/328 ... yes
83/329 ... yes
12/330 ... yes
83/331 ... yes
163/332 ... yes
168/333 ... yes
8/335 ... yes
8/336 ... yes
8/337 ... yes
71/338 ... yes
42/340 ... yes
163/347 ... yes
101/349 ... yes
101/350 ... yes
48/351 ... yes
77/352 ... yes
12/354 ... yes
83/355 ... yes
102/356 ... yes
12/357 ... yes
166/359 ... yes
71/360 ... yes
222/363 ... yes
83/364 ... yes
77/365 ... yes
83/366 ... yes
56/368 ... yes
6/369 ... yes
56/371 ... yes
12/373 ... yes
12/374 ... yes
12/375 ... yes
56/376 ... yes
43/377 ... yes
12/378 ... yes
12/379 ... yes
168/380 ... yes
107/383 ... yes
107/384 ... yes
83/385 ... yes
12/387 ... yes
83/388 ... yes
10/389 ... yes
15/390 ... yes
15/391 ... yes
10/395 ... yes
43/396 ... yes
168/397 ... yes
168/398 ... yes
62/399 ... yes
106/400 ... yes
83/401 ... yes
83/403 ... yes
109/404 ... yes
106/405 ... yes
109/406 ... yes
83/408 ... yes
8/409 ... yes
83/410 ... yes
168/411 ... yes
169/412 ... yes
97/413 ... yes
110/414 ... yes
1/415 ... yes
112/416 ... yes
106/417 ... yes
12/418 ... yes
71/419 ... yes
12/422 ... yes
106/423 ... yes
113/424 ... yes
113/425 ... yes
39/426 ... yes
113/427 ... yes
113/428 ... yes
174/429 ... yes
112/430 ... yes
12/431 ... yes
112/433 ... yes
77/435 ... yes
106/436 ... yes
82/438 ... yes
12/439 ... yes
112/440 ... yes
48/441 ... yes
112/442 ... yes
83/443 ... yes
45/444 ... yes
45/445 ... yes
112/446 ... yes
45/447 ... yes
45/448 ... yes
45/449 ... yes
43/450 ... yes
112/451 ... yes
43/452 ... yes
43/453 ... yes
176/457 ... yes
39/458 ... yes
131/459 ... yes
90/460 ... yes
133/462 ... yes
110/463 ... yes
131/464 ... yes
99/465 ... yes
173/466 ... yes
110/467 ... yes
112/469 ... yes
48/470 ... yes
139/471 ... yes
83/473 ... yes
48/474 ... yes
106/475 ... yes
6/476 ... yes
56/477 ... yes
42/478 ... yes
106/479 ... yes
169/480 ... yes
110/482 ... yes
48/484 ... yes
172/485 ... yes
174/486 ... yes
56/487 ... yes
144/488 ... yes
110/491 ... yes
166/492 ... yes
77/495 ... yes
166/497 ... yes
42/500 ... yes
188/503 ... yes
189/505 ... yes
190/506 ... yes
191/507 ... yes
144/510 ... yes
166/511 ... yes
43/513 ... yes
144/514 ... yes
83/515 ... yes
83/516 ... yes
83/517 ... yes
106/519 ... yes
83/520 ... yes
83/521 ... yes
158/523 ... yes
158/525 ... yes
171/526 ... yes
158/528 ... yes
56/529 ... yes
51/530 ... yes
166/532 ... yes
133/533 ... yes
131/534 ... yes
51/535 ... yes
12/536 ... yes
169/537 ... yes
166/538 ... yes
44/540 ... yes
162/541 ... yes
106/542 ... yes
44/543 ... yes
44/544 ... yes
111/545 ... yes
111/546 ... yes
111/547 ... yes
105/548 ... yes
168/550 ... yes
83/552 ... yes
83/553 ... yes
163/554 ... yes
163/555 ... yes
163/556 ... yes
83/557 ... yes
83/558 ... yes
168/559 ... yes
12/560 ... yes
162/561 ... yes
162/562 ... yes
12/564 ... yes
99/565 ... yes
168/566 ... yes
166/568 ... yes
166/569 ... yes
174/570 ... yes
158/571 ... yes
77/573 ... yes
168/574 ... yes
166/575 ... yes
231/577 ... yes
163/578 ... yes
165/580 ... yes
163/581 ... yes
164/582 ... yes
172/583 ... yes
166/584 ... yes
144/585 ... yes
111/586 ... yes
173/587 ... yes
83/588 ... yes
173/589 ... yes
172/590 ... yes
56/591 ... yes
56/592 ... yes
166/593 ... yes
43/594 ... yes
42/595 ... yes
167/596 ... yes
167/597 ... yes
111/598 ... yes
173/599 ... yes
168/600 ... yes
111/602 ... yes
179/603 ... yes
168/604 ... yes
71/606 ... yes
5/608 ... yes
166/609 ... yes
56/610 ... yes
5/611 ... yes
112/613 ... yes
189/615 ... yes
6/616 ... yes
166/617 ... yes
6/618 ... yes
173/619 ... yes
182/620 ... yes
166/622 ... yes
12/623 ... yes
51/624 ... yes
43/625 ... yes
57/626 ... yes
182/627 ... yes
166/628 ... yes
182/629 ... yes
182/630 ... yes
168/631 ... yes
57/632 ... yes
57/633 ... yes
71/634 ... yes
83/635 ... yes
183/637 ... yes
184/639 ... yes
185/641 ... yes
112/642 ... yes
194/643 ... yes
185/644 ... yes
185/645 ... yes
112/646 ... yes
187/648 ... yes
185/649 ... yes
168/650 ... yes
43/652 ... yes
182/653 ... yes
168/654 ... yes
168/655 ... yes
71/656 ... yes
51/658 ... yes
12/660 ... yes
111/661 ... yes
192/662 ... yes
45/665 ... yes
193/667 ... yes
193/668 ... yes
12/670 ... yes
194/671 ... yes
194/672 ... yes
42/674 ... yes
48/675 ... yes
48/676 ... yes
194/677 ... yes
183/678 ... yes
48/679 ... yes
48/680 ... yes
99/681 ... yes
196/682 ... yes
171/683 ... yes
168/684 ... yes
194/685 ... yes
173/687 ... yes
172/688 ... yes
170/689 ... yes
165/690 ... yes
167/692 ... yes
48/693 ... yes
168/694 ... yes
198/695 ... yes
198/696 ... yes
45/697 ... yes
48/698 ... yes
169/699 ... yes
171/700 ... yes
168/701 ... yes
183/703 ... yes
200/705 ... yes
201/706 ... yes
201/707 ... yes
201/708 ... yes
183/710 ... yes
168/711 ... yes
71/712 ... yes
219/713 ... yes
12/714 ... yes
198/715 ... yes
1/716 ... yes
183/717 ... yes
204/718 ... yes
168/719 ... yes
204/720 ... yes
183/723 ... yes
112/724 ... yes
196/725 ... yes
12/726 ... yes
168/727 ... yes
168/728 ... yes
172/729 ... yes
193/730 ... yes
172/732 ... yes
106/733 ... yes
196/734 ... yes
173/735 ... yes
173/736 ... yes
162/737 ... yes
210/739 ... yes
219/740 ... yes
217/741 ... yes
217/742 ... yes
42/743 ... yes
44/744 ... yes
219/745 ... yes
51/746 ... yes
219/747 ... yes
210/748 ... yes
51/749 ... yes
220/750 ... yes
221/751 ... yes
218/752 ... yes
218/753 ... yes
202/754 ... yes
222/755 ... yes
222/756 ... yes
222/757 ... yes
45/758 ... yes
223/759 ... yes
51/760 ... yes
219/761 ... yes
173/762 ... yes
224/763 ... yes
224/764 ... yes
226/765 ... yes
222/766 ... yes
226/767 ... yes
225/768 ... yes
219/769 ... yes
51/770 ... yes
227/771 ... yes
224/772 ... yes
218/773 ... yes
225/774 ... yes
225/775 ... yes
225/776 ... yes
230/777 ... yes
230/778 ... yes
231/779 ... yes
225/780 ... yes
224/781 ... yes
179/782 ... yes
1/783 ... yes
231/784 ... yes
168/785 ... yes
168/786 ... yes
205/787 ... yes
168/788 ... yes
45/789 ... yes
45/790 ... yes
1/791 ... yes
217/792 ... yes
224/793 ... yes
42/794 ... yes
234/795 ... yes
234/796 ... yes
234/797 ... yes
168/798 ... yes
179/799 ... yes
236/800 ... yes
224/801 ... yes
236/803 ... yes
204/804 ... yes
236/805 ... yes
231/806 ... yes
101/809 ... yes
238/811 ... yes
239/812 ... yes
239/813 ... yes
231/814 ... yes
240/815 ... yes
240/816 ... yes
240/817 ... yes
240/818 ... yes
45/819 ... yes
240/820 ... yes
240/821 ... yes
90/822 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.5)
Git version >= 2.22.0 ? ... yes (2.24.1)
Git user has default SSH configuration? ... yes
Active users: ... 31
Is authorized keys file accessible? ... yes
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished

Workaround

1. Check background migrations and ensure ResetMergeStatus is the only job returned in pending_job_classes

2. Stan's solution of #215141 (comment 355612810) and #215141 (comment 355618875) in tandem gets around the problem.