Correctly handle data-loss scenarios when encrypting columns

If the EncryptColumns background migration runs in a sidekiq with a
stale view of the database schema, or when the purported destination
columns don't actually exist, data loss can result. Attempt to work
around these issues by reloading schema information before running
the migration, and raising errors if the model reports that any of its
source or destination columns are missing.
237 jobs for 53763-fix-encrypt-columns-data-loss in 44 minutes (queued for 20 seconds)
Status Job ID Name Coverage
  Prepare
passed #126301290
gitlab-org
compile-assets

00:09:15

passed #126301289
gitlab-org
retrieve-tests-metadata

00:00:37

passed #126301291
gitlab-org
setup-test-env

00:07:53

 
  Test
passed #126301561
allowed to fail
code_quality

00:09:11

passed #126301553
gitlab-org
db:check-schema-pg

00:02:48

passed #126301552
gitlab-org
db:migrate:reset-mysql

00:08:53

passed #126301551
gitlab-org
db:migrate:reset-pg

00:03:48

passed #126301557
gitlab-org
db:rollback-mysql

00:04:36

passed #126301556
gitlab-org
db:rollback-pg

00:03:02

passed #126301564
allowed to fail
dependency_scanning

00:04:33

passed #126301547
gitlab-org
docs lint

00:02:12

passed #126301548
gitlab-org
downtime_check

00:02:03

passed #126301550
gitlab-org
ee_compat_check

00:03:04

passed #126301559
gitlab-org
gitlab:setup-mysql

00:05:19

passed #126301558
gitlab-org
gitlab:setup-pg

00:05:20

passed #126301574
gitlab-org
gitlab_git_test

00:01:58

passed #126305690
gitlab-org
karma

00:08:57

58.21%
passed #126301555
gitlab-org
migration:path-mysql

00:13:04

passed #126301554
gitlab-org
migration:path-pg

00:12:15

passed #126301572
qa-frontend-node:10

00:09:42

passed #126301570
qa-frontend-node:6

00:12:22

passed #126301571
qa-frontend-node:8

00:11:33

passed #126301573
allowed to fail
qa-frontend-node:latest

00:11:08

passed #126301566
gitlab-org
qa:internal

00:03:24

passed #126301568
gitlab-org
qa:selectors

00:03:07

passed #126301549
gitlab-org
rails4_gemfile_lock_check

00:02:50

passed #126301353
gitlab-org
rspec-mysql 1/50

00:10:03

passed #126301373
gitlab-org
rspec-mysql 10/50

00:12:44

passed #126301375
gitlab-org
rspec-mysql 11/50

00:14:39

passed #126301377
gitlab-org
rspec-mysql 12/50

00:11:24

passed #126301379
gitlab-org
rspec-mysql 13/50

00:12:33

passed #126301380
gitlab-org
rspec-mysql 14/50

00:12:47

passed #126301381
gitlab-org
rspec-mysql 15/50

00:15:03

passed #126301382
gitlab-org
rspec-mysql 16/50

00:12:46

passed #126301383
gitlab-org
rspec-mysql 17/50

00:15:16

passed #126301384
gitlab-org
rspec-mysql 18/50

00:11:58

passed #126301385
gitlab-org
rspec-mysql 19/50

00:14:54

passed #126301356
gitlab-org
rspec-mysql 2/50

00:13:20

passed #126301386
gitlab-org
rspec-mysql 20/50

00:16:56

passed #126301387
gitlab-org
rspec-mysql 21/50

00:16:54

passed #126301388
gitlab-org
rspec-mysql 22/50

00:14:54

passed #126301389
gitlab-org
rspec-mysql 23/50

00:15:17

passed #126301390
gitlab-org
rspec-mysql 24/50

00:16:57

passed #126301391
gitlab-org
rspec-mysql 25/50

00:13:56

passed #126301392
gitlab-org
rspec-mysql 26/50

00:14:11

passed #126301393
gitlab-org
rspec-mysql 27/50

00:15:27

passed #126301394
gitlab-org
rspec-mysql 28/50

00:14:15

passed #126301395
gitlab-org
rspec-mysql 29/50

00:15:21

passed #126301362
gitlab-org
rspec-mysql 3/50

00:12:40

passed #126301396
gitlab-org
rspec-mysql 30/50

00:12:06

passed #126301397
gitlab-org
rspec-mysql 31/50

00:18:30

passed #126301398
gitlab-org
rspec-mysql 32/50

00:13:57

passed #126301399
gitlab-org
rspec-mysql 33/50

00:15:20

passed #126301400
gitlab-org
rspec-mysql 34/50

00:13:42

passed #126301401
gitlab-org
rspec-mysql 35/50

00:15:12

passed #126301402
gitlab-org
rspec-mysql 36/50

00:18:44

passed #126301403
gitlab-org
rspec-mysql 37/50

00:15:15

passed #126301404
gitlab-org
rspec-mysql 38/50

00:16:41

passed #126301405
gitlab-org
rspec-mysql 39/50

00:11:48

passed #126301363
gitlab-org
rspec-mysql 4/50

00:16:36

passed #126301406
gitlab-org
rspec-mysql 40/50

00:14:49

passed #126301407
gitlab-org
rspec-mysql 41/50

00:14:27

passed #126301408
gitlab-org
rspec-mysql 42/50

00:17:08

passed #126301409
gitlab-org
rspec-mysql 43/50

00:18:00

passed #126301410
gitlab-org
rspec-mysql 44/50

00:13:20

passed #126301412
gitlab-org
rspec-mysql 45/50

00:14:21

passed #126301414
gitlab-org
rspec-mysql 46/50

00:15:10

passed #126301415
gitlab-org
rspec-mysql 47/50

00:14:36

passed #126301416
gitlab-org
rspec-mysql 48/50

00:14:37

passed #126301418
gitlab-org
rspec-mysql 49/50

00:14:27

passed #126301366
gitlab-org
rspec-mysql 5/50

00:13:28

passed #126301419
gitlab-org
rspec-mysql 50/50

00:17:07

passed #126301367
gitlab-org
rspec-mysql 6/50

00:12:39

passed #126301369
gitlab-org
rspec-mysql 7/50

00:10:54

passed #126301371
gitlab-org
rspec-mysql 8/50

00:11:09

passed #126301372
gitlab-org
rspec-mysql 9/50

00:12:42

passed #126301494
gitlab-org
rspec-mysql-rails4 1/50

00:10:00

passed #126301505
gitlab-org
rspec-mysql-rails4 10/50

00:11:10

passed #126301506
gitlab-org
rspec-mysql-rails4 11/50

00:12:05

passed #126301507
gitlab-org
rspec-mysql-rails4 12/50

00:11:21

passed #126301508
gitlab-org
rspec-mysql-rails4 13/50

00:13:22

passed #126301509
gitlab-org
rspec-mysql-rails4 14/50

00:13:04

passed #126301510
gitlab-org
rspec-mysql-rails4 15/50

00:13:43

passed #126301511
gitlab-org
rspec-mysql-rails4 16/50

00:12:48

passed #126301512
gitlab-org
rspec-mysql-rails4 17/50

00:12:36

passed #126301513
gitlab-org
rspec-mysql-rails4 18/50

00:11:41

passed #126301514
gitlab-org
rspec-mysql-rails4 19/50

00:11:08

passed #126301495
gitlab-org
rspec-mysql-rails4 2/50

00:13:01

passed #126301515
gitlab-org
rspec-mysql-rails4 20/50

00:15:25

passed #126301516
gitlab-org
rspec-mysql-rails4 21/50

00:15:11

passed #126301517
gitlab-org
rspec-mysql-rails4 22/50

00:15:25

passed #126301518
gitlab-org
rspec-mysql-rails4 23/50

00:13:56

passed #126301519
gitlab-org
rspec-mysql-rails4 24/50

00:15:13

passed #126301520
gitlab-org
rspec-mysql-rails4 25/50

00:13:19

passed #126301521
gitlab-org
rspec-mysql-rails4 26/50

00:12:33

passed #126301522
gitlab-org
rspec-mysql-rails4 27/50

00:14:51

passed #126301523
gitlab-org
rspec-mysql-rails4 28/50

00:13:19

passed #126301524
gitlab-org
rspec-mysql-rails4 29/50

00:15:35

passed #126301496
gitlab-org
rspec-mysql-rails4 3/50

00:14:20

passed #126301525
gitlab-org
rspec-mysql-rails4 30/50

00:13:09

passed #126301526
gitlab-org
rspec-mysql-rails4 31/50

00:17:55

passed #126301527
gitlab-org
rspec-mysql-rails4 32/50

00:13:58

passed #126301528
gitlab-org
rspec-mysql-rails4 33/50

00:12:37

passed #126301529
gitlab-org
rspec-mysql-rails4 34/50

00:13:58

passed #126301530
gitlab-org
rspec-mysql-rails4 35/50

00:14:37

passed #126301531
gitlab-org
rspec-mysql-rails4 36/50

00:18:30

passed #126301532
gitlab-org
rspec-mysql-rails4 37/50

00:11:42

passed #126301533
gitlab-org
rspec-mysql-rails4 38/50

00:18:14

passed #126301534
gitlab-org
rspec-mysql-rails4 39/50

00:12:12

passed #126301497
gitlab-org
rspec-mysql-rails4 4/50

00:16:55

passed #126301535
gitlab-org
rspec-mysql-rails4 40/50

00:12:33

passed #126301536
gitlab-org
rspec-mysql-rails4 41/50

00:11:46

passed #126301537
gitlab-org
rspec-mysql-rails4 42/50

00:16:01

passed #126301538
gitlab-org
rspec-mysql-rails4 43/50

00:16:41

passed #126305692
gitlab-org
rspec-mysql-rails4 44/50

00:10:55

passed #126301540
gitlab-org
rspec-mysql-rails4 45/50

00:14:21

passed #126301541
gitlab-org
rspec-mysql-rails4 46/50

00:11:50

passed #126301542
gitlab-org
rspec-mysql-rails4 47/50

00:13:15

passed #126301543
gitlab-org
rspec-mysql-rails4 48/50

00:15:14

passed #126301544
gitlab-org
rspec-mysql-rails4 49/50

00:14:47

passed #126301498
gitlab-org
rspec-mysql-rails4 5/50

00:13:21

passed #126301545
gitlab-org
rspec-mysql-rails4 50/50

00:13:58

passed #126301499
gitlab-org
rspec-mysql-rails4 6/50

00:13:00

passed #126301500
gitlab-org
rspec-mysql-rails4 7/50

00:11:01

passed #126301501
gitlab-org
rspec-mysql-rails4 8/50

00:13:08

passed #126301504
gitlab-org
rspec-mysql-rails4 9/50

00:14:59

passed #126301292
gitlab-org
rspec-pg 1/50

00:12:13

passed #126301303
gitlab-org
rspec-pg 10/50

00:11:03

passed #126301304
gitlab-org
rspec-pg 11/50

00:13:52

passed #126301305
gitlab-org
rspec-pg 12/50

00:15:18

passed #126301306
gitlab-org
rspec-pg 13/50

00:13:06

passed #126301308
gitlab-org
rspec-pg 14/50

00:12:35

passed #126301309
gitlab-org
rspec-pg 15/50

00:16:25

passed #126301310
gitlab-org
rspec-pg 16/50

00:13:36

passed #126301311
gitlab-org
rspec-pg 17/50

00:15:08

passed #126301312
gitlab-org
rspec-pg 18/50

00:14:08

passed #126301313
gitlab-org
rspec-pg 19/50

00:13:14

passed #126301293
gitlab-org
rspec-pg 2/50

00:15:28

passed #126301314
gitlab-org
rspec-pg 20/50

00:14:38

passed #126301315
gitlab-org
rspec-pg 21/50

00:14:35

passed #126301316
gitlab-org
rspec-pg 22/50

00:13:37

passed #126301317
gitlab-org
rspec-pg 23/50

00:14:25

passed #126301318
gitlab-org
rspec-pg 24/50

00:13:38

passed #126301319
gitlab-org
rspec-pg 25/50

00:13:01

passed #126301320
gitlab-org
rspec-pg 26/50

00:14:40

passed #126301322
gitlab-org
rspec-pg 27/50

00:13:51

passed #126301323
gitlab-org
rspec-pg 28/50

00:13:09

passed #126311277
gitlab-org
rspec-pg 29/50

00:11:54

passed #126301295
gitlab-org
rspec-pg 3/50

00:13:06

passed #126301325
gitlab-org
rspec-pg 30/50

00:13:24

passed #126301326
gitlab-org
rspec-pg 31/50

00:14:10

passed #126301327
gitlab-org
rspec-pg 32/50

00:13:28

passed #126301328
gitlab-org
rspec-pg 33/50

00:14:37

passed #126301329
gitlab-org
rspec-pg 34/50

00:12:45

passed #126301330
gitlab-org
rspec-pg 35/50

00:13:27

passed #126301331
gitlab-org
rspec-pg 36/50

00:15:24

passed #126301332
gitlab-org
rspec-pg 37/50

00:13:29

passed #126301333
gitlab-org
rspec-pg 38/50

00:15:10

passed #126301336
gitlab-org
rspec-pg 39/50

00:13:34

passed #126301297
gitlab-org
rspec-pg 4/50

00:16:53

passed #126301337
gitlab-org
rspec-pg 40/50

00:14:58

passed #126301338
gitlab-org
rspec-pg 41/50

00:14:13

passed #126301339
gitlab-org
rspec-pg 42/50

00:15:36

passed #126301340
gitlab-org
rspec-pg 43/50

00:13:52

passed #126301342
gitlab-org
rspec-pg 44/50

00:12:32

passed #126301343
gitlab-org
rspec-pg 45/50

00:14:15

passed #126301344
gitlab-org
rspec-pg 46/50

00:13:55

passed #126301346
gitlab-org
rspec-pg 47/50

00:13:59

passed #126301347
gitlab-org
rspec-pg 48/50

00:14:25

passed #126301348
gitlab-org
rspec-pg 49/50

00:13:19

passed #126301298
gitlab-org
rspec-pg 5/50

00:13:37

passed #126301351
gitlab-org
rspec-pg 50/50

00:13:40

passed #126301299
gitlab-org
rspec-pg 6/50

00:13:31

passed #126301300
gitlab-org
rspec-pg 7/50

00:11:01

passed #126301301
gitlab-org
rspec-pg 8/50

00:12:50

passed #126301302
gitlab-org
rspec-pg 9/50

00:14:07

passed #126301421
gitlab-org
rspec-pg-rails4 1/50

00:11:11

passed #126301442
gitlab-org
rspec-pg-rails4 10/50

00:11:37

passed #126301444
gitlab-org
rspec-pg-rails4 11/50

00:12:47

passed #126301446
gitlab-org
rspec-pg-rails4 12/50

00:11:43

passed #126301447
gitlab-org
rspec-pg-rails4 13/50

00:13:55

passed #126301448
gitlab-org
rspec-pg-rails4 14/50

00:12:29

passed #126301449
gitlab-org
rspec-pg-rails4 15/50

00:13:35

passed #126301450
gitlab-org
rspec-pg-rails4 16/50

00:12:20

passed #126305128
gitlab-org
rspec-pg-rails4 17/50

00:13:09

passed #126301456
gitlab-org
rspec-pg-rails4 18/50

00:13:32

passed #126301459
gitlab-org
rspec-pg-rails4 19/50

00:11:07

passed #126301423
gitlab-org
rspec-pg-rails4 2/50

00:12:36

passed #126301460
gitlab-org
rspec-pg-rails4 20/50

00:13:13

passed #126301461
gitlab-org
rspec-pg-rails4 21/50

00:12:35

passed #126301462
gitlab-org
rspec-pg-rails4 22/50

00:13:20

passed #126301463
gitlab-org
rspec-pg-rails4 23/50

00:13:33

passed #126301464
gitlab-org
rspec-pg-rails4 24/50

00:13:55

passed #126301465
gitlab-org
rspec-pg-rails4 25/50

00:12:41

passed #126301466
gitlab-org
rspec-pg-rails4 26/50

00:10:30

passed #126301467
gitlab-org
rspec-pg-rails4 27/50

00:13:27

passed #126301469
gitlab-org
rspec-pg-rails4 28/50

00:12:11

passed #126301470
gitlab-org
rspec-pg-rails4 29/50

00:12:16

passed #126301424
gitlab-org
rspec-pg-rails4 3/50

00:12:48

passed #126301471
gitlab-org
rspec-pg-rails4 30/50

00:10:35

passed #126301472
gitlab-org
rspec-pg-rails4 31/50

00:13:13

passed #126305448
gitlab-org
rspec-pg-rails4 32/50

00:12:08

passed #126305386
gitlab-org
rspec-pg-rails4 33/50

00:11:11

passed #126301475
gitlab-org
rspec-pg-rails4 34/50

00:13:59

passed #126301476
gitlab-org
rspec-pg-rails4 35/50

00:13:57

passed #126301477
gitlab-org
rspec-pg-rails4 36/50

00:14:00

passed #126301478
gitlab-org
rspec-pg-rails4 37/50

00:12:02

passed #126301480
gitlab-org
rspec-pg-rails4 38/50

00:13:23

passed #126301481
gitlab-org
rspec-pg-rails4 39/50

00:12:33

passed #126301431
gitlab-org
rspec-pg-rails4 4/50

00:16:48

passed #126301482
gitlab-org
rspec-pg-rails4 40/50

00:12:41

passed #126301483
gitlab-org
rspec-pg-rails4 41/50

00:13:17

passed #126301484
gitlab-org
rspec-pg-rails4 42/50

00:15:42

passed #126301485
gitlab-org
rspec-pg-rails4 43/50

00:11:29

passed #126301486
gitlab-org
rspec-pg-rails4 44/50

00:10:15

passed #126301487
gitlab-org
rspec-pg-rails4 45/50

00:13:13

passed #126301488
gitlab-org
rspec-pg-rails4 46/50

00:10:42

passed #126301489
gitlab-org
rspec-pg-rails4 47/50

00:11:20

passed #126301491
gitlab-org
rspec-pg-rails4 48/50

00:11:35

passed #126301492
gitlab-org
rspec-pg-rails4 49/50

00:10:24

passed #126301433
gitlab-org
rspec-pg-rails4 5/50

00:12:49

passed #126301493
gitlab-org
rspec-pg-rails4 50/50

00:13:22

passed #126301436
gitlab-org
rspec-pg-rails4 6/50

00:13:06

passed #126301437
gitlab-org
rspec-pg-rails4 7/50

00:09:26

passed #126301440
gitlab-org
rspec-pg-rails4 8/50

00:13:23

passed #126301441
gitlab-org
rspec-pg-rails4 9/50

00:12:45

passed #126301562
allowed to fail
sast

00:04:05

passed #126301546
gitlab-org
static-analysis

00:09:44

failed #126301560
gitlab-org
karma

00:01:10

failed #126301539
gitlab-org
rspec-mysql-rails4 44/50

00:01:15

failed #126301324
gitlab-org
rspec-pg 29/50

00:18:51

failed #126301451
gitlab-org
rspec-pg-rails4 17/50

00:00:47

failed #126301473
gitlab-org
rspec-pg-rails4 32/50

00:01:04

failed #126301474
gitlab-org
rspec-pg-rails4 33/50

00:00:57

 
  Post Test
passed #126301577
gitlab-org
coverage

00:03:40

100.0%
passed #126301576
gitlab-org allowed to fail
flaky-examples-check

00:02:20

passed #126301578
gitlab-org
lint:javascript:report

00:01:09