Allow external diffs to be used conditionally

Since external diffs are likely to be a bit slower than in-database
ones, add a mode that makes diffs external after they've been obsoleted
by events. This should strike a balance between performance and disk
space.

A background cron drives the majority of migrations, since diffs become
outdated through user actions.
135 jobs for 54670-external-diffs-when-outdated in 51 minutes and 8 seconds (queued for 10 seconds)
Status Job ID Name Coverage
  Prepare
passed #185395463
gitlab-org
compile-assets

00:08:40

passed #185395462
gitlab-org
retrieve-tests-metadata

00:02:07

passed #185395464
gitlab-org
setup-test-env

00:08:58

 
  Test
passed #185395465
allowed to fail
code_quality

00:10:01

passed #185395591
gitlab-org
db:check-schema-pg

00:05:01

passed #185395590
gitlab-org
db:migrate:reset-mysql

00:04:59

passed #185395589
gitlab-org
db:migrate:reset-pg

00:03:26

passed #185395595
gitlab-org
db:rollback-mysql

00:04:57

passed #185395594
gitlab-org
db:rollback-pg

00:03:44

passed #185395603
allowed to fail
dependency_scanning

00:04:27

passed #185395585
gitlab-org
docs lint

00:02:53

passed #185395586
gitlab-org
downtime_check

00:02:46

passed #185395588
gitlab-org
ee_compat_check

00:04:32

passed #185395598
gitlab-org
gitlab:setup-mysql

00:08:45

passed #185395596
gitlab-org
gitlab:setup-pg

00:05:49

passed #185395599
gitlab-org allowed to fail
gitlab:ui:visual

00:06:12

passed #185395609
gitlab-org
gitlab_git_test

00:00:37

passed #185395601
gitlab-org
jest

00:12:37

passed #185395600
gitlab-org
karma

00:11:40

58.1%
passed #185395593
gitlab-org
migration:path-mysql

00:17:03

passed #185395592
gitlab-org
migration:path-pg

00:15:40

passed #185395607
gitlab-org
qa-frontend-node:10

00:11:11

passed #185395606
gitlab-org
qa-frontend-node:8

00:14:15

passed #185395608
gitlab-org allowed to fail
qa-frontend-node:latest

00:11:27

passed #185395604
gitlab-org
qa:internal

00:05:46

passed #185395605
gitlab-org
qa:selectors

00:05:48

passed #185395522
gitlab-org
rspec-mysql 1/50

00:17:31

passed #185395534
gitlab-org
rspec-mysql 10/50

00:27:40

passed #185395535
gitlab-org
rspec-mysql 11/50

00:20:09

passed #185395536
gitlab-org
rspec-mysql 12/50

00:20:10

passed #185395537
gitlab-org
rspec-mysql 13/50

00:15:58

passed #185395538
gitlab-org
rspec-mysql 14/50

00:19:08

passed #185395539
gitlab-org
rspec-mysql 15/50

00:21:15

passed #185395540
gitlab-org
rspec-mysql 16/50

00:16:06

passed #185395541
gitlab-org
rspec-mysql 17/50

00:20:13

passed #185395542
gitlab-org
rspec-mysql 18/50

00:23:21

passed #185395543
gitlab-org
rspec-mysql 19/50

00:22:12

passed #185395523
gitlab-org
rspec-mysql 2/50

00:15:41

passed #185395544
gitlab-org
rspec-mysql 20/50

00:21:31

passed #185395545
gitlab-org
rspec-mysql 21/50

00:21:07

passed #185395546
gitlab-org
rspec-mysql 22/50

00:24:15

passed #185395547
gitlab-org
rspec-mysql 23/50

00:22:00

passed #185395548
gitlab-org
rspec-mysql 24/50

00:20:49

passed #185395549
gitlab-org
rspec-mysql 25/50

00:25:43

passed #185395550
gitlab-org
rspec-mysql 26/50

00:20:17

passed #185395551
gitlab-org
rspec-mysql 27/50

00:21:35

passed #185395552
gitlab-org
rspec-mysql 28/50

00:23:56

passed #185395553
gitlab-org
rspec-mysql 29/50

00:18:10

passed #185395524
gitlab-org
rspec-mysql 3/50

00:24:02

passed #185395554
gitlab-org
rspec-mysql 30/50

00:23:36

passed #185395555
gitlab-org
rspec-mysql 31/50

00:24:07

passed #185395556
gitlab-org
rspec-mysql 32/50

00:22:42

passed #185395557
gitlab-org
rspec-mysql 33/50

00:19:57

passed #185395559
gitlab-org
rspec-mysql 34/50

00:22:10

passed #185395560
gitlab-org
rspec-mysql 35/50

00:24:38

passed #185395561
gitlab-org
rspec-mysql 36/50

00:22:06

passed #185395562
gitlab-org
rspec-mysql 37/50

00:20:39

passed #185395563
gitlab-org
rspec-mysql 38/50

00:22:09

passed #185395564
gitlab-org
rspec-mysql 39/50

00:21:53

passed #185395526
gitlab-org
rspec-mysql 4/50

00:16:47

passed #185395566
gitlab-org
rspec-mysql 40/50

00:19:34

passed #185395568
gitlab-org
rspec-mysql 41/50

00:21:33

passed #185395570
gitlab-org
rspec-mysql 42/50

00:21:31

passed #185395571
gitlab-org
rspec-mysql 43/50

00:20:00

passed #185395572
gitlab-org
rspec-mysql 44/50

00:25:24

passed #185395573
gitlab-org
rspec-mysql 45/50

00:18:21

passed #185395574
gitlab-org
rspec-mysql 46/50

00:22:46

passed #185395575
gitlab-org
rspec-mysql 47/50

00:22:05

passed #185395577
gitlab-org
rspec-mysql 48/50

00:18:48

passed #185395578
gitlab-org
rspec-mysql 49/50

00:26:05

passed #185395528
gitlab-org
rspec-mysql 5/50

00:30:05

passed #185395579
gitlab-org
rspec-mysql 50/50

00:22:44

passed #185407810
gitlab-org
rspec-mysql 6/50

00:24:36

passed #185395531
gitlab-org
rspec-mysql 7/50

00:21:31

passed #185395532
gitlab-org
rspec-mysql 8/50

00:24:04

passed #185395533
gitlab-org
rspec-mysql 9/50

00:19:02

passed #185395582
gitlab-org allowed to fail
rspec-mysql-quarantine

00:11:50

passed #185395466
gitlab-org
rspec-pg 1/50

00:19:31

passed #185395476
gitlab-org
rspec-pg 10/50

00:27:06

passed #185395478
gitlab-org
rspec-pg 11/50

00:19:49

passed #185395480
gitlab-org
rspec-pg 12/50

00:21:47

passed #185395481
gitlab-org
rspec-pg 13/50

00:14:17

passed #185395482
gitlab-org
rspec-pg 14/50

00:21:04

passed #185395483
gitlab-org
rspec-pg 15/50

00:17:19

passed #185395484
gitlab-org
rspec-pg 16/50

00:17:15

passed #185395485
gitlab-org
rspec-pg 17/50

00:20:11

passed #185395486
gitlab-org
rspec-pg 18/50

00:18:32

passed #185395487
gitlab-org
rspec-pg 19/50

00:20:35

passed #185395467
gitlab-org
rspec-pg 2/50

00:14:27

passed #185395488
gitlab-org
rspec-pg 20/50

00:17:18

passed #185395489
gitlab-org
rspec-pg 21/50

00:21:05

passed #185395490
gitlab-org
rspec-pg 22/50

00:20:19

passed #185395491
gitlab-org
rspec-pg 23/50

00:18:15

passed #185395492
gitlab-org
rspec-pg 24/50

00:22:51

passed #185395493
gitlab-org
rspec-pg 25/50

00:23:14

passed #185395494
gitlab-org
rspec-pg 26/50

00:23:52

passed #185395496
gitlab-org
rspec-pg 27/50

00:20:21

passed #185395498
gitlab-org
rspec-pg 28/50

00:19:27

passed #185395500
gitlab-org
rspec-pg 29/50

00:20:18

passed #185395468
gitlab-org
rspec-pg 3/50

00:24:19

passed #185395501
gitlab-org
rspec-pg 30/50

00:20:15

passed #185395502
gitlab-org
rspec-pg 31/50

00:18:42

passed #185395503
gitlab-org
rspec-pg 32/50

00:21:23

passed #185395504
gitlab-org
rspec-pg 33/50

00:18:33

passed #185395505
gitlab-org
rspec-pg 34/50

00:19:09

passed #185395506
gitlab-org
rspec-pg 35/50

00:29:38

passed #185395507
gitlab-org
rspec-pg 36/50

00:22:53

passed #185395508
gitlab-org
rspec-pg 37/50

00:17:03

passed #185395509
gitlab-org
rspec-pg 38/50

00:20:09

passed #185395510
gitlab-org
rspec-pg 39/50

00:19:54

passed #185395469
gitlab-org
rspec-pg 4/50

00:18:35

passed #185395511
gitlab-org
rspec-pg 40/50

00:20:42

passed #185395512
gitlab-org
rspec-pg 41/50

00:24:13

passed #185395513
gitlab-org
rspec-pg 42/50

00:18:54

passed #185395514
gitlab-org
rspec-pg 43/50

00:20:27

passed #185395515
gitlab-org
rspec-pg 44/50

00:23:17

passed #185395516
gitlab-org
rspec-pg 45/50

00:22:32

passed #185395517
gitlab-org
rspec-pg 46/50

00:20:46

passed #185395518
gitlab-org
rspec-pg 47/50

00:22:30

passed #185395519
gitlab-org
rspec-pg 48/50

00:17:31

passed #185395520
gitlab-org
rspec-pg 49/50

00:18:44

passed #185395470
gitlab-org
rspec-pg 5/50

00:29:49

passed #185395521
gitlab-org
rspec-pg 50/50

00:28:27

passed #185395471
gitlab-org
rspec-pg 6/50

00:26:57

passed #185395472
gitlab-org
rspec-pg 7/50

00:16:49

passed #185395473
gitlab-org
rspec-pg 8/50

00:23:34

passed #185395474
gitlab-org
rspec-pg 9/50

00:14:49

passed #185395581
gitlab-org allowed to fail
rspec-pg-quarantine

00:09:19

passed #185395602
allowed to fail
sast

00:24:32

passed #185395583
gitlab-org
static-analysis

00:12:35

failed #185395530
gitlab-org
rspec-mysql 6/50

00:06:04

 
  Post Test
passed #185395613
gitlab-org
coverage

00:06:21

95.2%
failed #185395610
gitlab-org allowed to fail
flaky-examples-check

00:01:29

passed #185395617
gitlab-org
jsdoc

00:02:27

passed #185395614
gitlab-org
lint:javascript:report

00:01:45

 
Name Stage Failure
failed
flaky-examples-check Post Test
    "first_flaky_at": "2019-03-27 17:06:49 +0000",
"last_flaky_at": "2019-03-27 17:06:49 +0000",
"last_flaky_job": "https://gitlab.com/nick.thomas/gitlab-ce/-/jobs/185395570"
}
}
Running after script...
$ date
Wed Mar 27 17:38:13 UTC 2019
ERROR: Job failed: exit code 1