Introduce CounterAttribute concern

* Log counter increments to an events table
* Consolidate counters asynchronously
* Read values considering those from events
175 jobs for !20565 with efficient-counters-flat-schema in 89 minutes and 22 seconds (queued for 6 seconds)
detached
Status Job ID Name Coverage
  Prepare
passed #358703567
docker gitlab-org
build-qa-image

00:06:00

passed #358703561
gitlab-org
compile-assets pull-cache

00:07:30

passed #358703563
gitlab-org
compile-assets pull-cache foss

00:07:13

manual #358703569
gitlab-org allowed to fail manual
manual:review-cleanup
passed #358703571
retrieve-tests-metadata

00:01:57

passed #358703570
gitlab-org
review-stop-failed-deployment

00:00:58

passed #358703565
gitlab-org
setup-test-env

00:07:11

 
  Test
passed #358703600
allowed to fail
code_quality

00:11:06

passed #358703768
gitlab-org
danger-review

00:02:51

passed #358703762
gitlab-org
db:check-schema

00:03:43

passed #358703763
gitlab-org
db:migrate-from-v11.11.0

00:13:49

passed #358703761
gitlab-org
db:migrate:reset

00:04:23

passed #358703764
gitlab-org
db:rollback

00:04:51

passed #358703767
gitlab-org
db:rollback geo

00:04:14

passed #358703604
allowed to fail
dependency_scanning

00:10:09

passed #358703760
gitlab-org
downtime_check

00:03:09

passed #358703573
docker gitlab-org
gitlab:assets:compile pull-cache

00:21:39

passed #358703765
gitlab-org
gitlab:setup

00:03:50

passed #358703769
gitlab-org
gitlab_git_test

00:00:33

passed #358703572
gitlab-org
graphql-reference-verify

00:04:10

passed #358703576
gitlab-org
jest

00:21:42

passed #358703579
gitlab-org
jest-foss

00:21:09

passed #358703574
gitlab-org
karma

00:20:22

66.33%
passed #358703575
gitlab-org
karma-foss

00:13:04

passed #358703592
gitlab-org
memory-on-boot

00:06:30

passed #358703588
gitlab-org
memory-static

00:05:39

passed #358703582
gitlab-org
qa-frontend-node:10

00:15:23

passed #358703581
gitlab-org
qa-frontend-node:8

00:16:10

passed #358703584
gitlab-org allowed to fail
qa-frontend-node:latest

00:15:19

passed #358703594
gitlab-org
qa:internal

00:02:31

passed #358703596
gitlab-org
qa:selectors

00:02:04

passed #358703598
gitlab-org
qa:selectors-foss

00:01:59

passed #358703757
gitlab-org
rspec fast_spec_helper

00:04:46

passed #358703661
gitlab-org
rspec integration pg9 1/6

00:17:42

passed #358703662
gitlab-org
rspec integration pg9 2/6

00:17:33

passed #358703663
gitlab-org
rspec integration pg9 3/6

00:21:46

passed #358703664
gitlab-org
rspec integration pg9 4/6

00:21:43

passed #358703665
gitlab-org
rspec integration pg9 5/6

00:19:04

passed #358703666
gitlab-org
rspec integration pg9 6/6

00:21:34

passed #358703667
gitlab-org
rspec integration pg9-foss 1/6

00:16:22

passed #358703668
gitlab-org
rspec integration pg9-foss 2/6

00:18:04

passed #358703669
gitlab-org
rspec integration pg9-foss 3/6

00:17:12

passed #358703670
gitlab-org
rspec integration pg9-foss 4/6

00:17:07

passed #358703671
gitlab-org
rspec integration pg9-foss 5/6

00:13:29

passed #358703672
gitlab-org
rspec integration pg9-foss 6/6

00:15:22

passed #358703673
gitlab-org
rspec system pg9 1/24

00:15:04

passed #358703684
gitlab-org
rspec system pg9 10/24

00:15:47

passed #358703685
gitlab-org
rspec system pg9 11/24

00:13:16

passed #358703686
gitlab-org
rspec system pg9 12/24

00:16:37

passed #358703687
gitlab-org
rspec system pg9 13/24

00:17:00

passed #358703688
gitlab-org
rspec system pg9 14/24

00:15:44

passed #358703689
gitlab-org
rspec system pg9 15/24

00:15:12

passed #358703690
gitlab-org
rspec system pg9 16/24

00:16:48

passed #358703691
gitlab-org
rspec system pg9 17/24

00:12:57

passed #358703692
gitlab-org
rspec system pg9 18/24

00:19:19

passed #358703693
gitlab-org
rspec system pg9 19/24

00:15:50

passed #358703675
gitlab-org
rspec system pg9 2/24

00:16:29

passed #358703694
gitlab-org
rspec system pg9 20/24

00:16:00

passed #358703696
gitlab-org
rspec system pg9 21/24

00:13:47

passed #358703698
gitlab-org
rspec system pg9 22/24

00:16:20

passed #358703699
gitlab-org
rspec system pg9 23/24

00:14:17

passed #358703701
gitlab-org
rspec system pg9 24/24

00:18:37

passed #358703676
gitlab-org
rspec system pg9 3/24

00:13:45

passed #358703678
gitlab-org
rspec system pg9 4/24

00:14:54

passed #358703679
gitlab-org
rspec system pg9 5/24

00:12:54

passed #358703680
gitlab-org
rspec system pg9 6/24

00:16:37

passed #358703681
gitlab-org
rspec system pg9 7/24

00:21:01

passed #358703682
gitlab-org
rspec system pg9 8/24

00:17:27

passed #358703683
gitlab-org
rspec system pg9 9/24

00:15:52

passed #358703702
gitlab-org
rspec system pg9-foss 1/24

00:13:38

passed #358703713
gitlab-org
rspec system pg9-foss 10/24

00:14:34

passed #358703714
gitlab-org
rspec system pg9-foss 11/24

00:12:51

passed #358703715
gitlab-org
rspec system pg9-foss 12/24

00:14:12

failed #358703716
gitlab-org
rspec system pg9-foss 13/24

00:19:11

passed #358703717
gitlab-org
rspec system pg9-foss 14/24

00:14:49

passed #358703718
gitlab-org
rspec system pg9-foss 15/24

00:14:05

passed #358703719
gitlab-org
rspec system pg9-foss 16/24

00:13:15

passed #358703720
gitlab-org
rspec system pg9-foss 17/24

00:14:51

passed #358703722
gitlab-org
rspec system pg9-foss 18/24

00:17:02

passed #358703724
gitlab-org
rspec system pg9-foss 19/24

00:16:46

passed #358703703
gitlab-org
rspec system pg9-foss 2/24

00:14:48

passed #358703726
gitlab-org
rspec system pg9-foss 20/24

00:15:31

passed #358703727
gitlab-org
rspec system pg9-foss 21/24

00:12:06

passed #358703728
gitlab-org
rspec system pg9-foss 22/24

00:12:25

passed #358703729
gitlab-org
rspec system pg9-foss 23/24

00:13:31

passed #358703731
gitlab-org
rspec system pg9-foss 24/24

00:16:54

passed #358703704
gitlab-org
rspec system pg9-foss 3/24

00:14:59

passed #358703705
gitlab-org
rspec system pg9-foss 4/24

00:13:07

passed #358703707
gitlab-org
rspec system pg9-foss 5/24

00:13:40

passed #358703709
gitlab-org
rspec system pg9-foss 6/24

00:14:38

passed #358703710
gitlab-org
rspec system pg9-foss 7/24

00:13:41

passed #358703711
gitlab-org
rspec system pg9-foss 8/24

00:12:50

passed #358703712
gitlab-org
rspec system pg9-foss 9/24

00:16:18

passed #358703605
gitlab-org
rspec unit pg9 1/20

00:19:33

passed #358703617
gitlab-org
rspec unit pg9 10/20

00:22:18

passed #358703618
gitlab-org
rspec unit pg9 11/20

00:20:03

passed #358703619
gitlab-org
rspec unit pg9 12/20

01:21:45

passed #358703620
gitlab-org
rspec unit pg9 13/20

00:15:03

passed #358703621
gitlab-org
rspec unit pg9 14/20

00:17:52

passed #358703624
gitlab-org
rspec unit pg9 15/20

00:21:37

passed #358703626
gitlab-org
rspec unit pg9 16/20

00:20:14

passed #358703628
gitlab-org
rspec unit pg9 17/20

00:15:17

passed #358703631
gitlab-org
rspec unit pg9 18/20

00:16:30

passed #358703633
gitlab-org
rspec unit pg9 19/20

00:14:41

passed #358703607
gitlab-org
rspec unit pg9 2/20

00:13:55

passed #358703635
gitlab-org
rspec unit pg9 20/20

00:19:06

passed #358703609
gitlab-org
rspec unit pg9 3/20

00:13:41

failed #358703611
gitlab-org
rspec unit pg9 4/20

00:18:39

passed #358703612
gitlab-org
rspec unit pg9 5/20

00:17:17

passed #358703613
gitlab-org
rspec unit pg9 6/20

00:17:48

passed #358703614
gitlab-org
rspec unit pg9 7/20

00:16:38

passed #358703615
gitlab-org
rspec unit pg9 8/20

00:19:03

failed #358703616
gitlab-org
rspec unit pg9 9/20

00:15:33

passed #358703637
gitlab-org
rspec unit pg9-foss 1/20

00:18:17

passed #358703648
gitlab-org
rspec unit pg9-foss 10/20

00:20:06

passed #358703649
gitlab-org
rspec unit pg9-foss 11/20

00:14:48

passed #358703650
gitlab-org
rspec unit pg9-foss 12/20

01:01:57

passed #358703652
gitlab-org
rspec unit pg9-foss 13/20

00:19:20

passed #358703654
gitlab-org
rspec unit pg9-foss 14/20

00:18:07

passed #358703655
gitlab-org
rspec unit pg9-foss 15/20

00:15:57

passed #358703656
gitlab-org
rspec unit pg9-foss 16/20

00:17:24

passed #358703657
gitlab-org
rspec unit pg9-foss 17/20

00:15:21

passed #358703658
gitlab-org
rspec unit pg9-foss 18/20

00:16:49

passed #358703659
gitlab-org
rspec unit pg9-foss 19/20

00:16:49

passed #358703640
gitlab-org
rspec unit pg9-foss 2/20

00:12:43

passed #358703660
gitlab-org
rspec unit pg9-foss 20/20

00:15:03

passed #358703641
gitlab-org
rspec unit pg9-foss 3/20

00:12:43

failed #358703642
gitlab-org
rspec unit pg9-foss 4/20

00:15:16

passed #358703643
gitlab-org
rspec unit pg9-foss 5/20

00:16:04

passed #358703644
gitlab-org
rspec unit pg9-foss 6/20

00:15:54

passed #358703645
gitlab-org
rspec unit pg9-foss 7/20

00:15:09

passed #358703646
gitlab-org
rspec unit pg9-foss 8/20

00:14:47

failed #358703647
gitlab-org
rspec unit pg9-foss 9/20

00:12:52

passed #358703755
gitlab-org
rspec-ee integration pg10 geo

00:08:45

passed #358703740
gitlab-org
rspec-ee integration pg9 1/3

00:14:40

passed #358703741
gitlab-org
rspec-ee integration pg9 2/3

00:17:56

passed #358703742
gitlab-org
rspec-ee integration pg9 3/3

00:13:42

passed #358703751
gitlab-org
rspec-ee integration pg9 geo

00:07:59

passed #358703756
gitlab-org
rspec-ee system pg10 geo

00:07:05

passed #358703744
gitlab-org
rspec-ee system pg9 1/5

00:16:57

passed #358703745
gitlab-org
rspec-ee system pg9 2/5

00:17:58

passed #358703746
gitlab-org
rspec-ee system pg9 3/5

00:20:11

passed #358703747
gitlab-org
rspec-ee system pg9 4/5

00:20:16

passed #358703748
gitlab-org
rspec-ee system pg9 5/5

00:17:21

passed #358703752
gitlab-org
rspec-ee system pg9 geo

00:06:47

passed #358703753
gitlab-org
rspec-ee unit pg10 geo 1/2

00:14:17

passed #358703754
gitlab-org
rspec-ee unit pg10 geo 2/2

00:13:15

passed #358703732
gitlab-org
rspec-ee unit pg9 1/7

00:15:15

passed #358703734
gitlab-org
rspec-ee unit pg9 2/7

00:15:54

passed #358703735
gitlab-org
rspec-ee unit pg9 3/7

00:20:22

passed #358703736
gitlab-org
rspec-ee unit pg9 4/7

00:13:50

passed #358703737
gitlab-org
rspec-ee unit pg9 5/7

00:15:37

passed #358703738
gitlab-org
rspec-ee unit pg9 6/7

00:15:33

passed #358703739
gitlab-org
rspec-ee unit pg9 7/7

00:19:12

passed #358703749
gitlab-org
rspec-ee unit pg9 geo 1/2

00:16:24

passed #358703750
gitlab-org
rspec-ee unit pg9 geo 2/2

00:13:27

passed #358703602
allowed to fail
sast

00:09:50

passed #358703758
gitlab-org
static-analysis 1/2

00:11:30

passed #358703759
gitlab-org
static-analysis 2/2

00:10:41

passed #358703586
gitlab-org
webpack-dev-server

00:05:30

 
  Review Prepare
passed #358703771
gitlab-org
review-build-cng

00:30:33

 
  Review
failed #358703773
gitlab-org allowed to fail
review-deploy

00:20:09

skipped #358703774
gitlab-org allowed to fail manual
review-stop
 
  Qa
passed #358703777
allowed to fail
dast

00:06:04

manual #358703776
allowed to fail manual
package-and-qa-manual
passed #358703785
docker gitlab-org allowed to fail
review-performance

00:05:15

manual #358703779
docker gitlab-org allowed to fail manual
review-qa-all 1/5
manual #358703780
docker gitlab-org allowed to fail manual
review-qa-all 2/5
manual #358703782
docker gitlab-org allowed to fail manual
review-qa-all 3/5
manual #358703783
docker gitlab-org allowed to fail manual
review-qa-all 4/5
manual #358703784
docker gitlab-org allowed to fail manual
review-qa-all 5/5
failed #358703778
docker gitlab-org allowed to fail
review-qa-smoke

00:07:01

 
  Post Test
skipped #358703786
gitlab-org
coverage
skipped #358703788
gitlab-org allowed to fail
flaky-examples-check
skipped #358703787
gitlab-org allowed to fail manual
parallel-spec-reports
 
Name Stage Failure
failed
rspec unit pg9-foss 9/20 Test
rspec_flaky/: found 4 matching files               
rspec_profiling/: found 1 matching files

WARNING: tmp/capybara/: no matching files
tmp/memory_test/: found 2 matching files

Uploading artifacts to coordinator... ok
id=358703647 responseStatus=201 Created token=vu1SzKqg
Uploading artifacts...
junit_rspec.xml: found 1 matching files

Uploading artifacts to coordinator... ok
id=358703647 responseStatus=201 Created token=vu1SzKqg
ERROR: Job failed: exit code 1
failed
rspec unit pg9-foss 4/20 Test
rspec_flaky/: found 4 matching files               
rspec_profiling/: found 1 matching files

WARNING: tmp/capybara/: no matching files
tmp/memory_test/: found 2 matching files

Uploading artifacts to coordinator... ok
id=358703642 responseStatus=201 Created token=YJR9NTMX
Uploading artifacts...
junit_rspec.xml: found 1 matching files

Uploading artifacts to coordinator... ok
id=358703642 responseStatus=201 Created token=YJR9NTMX
ERROR: Job failed: exit code 1
failed
rspec unit pg9 9/20 Test
rspec_flaky/: found 4 matching files               
rspec_profiling/: found 1 matching files

WARNING: tmp/capybara/: no matching files
tmp/memory_test/: found 2 matching files

Uploading artifacts to coordinator... ok
id=358703616 responseStatus=201 Created token=BYMTyy1M
Uploading artifacts...
junit_rspec.xml: found 1 matching files

Uploading artifacts to coordinator... ok
id=358703616 responseStatus=201 Created token=BYMTyy1M
ERROR: Job failed: exit code 1
failed
rspec unit pg9 4/20 Test
rspec_flaky/: found 4 matching files               
rspec_profiling/: found 1 matching files

WARNING: tmp/capybara/: no matching files
tmp/memory_test/: found 2 matching files

Uploading artifacts to coordinator... ok
id=358703611 responseStatus=201 Created token=_zQAzzRH
Uploading artifacts...
junit_rspec.xml: found 1 matching files

Uploading artifacts to coordinator... ok
id=358703611 responseStatus=201 Created token=_zQAzzRH
ERROR: Job failed: exit code 1
failed
rspec system pg9-foss 13/24 Test
rspec_flaky/: found 4 matching files               
rspec_profiling/: found 1 matching files

tmp/capybara/: found 9 matching files

tmp/memory_test/: found 2 matching files

Uploading artifacts to coordinator... ok
id=358703716 responseStatus=201 Created token=SWnGh4mQ
Uploading artifacts...
junit_rspec.xml: found 1 matching files

Uploading artifacts to coordinator... ok
id=358703716 responseStatus=201 Created token=SWnGh4mQ
ERROR: Job failed: exit code 1
failed
review-deploy Review
      helm upgrade --install     --wait     --timeout 900     --set ci.branch="efficient-counters-flat-schema"     --set ci.commit.sha="a0173c8e"     --set ci.job.url="https://gitlab.com/gitlab-org/gitlab/-/jobs/358703773"     --set ci.pipeline.url="https://gitlab.com/gitlab-org/gitlab/pipelines/97889539"     --set releaseOverride="review-efficient-0y0rmk"     --set global.hosts.hostSuffix="review-efficient-0y0rmk"     --set global.hosts.domain="gitlab-review.app"     --set gitlab.migrations.image.repository="registry.gitlab.com/gitlab-org/build/cng-mirror/gitlab-rails-ee"     --set gitlab.migrations.image.tag="efficient-counters-flat-schema"     --set gitlab.gitaly.image.repository="registry.gitlab.com/gitlab-org/build/cng-mirror/gitaly"     --set gitlab.gitaly.image.tag="v1.72.0"     --set gitlab.gitlab-shell.image.repository="registry.gitlab.com/gitlab-org/build/cng-mirror/gitlab-shell"     --set gitlab.gitlab-shell.image.tag="v10.2.0"     --set gitlab.sidekiq.image.repository="registry.gitlab.com/gitlab-org/build/cng-mirror/gitlab-sidekiq-ee"     --set gitlab.sidekiq.image.tag="efficient-counters-flat-schema"     --set gitlab.unicorn.image.repository="registry.gitlab.com/gitlab-org/build/cng-mirror/gitlab-unicorn-ee"     --set gitlab.unicorn.image.tag="efficient-counters-flat-schema"     --set gitlab.unicorn.workhorse.image="registry.gitlab.com/gitlab-org/build/cng-mirror/gitlab-workhorse-ee"     --set gitlab.unicorn.workhorse.tag="efficient-counters-flat-schema"     --set gitlab.task-runner.image.repository="registry.gitlab.com/gitlab-org/build/cng-mirror/gitlab-task-runner-ee"     --set gitlab.task-runner.image.tag="efficient-counters-flat-schema"   --set global.gitlab.license.secret="review-efficient-0y0rmk-gitlab-license"   --namespace="review-apps-ee"   --version="97889539-358703773"   -f "https://gitlab.com/gitlab-org/gitlab/raw/master/scripts/review_apps/base-config.yaml"   "review-efficient-0y0rmk" .
Release "review-efficient-0y0rmk" does not exist. Installing it now.
Running after script...
$ date
Fri Nov 22 16:11:43 UTC 2019
Uploading artifacts...
review_app_url.txt: found 1 matching files
Uploading artifacts to coordinator... ok
id=358703773 responseStatus=201 Created token=F_nQXvGj
ERROR: Job failed: exit code 1
failed
review-qa-smoke Qa
  /home/gitlab/qa/qa/scenario/bootable.rb:28:in `launch!'
bin/qa:7:in `<top (required)>'
Running after script...
$ date
Fri Nov 22 16:18:47 UTC 2019
Uploading artifacts...
./qa/gitlab-qa-run-*: found 2 matching files
Uploading artifacts to coordinator... ok
id=358703778 responseStatus=201 Created token=gMGRWs53
ERROR: Job failed: exit code 1