Skip to content

Workaround SimpleCov / Coverage limitation around `load`

Thong Kuah requested to merge workaround_multiple_load_coverage into master

What does this MR do and why?

See #408630

Before:

(arm64) tkgl2:gitlab tkuah$ rm -rf coverage/
(arm64) tkgl2:gitlab tkuah$ bundle exec rspec --seed 37811 -f d /Users/tkuah/code/gdk-ee/gitlab/spec/initializers/active_record_transaction_observer_spec.rb
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Randomized with seed 37811

Test environment set up in 5.671124 seconds

ActiveRecord Transaction Observer
  # order random
  when DBMS is available
    calls Gitlab::Database::Transaction::Observer
    when flipper table does not exist
      does not calls Gitlab::Database::Transaction::Observer
  when DBMS is not available
    does not calls Gitlab::Database::Transaction::Observer

Finished in 8.05 seconds (files took 12.26 seconds to load)
3 examples, 0 failures

Randomized with seed 37811

[TEST PROF INFO] Time spent in factories: 00:00.006 (0.06% of total time)
Coverage report generated for RSpec to /Users/tkuah/code/gdk-ee/gitlab/coverage. 64827 / 380515 LOC (17.04%) covered.
Coverage report generated for RSpec to /Users/tkuah/code/gdk-ee/gitlab/coverage/coverage.xml
Lcov style coverage report generated for RSpec to /Users/tkuah/code/gdk-ee/gitlab/coverage/lcov/gitlab.lcov
(arm64) tkgl2:gitlab tkuah$ grep -A10 transaction_observer.rb /Users/tkuah/code/gdk-ee/gitlab/coverage/coverage.xml 
        <class name="active_record_transaction_observer" filename="config/initializers/active_record_transaction_observer.rb" line-rate="0.75" branch-rate="0" complexity="0">
          <methods/>
          <lines>
            <line number="3" branch="false" hits="1"/>
            <line number="5" branch="false" hits="1"/>
            <line number="8" branch="false" hits="1"/>
            <line number="10" branch="false" hits="1"/>
            <line number="14" branch="false" hits="1"/>
            <line number="16" branch="false" hits="0"/>
            <line number="19" branch="false" hits="1"/>
            <line number="20" branch="false" hits="0"/>

After :

(arm64) tkgl2:gitlab tkuah$ rm -rf coverage/
(arm64) tkgl2:gitlab tkuah$ bundle exec rspec --seed 37811 -f d /Users/tkuah/code/gdk-ee/gitlab/spec/initializers/active_record_transaction_observer_spec.rb
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Randomized with seed 37811

Test environment set up in 5.780364 seconds

ActiveRecord Transaction Observer
  # order random
  when DBMS is available
    calls Gitlab::Database::Transaction::Observer
    when flipper table does not exist
      does not calls Gitlab::Database::Transaction::Observer
  when DBMS is not available
    does not calls Gitlab::Database::Transaction::Observer

Finished in 14.89 seconds (files took 12.34 seconds to load)
3 examples, 0 failures

Randomized with seed 37811

[TEST PROF INFO] Time spent in factories: 00:00.005 (0.03% of total time)
Coverage report generated for RSpec, RSpec1, RSpec11, RSpec111 to /Users/tkuah/code/gdk-ee/gitlab/coverage. 64798 / 380514 LOC (17.03%) covered.
Coverage report generated for RSpec, RSpec1, RSpec11, RSpec111 to /Users/tkuah/code/gdk-ee/gitlab/coverage/coverage.xml
Lcov style coverage report generated for RSpec, RSpec1, RSpec11, RSpec111 to /Users/tkuah/code/gdk-ee/gitlab/coverage/lcov/gitlab.lcov
(arm64) tkgl2:gitlab tkuah$ grep -A10 transaction_observer.rb /Users/tkuah/code/gdk-ee/gitlab/coverage/coverage.xml 
        <class name="active_record_transaction_observer" filename="config/initializers/active_record_transaction_observer.rb" line-rate="1.0" branch-rate="0" complexity="0">
          <methods/>
          <lines>
            <line number="3" branch="false" hits="4"/>
            <line number="5" branch="false" hits="4"/>
            <line number="8" branch="false" hits="4"/>
            <line number="10" branch="false" hits="1"/>
            <line number="14" branch="false" hits="4"/>
            <line number="16" branch="false" hits="1"/>
            <line number="19" branch="false" hits="4"/>
            <line number="20" branch="false" hits="1"/>

Workaround SimpleCov / Coverage limitation around load

SimpleCov only reports on coverage for the last Kernel.load call, so we are missing coverage data when we test initializers. Add a workaround to spin up new SimpleCov to cover, and merge each load.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Thong Kuah

Merge request reports