Consider running specs with FOSS_ONLY=1 before merging.

(IS_GITLAB_EE was changed to FOSS_ONLY in !18517 (merged))

!16395 (merged) introduced a failure on the gitlab-foss project because:

  • A background migration on lib/gitlab/background_migration/ referenced a EE specific model.
  • Spec for the background migration referenced a EE specific factory
  • Pipeline didn't fail because on execution time, both the EE model and the EE factory are available.

If we manually check the spec with IS_GITLAB_EE=0 it fails:

  1) Gitlab::BackgroundMigration::UpdateVulnerabilityConfidence updates confidence level for container scanning reports
     Failure/Error: let(:vul1) { attributes_for(:vulnerabilities_occurrence) }

     ArgumentError:
       Factory not registered: vulnerabilities_occurrence
     # /Users/mayra_cabrera/.rvm/gems/ruby-2.6.3/gems/factory_bot-4.8.2/lib/factory_bot/registry.rb:24:in `find'
     # /Users/mayra_cabrera/.rvm/gems/ruby-2.6.3/gems/factory_bot-4.8.2/lib/factory_bot/decorator.rb:10:in `method_missing'
     # /Users/mayra_cabrera/.rvm/gems/ruby-2.6.3/gems/factory_bot-4.8.2/lib/factory_bot.rb:100:in `factory_by_name'
     # /Users/mayra_cabrera/.rvm/gems/ruby-2.6.3/gems/factory_bot-4.8.2/lib/factory_bot/factory_runner.rb:12:in `run'
     # /Users/mayra_cabrera/.rvm/gems/ruby-2.6.3/gems/factory_bot-4.8.2/lib/factory_bot/strategy_syntax_method_registrar.rb:20:in `block in define_singular_strategy_method'
     # ./spec/lib/gitlab/background_migration/update_vulnerability_confidence_spec.rb:10:in `block (2 levels) in <top (required)>'
     # ./spec/lib/gitlab/background_migration/update_vulnerability_confidence_spec.rb:28:in `block (2 levels) in <top (required)>'
     # ./spec/support/database_cleaner.rb:43:in `block (2 levels) in <main>'
     # /Users/mayra_cabrera/.rvm/gems/ruby-2.6.3/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:123:in `block in run'
     # /Users/mayra_cabrera/.rvm/gems/ruby-2.6.3/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:110:in `loop'
     # /Users/mayra_cabrera/.rvm/gems/ruby-2.6.3/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:110:in `run'
     # /Users/mayra_cabrera/.rvm/gems/ruby-2.6.3/gems/rspec-retry-0.6.1/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
     # /Users/mayra_cabrera/.rvm/gems/ruby-2.6.3/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:37:in `block (2 levels) in setup'

Finished in 4.25 seconds (files took 13.62 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./spec/lib/gitlab/background_migration/update_vulnerability_confidence_spec.rb:14 # Gitlab::BackgroundMigration::UpdateVulnerabilityConfidence updates confidence level for container scanning reports

The solution to fix the above failure is simple: to move the files under their respective ee/ directories. But I can imagine this happening again as it basically depends on the reviewer/maintainer to catch these errors.

Should we consider running specs with IS_GITLAB_EE=0 before merging to avoid this type of errors?

Note: IS_GITLAB_EE was introduced on !17212 (merged)

Action items

Edited by Lin Jen-Shin