spec/tasks/gitlab/usage_data_rake_spec.rb | gitlab:usage data take tasks generate_and_send generate_ci_template_events generates /Users/remy/Code/GitLab/gdk/gitlab/lib/gitlab/usage_data_counters/known_events/ci_templates.yml

Summary

This looks order-dependent and caused by an earlier test.

Steps to reproduce

bin/rspec spec/services/ci/pipeline_processing/atomic_processing_service_spec.rb spec/services/merge_requests/refresh_service_spec.rb spec/policies/merge_request_policy_spec.rb spec/tasks/gitlab/usage_data_rake_spec.rb

Error:

  1) gitlab:usage data take tasks generate_and_send generate_ci_template_events generates /Users/remy/Code/GitLab/gdk/gitlab/lib/gitlab/usage_data_counters/known_events/ci_templates.yml
     Failure/Error: data = YAML.safe_load(template.content, aliases: true)
     
     Psych::DisallowedClass:
       Tried to load unspecified class: Gitlab::Ci::Config::Yaml::Tags::Reference
     # ./lib/gitlab/usage_data_counters/ci_template_unique_counter.rb:46:in `block in all_included_templates'
     # ./lib/gitlab/usage_data_counters/ci_template_unique_counter.rb:43:in `tap'
     # ./lib/gitlab/usage_data_counters/ci_template_unique_counter.rb:43:in `all_included_templates'
     # ./lib/gitlab/usage_data_counters/ci_template_unique_counter.rb:49:in `block (2 levels) in all_included_templates'
     # ./lib/gitlab/usage_data_counters/ci_template_unique_counter.rb:47:in `each'
     # ./lib/gitlab/usage_data_counters/ci_template_unique_counter.rb:47:in `block in all_included_templates'
     # ./lib/gitlab/usage_data_counters/ci_template_unique_counter.rb:43:in `tap'
     # ./lib/gitlab/usage_data_counters/ci_template_unique_counter.rb:43:in `all_included_templates'
     # ./lib/tasks/gitlab/usage_data.rake:70:in `implicit_auto_devops_includes'
     # ./lib/tasks/gitlab/usage_data.rake:46:in `block (3 levels) in <main>'
     # ./spec/support/helpers/rake_helpers.rb:6:in `run_rake_task'
     # ./spec/tasks/gitlab/usage_data_rake_spec.rb:82:in `block (4 levels) in <main>'
     # ./spec/tasks/gitlab/usage_data_rake_spec.rb:76:in `block (4 levels) in <main>'
     # ./spec/spec_helper.rb:415:in `block (3 levels) in <main>'
     # ./spec/support/sidekiq_middleware.rb:18:in `with_sidekiq_server_middleware'
     # ./spec/spec_helper.rb:407:in `block (2 levels) in <main>'
     # ./spec/spec_helper.rb:403:in `block (3 levels) in <main>'
     # ./lib/gitlab/application_context.rb:59:in `with_raw_context'
     # ./spec/spec_helper.rb:403:in `block (2 levels) in <main>'
     # ./spec/spec_helper.rb:239:in `block (2 levels) in <main>'
     # ./spec/support/system_exit_detected.rb:7:in `block (2 levels) in <main>'
     # ./spec/support/database/prevent_cross_joins.rb:106:in `block (3 levels) in <main>'
     # ./spec/support/database/prevent_cross_joins.rb:60:in `with_cross_joins_prevented'
     # ./spec/support/database/prevent_cross_joins.rb:106:in `block (2 levels) in <main>'
     # -e:1:in `<main>'

Proposed Resolution

Please refer to the Resolution guidance to learn more about resolution of broken master.

added Engineering Productivity bugtransient failureflaky-test priority2 severity3 typebug labels

Running bin/rspec spec/tasks/gitlab/usage_data_rake_spec.rb locally succeeds.

I can reproduce with bin/rspec spec/services/merge_requests/refresh_service_spec.rb spec/tasks/gitlab/usage_data_rake_spec.rb...

The minimal reproduction step seems to be by adding :focus on an example from spec/services/merge_requests/refresh_service_spec.rb which has the :sidekiq_inline metadata, and adding :focus on line 81 of spec/tasks/gitlab/usage_data_rake_spec.rb, and running bin/rspec spec/services/merge_requests/refresh_service_spec.rb spec/tasks/gitlab/usage_data_rake_spec.rb

Somehow the !reference keyword from lib/gitlab/ci/templates/Jobs/Dependency-Scanning.gitlab-ci.yml triggers the

     Psych::DisallowedClass:
       Tried to load unspecified class: Gitlab::Ci::Config::Yaml::Tags::Reference

error.

Here is the diff I used to find this:

diff --git a/lib/gitlab/usage_data_counters/ci_template_unique_counter.rb b/lib/gitlab/usage_data_counters/ci_template_unique_counter.rb
index eb040e9e819b..40303d59af60 100644
--- a/lib/gitlab/usage_data_counters/ci_template_unique_counter.rb
+++ b/lib/gitlab/usage_data_counters/ci_template_unique_counter.rb
@@ -42,6 +42,7 @@ def all_included_templates(template_name)
         expanded_template_name = expand_template_name(template_name)
         results = [expanded_template_name].tap do |result|
           template = Gitlab::Template::GitlabCiYmlTemplate.find(template_name.chomp('.gitlab-ci.yml'))
+          p template
           data = YAML.safe_load(template.content, aliases: true)
           [data['include']].compact.flatten.each do |ci_include|
             if ci_include_template = ci_include['template']
diff --git a/spec/services/merge_requests/refresh_service_spec.rb b/spec/services/merge_requests/refresh_service_spec.rb
index 5174ceaaa823..3f310098ad5c 100644
--- a/spec/services/merge_requests/refresh_service_spec.rb
+++ b/spec/services/merge_requests/refresh_service_spec.rb
@@ -189,7 +189,7 @@
 
       subject { service.new(project: @project, current_user: @user).execute(@oldrev, @newrev, 'refs/heads/master') }
 
-      it 'updates the head_pipeline_id for @merge_request', :sidekiq_inline do
+      it 'updates the head_pipeline_id for @merge_request', :sidekiq_inline, :focus do
         expect { subject }.to change { @merge_request.reload.head_pipeline_id }.from(nil).to(pipeline.id)
       end
 
diff --git a/spec/tasks/gitlab/usage_data_rake_spec.rb b/spec/tasks/gitlab/usage_data_rake_spec.rb
index 95ebaf6ea24c..d995b1d5c392 100644
--- a/spec/tasks/gitlab/usage_data_rake_spec.rb
+++ b/spec/tasks/gitlab/usage_data_rake_spec.rb
@@ -2,7 +2,7 @@
 
 require 'rake_helper'
 
-RSpec.describe 'gitlab:usage data take tasks', :silence_stdout do
+RSpec.describe 'gitlab:usage data take tasks' do
   include StubRequests
   include UsageDataHelpers
 
@@ -78,7 +78,7 @@
         `git checkout -- #{Gitlab::UsageDataCounters::CiTemplateUniqueCounter::KNOWN_EVENTS_FILE_PATH}`
       end
 
-      it "generates #{Gitlab::UsageDataCounters::CiTemplateUniqueCounter::KNOWN_EVENTS_FILE_PATH}" do
+      it "generates #{Gitlab::UsageDataCounters::CiTemplateUniqueCounter::KNOWN_EVENTS_FILE_PATH}", :focus do
         run_rake_task('gitlab:usage_data:generate_ci_template_events')
 
         expect(File.exist?(Gitlab::UsageDataCounters::CiTemplateUniqueCounter::KNOWN_EVENTS_FILE_PATH)).to be true

changed the description

added groupanalytics instrumentation label

added devopsanalyze sectionanalytics labels

added workflowplanning breakdown label

changed milestone to %15.9

added typemaintenance label and removed bugtransient typebug labels

removed Engineering Productivity label

changed milestone to %15.10

Happening here too https://gitlab.com/gitlab-org/gitlab/-/jobs/3622876604

Same thing here https://gitlab.com/gitlab-org/gitlab/-/jobs/3621986255

mentioned in merge request !109211 (merged)

mentioned in merge request !108252 (merged)

Interesting! A few months ago we ran into #368951 (comment 1065719944), but never could reproduce the issue. Could this issue be related to that?

Thanks for the pointer! I opened !109258 (merged) to improve that in the meantime.

marked this issue as related to #368951

mentioned in merge request !109258 (merged)

assigned to @rymai

mentioned in issue gitlab-org/quality/engineering-productivity/master-broken-incidents#706 (closed)

A few instances that happened around the same interval, which I find quite intriguing for a flaky test:

• gitlab-org/quality/engineering-productivity/master-broken-incidents#706 (closed)
• gitlab-org/quality/engineering-productivity/master-broken-incidents#707 (closed)
• gitlab-org/quality/engineering-productivity/master-broken-incidents#708 (closed)
• gitlab-org/quality/engineering-productivity/master-broken-incidents#709 (closed)
• gitlab-org/quality/engineering-productivity/master-broken-incidents#710 (closed)
• gitlab-org/quality/engineering-productivity/master-broken-incidents#711 (closed)

I'm quarantining the test with !110533 (merged) for now.

added 1 deleted label

@gitlab-org/analytics-section/product-intelligence/engineers Add your estimation to this thread using the guide

@bastirehm This is an issue that has already been worked on by Quality team here - does the PI team take it over now?

@michold good point.

@rymai I assume you're going to continue taking care of this or is there any action from Product Intelligence needed?

Yes!

removed milestone %15.10

closed

changed milestone to %15.9

added workflowcomplete label and removed workflowplanning breakdown label