Skip to content

Increase the query limit for a flaky GraphQL test

Rémy Coutable requested to merge fix-flaky-graphql-test into master

What does this MR do and why?

This test is the most flaky one, as reported in https://app.periscopedata.com/app/gitlab/888968/EP---Flaky-tests?widget=12112039&udv=1474231.

The test only passes on the second try, so it makes sense to increase the limit, so that it passes on first try.

The test was updated with 95dd6407 to check the query limit.

How to set up and validate locally

Without the change

The test only passes on retry:

$ RETRIES=0 bin/rspec -f doc 'spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb[1:4:1]'
Running via Spring preloader in process 32766
Run options: include {:focus=>true, :ids=>{"./spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb"=>["1:4:1"]}}

Test environment set up in 7.986136 seconds

Setting assignees of a merge request
  when passing an empty list of assignees
    removes assignee (FAILED - 1)

Failures:

  1) Setting assignees of a merge request when passing an empty list of assignees removes assignee
     Failure/Error: expect(recorder.count).to be <= db_query_limit
     
       expected: <= 31
            got:    35
     # ./spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb:53:in `run_mutation!'
     # ./spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb:118:in `block (3 levels) in <main>'
     # ./spec/spec_helper.rb:464:in `block (3 levels) in <main>'
     # ./spec/support/sidekiq_middleware.rb:18:in `with_sidekiq_server_middleware'
     # ./spec/spec_helper.rb:456:in `block (2 levels) in <main>'
     # ./spec/spec_helper.rb:452:in `block (3 levels) in <main>'
     # ./lib/gitlab/application_context.rb:61:in `with_raw_context'
     # ./spec/spec_helper.rb:452:in `block (2 levels) in <main>'
     # ./spec/spec_helper.rb:240:in `block (2 levels) in <main>'
     # ./spec/support/system_exit_detected.rb:7:in `block (2 levels) in <main>'
     # ./spec/support/database/prevent_cross_joins.rb:108:in `block (3 levels) in <main>'
     # ./spec/support/database/prevent_cross_joins.rb:62:in `with_cross_joins_prevented'
     # ./spec/support/database/prevent_cross_joins.rb:108:in `block (2 levels) in <main>'
     # -e:1:in `<main>'

Finished in 18.3 seconds (files took 4.05 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb:117 # Setting assignees of a merge request when passing an empty list of assignees removes assignee

$ RETRIES=1 bin/rspec -f doc 'spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb[1:4:1]'
Running via Spring preloader in process 28117
Run options: include {:focus=>true, :ids=>{"./spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb"=>["1:4:1"]}}

Test environment set up in 8.153494 seconds

Setting assignees of a merge request
  when passing an empty list of assignees
    removes assignee
      
1st Try error in ./spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb:117:
expected: <= 31
     got:    35
      
RSpec::Retry: 2nd try ./spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb:117

Finished in 19.71 seconds (files took 4.36 seconds to load)
1 example, 0 failures

With the change

The test passes on first try:

$ RETRIES=0 bin/rspec -f doc 'spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb[1:4:1]'
Running via Spring preloader in process 33672
Run options: include {:focus=>true, :ids=>{"./spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb"=>["1:4:1"]}}

Test environment set up in 8.19533 seconds

Setting assignees of a merge request
  when passing an empty list of assignees
    removes assignee

Finished in 18.49 seconds (files took 3.71 seconds to load)
1 example, 0 failures

$ RETRIES=1 bin/rspec -f doc 'spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb[1:4:1]'
Running via Spring preloader in process 34635
Run options: include {:focus=>true, :ids=>{"./spec/requests/api/graphql/mutations/merge_requests/set_assignees_spec.rb"=>["1:4:1"]}}

Test environment set up in 8.978438 seconds

Setting assignees of a merge request
  when passing an empty list of assignees
    removes assignee

Finished in 19.09 seconds (files took 4.25 seconds to load)
1 example, 0 failures

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Rémy Coutable

Merge request reports