Multiple failures for the review-qa-all job

Noticing some failures in the MR pipelines (review-qa-all job) that don't seem to relate to MR changes, and have overlap with other failed MR jobs. Will continue to investigate other failed MR jobs as well.

This issue is for compiling a list of these failures and the jobs they fail in, in hopes of triaging whether or not they should run in this particular job, or should be looked into as failures that aren't due to the environment.

Review-qa-all failures:

  • browser_ui/14_non_devops/service_ping_default_enabled_spec.rb - cannot run against Review App due to default setup, excluded in !80858 (merged)

Service ping default enabled When using default enabled from gitlab.yml config has service ping toggle enabled

Stacktrace:

expected `QA::Page::Admin::Settings::Component::UsageStatistics.has_disabled_usage_data_checkbox?` to be falsey, got true

Failing jobs

Screenshot
  • browser_ui/11_fulfillment/purchase/user_registration_billing_spec.rb - billable users are not updated in real time in Review Apps, the test needs to be excluded - !90667 (merged)

Fulfillment Purchase User Registration when adding and removing a group member consumes a seat on the license

Stacktrace:


Failure/Error: expect(dashboard.billable_users.to_i).to eq(billable_users.to_i + 1)

  expected: 11
       got: 10

Failing jobs

  • browser_ui/3_create/design_management/archive_design_content_spec.rb - This and all other falling design tests: GET request for raw_image returns 404. Happens on some deployed environments but not others. Opened #366839 (closed) for investigation, the tests will be quarantined meanwhile.

Create Design Management user archives a design

Stacktrace

Failure/Error: let(:first_design) { Resource::Design.fabricate! }

Capybara::ElementNotFound:
  Unable to find visible css "[data-qa-selector=\"design_image\"][data-qa-filename=\"banana_sample.gif\"],.qa-design-image"

Failing jobs

Screenshot image
  • browser_ui/3_create/design_management/add_design_content_spec.rb

Create Design Management user adds a design and annotates it

Stacktrace

Unable to find visible css "[data-qa-selector=\"design_image\"][data-qa-filename=\"banana_sample.gif\"],.qa-design-image"

Failing jobs

Screenshot image
  • browser_ui/3_create/design_management/modify_design_content_spec.rb

Create Design Management user adds a design and modifies it

Stacktrace


Failure/Error:
  Resource::Design.fabricate! do |design|
    design.filename = 'testfile.png'
  end

Capybara::ElementNotFound:
  Unable to find visible css "[data-qa-selector=\"design_image\"][data-qa-filename=\"testfile.png\"],.qa-design-image"

Failed jobs

  • browser_ui/3_create/repository/push_mirroring_lfs_over_http_spec.rb - Logs indicate connection error on mirror update. Permission denied/Connection timed out on fetch remote. Appears to be environment related and looks similar to gitlab-com/gl-infra/production#6621 (closed) so I will follow that issue findings and quarantine the test against Review App meanwhile. Quarantine MR - !92057 (merged)

Create Push mirror a repository over HTTP configures and syncs LFS objects for a (push) mirrored repository

Stacktrace

 Failure/Error: mirror_settings.update target_project_uri # rubocop:disable Rails/SaveBang

QA::Support::Repeater::WaitExceededError:
  Wait failed after 180 seconds

Different error (same problem) after changes in !80573 (merged):

     Failure/Error: mirror_settings.verify_update(target_project_uri)
     
     QA::Support::Repeater::WaitExceededError:
       Wait failed after 60 seconds

Failed jobs

Screenshot image

Possible fixes

Edited by Anastasia McDonald