Skip to content

Improve usage data tests

Doug Stull requested to merge improve-usage-data-tests into master

What does this MR do?

Improves the efficiency of the test suite by reducing creation of database objects.

Specifically

  • Reduce database object creation to minimal for suite.
  • Refactor into factories and helpers to help test readability.
  • Utilize shared_examples to better leverage standard rspec constructs instead of loops.

Factory Doctor Analysis

  • Analyzed locally using this command FDOC=1 be rspec spec/lib/gitlab/usage_data_spec.rb

Potential time improvements

  • Time wasted ~11s
  • Total spec runtime ~44s
initial output
Total (potentially) bad examples: 22
Total wasted time: 00:11.122

Gitlab::UsageData (./spec/lib/gitlab/usage_data_spec.rb:5) (198 records created, 00:11.122)
  is expected to all respond to #totals (./spec/lib/gitlab/usage_data_spec.rb:278) – 9 records created, 00:00.527
  does not have any conflicts (./spec/lib/gitlab/usage_data_spec.rb:287) – 9 records created, 00:00.516
  is expected to all be a kind of Hash (./spec/lib/gitlab/usage_data_spec.rb:283) – 9 records created, 00:00.495
  is expected to all have attributes {:keys => (all be a kind of Symbol), :values => (all be a kind of Integer)} (./spec/lib/gitlab/usage_data_spec.rb:284) – 9 records created, 00:00.396
  gathers feature usage data (./spec/lib/gitlab/usage_data_spec.rb:297) – 9 records created, 00:00.508
  gathers components usage data (./spec/lib/gitlab/usage_data_spec.rb:314) – 9 records created, 00:00.502
  works when queries time out in new (./spec/lib/gitlab/usage_data_spec.rb:330) – 9 records created, 00:00.504
  works when queries time out in to_json (./spec/lib/gitlab/usage_data_spec.rb:337) – 9 records created, 00:00.709
  gathers variable data (./spec/lib/gitlab/usage_data_spec.rb:348) – 9 records created, 00:00.834
  returns the count when counting succeeds (./spec/lib/gitlab/usage_data_spec.rb:378) – 9 records created, 00:00.402
  returns the fallback value when counting fails (./spec/lib/gitlab/usage_data_spec.rb:384) – 9 records created, 00:00.496
  is expected to all respond to #totals (./spec/lib/gitlab/usage_data_spec.rb:278) – 9 records created, 00:00.494
  does not have any conflicts (./spec/lib/gitlab/usage_data_spec.rb:287) – 9 records created, 00:00.395
  is expected to all be a kind of Hash (./spec/lib/gitlab/usage_data_spec.rb:283) – 9 records created, 00:00.478
  is expected to all have attributes {:keys => (all be a kind of Symbol), :values => (all be a kind of Integer)} (./spec/lib/gitlab/usage_data_spec.rb:284) – 9 records created, 00:00.499
  gathers feature usage data (./spec/lib/gitlab/usage_data_spec.rb:297) – 9 records created, 00:00.469
  gathers components usage data (./spec/lib/gitlab/usage_data_spec.rb:314) – 9 records created, 00:00.414
  works when queries time out in new (./spec/lib/gitlab/usage_data_spec.rb:330) – 9 records created, 00:00.499
  works when queries time out in to_json (./spec/lib/gitlab/usage_data_spec.rb:337) – 9 records created, 00:00.488
  gathers variable data (./spec/lib/gitlab/usage_data_spec.rb:348) – 9 records created, 00:00.464
  returns the count when counting succeeds (./spec/lib/gitlab/usage_data_spec.rb:378) – 9 records created, 00:00.494
  returns the fallback value when counting fails (./spec/lib/gitlab/usage_data_spec.rb:384) – 9 records created, 00:00.528

Process

Performance centric
  • Move individual tests out of the data creation loop.
  • Notice speed improvements by moving those out of data creation loop.
  • Move record creation to a factory.
  • Move the record creation out of before and into a let! to reduce data creation for each block.
  • Move ProjectFeature.first.update_attribute('repository_access_level', 0) into factory as well to reduce factory recreation on attribute changes.
  • Ensured Danger rules for ~"group::telemetry" new files are honored https://gitlab.com/gitlab-org/gitlab/-/blob/36a5f50e566594c1ee94efff2b58760a91c871de/danger/telemetry/Dangerfile#L9

Realized greater than 11s time reduction from initial runtime of ~44s and current runtime of ~31s.

  • Factory Doctor output Finished in 31.18 seconds (files took 35.65 seconds to load)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Doug Stull

Merge request reports