Skip to content

Add feature_category as metadata to rspec flaky examples

Alina Mihaila requested to merge am-add-acquisition-tag-to-tests into master

What does this MR do and why?

Describe in detail what your merge request does and why.

Related #378566 (closed)

Add related feature_category POC for spec files

  • Update 4 spec files with the feature_category, 2 of them picked from RSpec flaky tests artifacts
  • Update the example and flaky_example to include an attribute for feature_category with this we add the feature_category in example metadata and later can be used in reports. This will follow-up

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

This will produce rspec flaky report with the following information

# Testing a flaky test
# In ee/spec/features/registrations/saas/standard_flow_company_creating_project_spec.rb

raise 'e' if  rand(1..100) % 2 == 0

# Then running the test with retry
RETRIES=2  FLAKY_RSPEC_GENERATE_REPORT=1 bin/rspec ee/spec/features/registrations/saas/standard_flow_company_creating_project_spec.rb:8
{
  "d67e883033074d259f74657557fcd639": {
    "first_flaky_at": "2022-11-08 13:26:21 +0200",
    "last_flaky_at": "2022-11-08 13:26:21 +0200",
    "last_flaky_job": null,
    "last_attempts_count": 3,
    "flaky_reports": 1,
    "feature_category": "onboarding",
    "example_id": "./ee/spec/features/registrations/saas/standard_flow_company_creating_project_spec.rb[1:1:1]",
    "file": "./ee/spec/features/registrations/saas/standard_flow_company_creating_project_spec.rb",
    "line": 8,
    "description": "Standard flow for user picking company and creating a project when opting into a trial registers the user and creates a group and project reaching onboarding"
  }
}

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Alina Mihaila

Merge request reports