Add feature_category as metadata to rspec flaky examples
What does this MR do and why?
Describe in detail what your merge request does and why.
Related #378566 (closed)
Add related feature_category POC for spec files
- Update 4 spec files with the feature_category, 2 of them picked from RSpec flaky tests artifacts
- Update the example and flaky_example to include an attribute for
feature_category
with this we add the feature_category in example metadata and later can be used in reports. This will follow-up- For example https://app.periscopedata.com/app/gitlab/888968/EP---Flaky-tests
- Or create automatically flaky test issues for specific feature categories. From the category, we can automatically assign the group.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
This will produce rspec flaky report with the following information
# Testing a flaky test
# In ee/spec/features/registrations/saas/standard_flow_company_creating_project_spec.rb
raise 'e' if rand(1..100) % 2 == 0
# Then running the test with retry
RETRIES=2 FLAKY_RSPEC_GENERATE_REPORT=1 bin/rspec ee/spec/features/registrations/saas/standard_flow_company_creating_project_spec.rb:8
{
"d67e883033074d259f74657557fcd639": {
"first_flaky_at": "2022-11-08 13:26:21 +0200",
"last_flaky_at": "2022-11-08 13:26:21 +0200",
"last_flaky_job": null,
"last_attempts_count": 3,
"flaky_reports": 1,
"feature_category": "onboarding",
"example_id": "./ee/spec/features/registrations/saas/standard_flow_company_creating_project_spec.rb[1:1:1]",
"file": "./ee/spec/features/registrations/saas/standard_flow_company_creating_project_spec.rb",
"line": 8,
"description": "Standard flow for user picking company and creating a project when opting into a trial registers the user and creates a group and project reaching onboarding"
}
}
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Alina Mihaila