Enable parallel_tests for test-on-cng pipeline

What does this MR do and why?

• Fix issue with using same environment variable to trigger parallel run and detect if test is running as a parallel process which can lead to incorrect detection of parallel process.
• Update retry function to correctly merge parallel run results
• Enable parallel for test-on-cng

Closes: gitlab-org/quality/quality-engineering/team-tasks#3034 (closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

changed milestone to %17.5

added maintenancepipelines teamTest and Tools Infrastructure typemaintenance labels

assigned to @acunskis

changed the description

added pipelinetier-1 label

added QA label

Pipeline Changes

This merge request contains changes to the pipeline configuration for the GitLab project.

Please consider the effect of the changes in this merge request on the following:

• Effects on different pipeline types
• Effects on non-canonical projects:
  • gitlab-foss
  • security
  • dev
  • personal forks
• Effects on pipeline performance

Please consider communicating these changes to the broader team following the communication guideline for pipeline changes

Reviewer roulette

Category	Reviewer	Maintainer
QA	@nprabakaran (UTC+2, 1 hour behind author)	@treagitlab (UTC-7, 10 hours behind author)
~"Pipeline configuration"	Reviewer review is optional for ~"Pipeline configuration"	@mlapierre (UTC+10, 7 hours ahead of author)
~"Tooling"	Reviewer review is optional for ~"Tooling"	@rymai (UTC+2, 1 hour behind author)

Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

If needed, you can retry the danger-review job that generated this comment.

Generated by Danger

added Quality label

changed the description

E2E Test Result Summary

allure-report-publisher generated test report!

e2e-test-on-gdk: test report for 429e0644

expand test summary

+------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Plan        | 76     | 0      | 0       | 1     | 76    | ✅     |
| Data Stores | 33     | 0      | 1       | 0     | 34    | ✅     |
| Govern      | 73     | 0      | 0       | 0     | 73    | ✅     |
| Create      | 128    | 0      | 18      | 0     | 146   | ✅     |
| Package     | 24     | 0      | 11      | 0     | 35    | ✅     |
| Verify      | 45     | 0      | 2       | 0     | 47    | ✅     |
| Fulfillment | 2      | 0      | 0       | 0     | 2     | ✅     |
| Analytics   | 2      | 0      | 0       | 0     | 2     | ✅     |
| Manage      | 1      | 0      | 1       | 0     | 2     | ✅     |
| Monitor     | 8      | 0      | 0       | 0     | 8     | ✅     |
| Release     | 5      | 0      | 0       | 0     | 5     | ✅     |
| Secure      | 4      | 0      | 0       | 0     | 4     | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 401    | 0      | 33      | 1     | 434   | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

e2e-test-on-cng: test report for 82da110f

expand test summary

+------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Govern      | 82     | 0      | 7       | 0     | 89    | ✅     |
| Data Stores | 33     | 0      | 10      | 0     | 43    | ✅     |
| Create      | 139    | 0      | 20      | 0     | 159   | ✅     |
| Plan        | 86     | 0      | 8       | 0     | 94    | ✅     |
| Verify      | 50     | 0      | 15      | 0     | 65    | ✅     |
| Package     | 24     | 0      | 14      | 0     | 38    | ✅     |
| Monitor     | 8      | 0      | 12      | 0     | 20    | ✅     |
| Secure      | 4      | 0      | 2       | 0     | 6     | ✅     |
| Manage      | 1      | 0      | 9       | 0     | 10    | ✅     |
| Configure   | 0      | 0      | 3       | 0     | 3     | ➖     |
| Release     | 5      | 0      | 1       | 0     | 6     | ✅     |
| Ai-powered  | 0      | 0      | 2       | 0     | 2     | ➖     |
| Fulfillment | 2      | 0      | 7       | 0     | 9     | ✅     |
| Growth      | 0      | 0      | 2       | 0     | 2     | ➖     |
| ModelOps    | 0      | 0      | 1       | 0     | 1     | ➖     |
| Analytics   | 2      | 0      | 0       | 0     | 2     | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 436    | 0      | 113     | 0     | 549   | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

e2e-test-on-omnibus: test report for 429e0644

expand test summary

+---------------------------------------------------------------------+
|                           suites summary                            |
+----------------+--------+--------+---------+-------+-------+--------+
|                | passed | failed | skipped | flaky | total | result |
+----------------+--------+--------+---------+-------+-------+--------+
| Data Stores    | 46     | 0      | 11      | 0     | 57    | ✅     |
| Manage         | 27     | 0      | 18      | 5     | 45    | ✅     |
| GitLab Metrics | 2      | 0      | 1       | 0     | 3     | ✅     |
| Systems        | 6      | 0      | 1       | 0     | 7     | ✅     |
| Govern         | 108    | 0      | 6       | 2     | 114   | ✅     |
| Create         | 558    | 1      | 80      | 1     | 639   | ❌     |
| Package        | 32     | 0      | 13      | 0     | 45    | ✅     |
| Analytics      | 3      | 0      | 0       | 0     | 3     | ✅     |
| Plan           | 83     | 0      | 8       | 0     | 91    | ✅     |
| Secure         | 5      | 0      | 2       | 0     | 7     | ✅     |
| Fulfillment    | 4      | 0      | 7       | 0     | 11    | ✅     |
| Verify         | 50     | 0      | 15      | 1     | 65    | ✅     |
| Monitor        | 12     | 0      | 13      | 0     | 25    | ✅     |
| Configure      | 1      | 0      | 3       | 0     | 4     | ✅     |
| Ai-powered     | 1      | 0      | 2       | 0     | 3     | ✅     |
| Release        | 5      | 0      | 1       | 0     | 6     | ✅     |
| ModelOps       | 0      | 0      | 1       | 0     | 1     | ➖     |
| Growth         | 0      | 0      | 2       | 0     | 2     | ➖     |
+----------------+--------+--------+---------+-------+-------+--------+
| Total          | 943    | 1      | 184     | 9     | 1128  | ❌     |
+----------------+--------+--------+---------+-------+-------+--------+

added 2 commits

• 1ffab14d - Enable parallel_test for test-on-cng
• 31c60250 - Test parallel run

Compare with previous version

added 4 commits

• a1943c46 - Use separate variable to trigger parallel tests
• 8f564189 - Update e2e rspec retry function to support parallel report merging
• 869c1fad - Enable parallel_test for test-on-cng
• 82da110f - Test parallel run

Compare with previous version

Example run with correctly functioning retry merging: https://gitlab.com/gitlab-org/gitlab/-/pipelines/1476603810

added 53 commits

• 82da110f...c9883c18 - 50 commits from branch master
• 654959c4 - Use separate variable to trigger parallel tests
• 7db312eb - Update e2e rspec retry function to support parallel report merging
• 429e0644 - Enable parallel_test for test-on-cng

Compare with previous version

enabled automatic add to merge train when checks pass

@svistas Would You mind reviewing? Noticed one issue with the base implementation of parallel_tests

@ddieulivol Could You please review the CI part?

requested review from @ddieulivol

requested review from @svistas

changed the description

approved this merge request

added pipeline:mr-approved label

added pipelinetier-3 pipeline:run-e2e-omnibus-once labels and removed pipelinetier-1 label

Before you set this MR to auto-merge

This merge request will progress on pipeline tiers until it reaches the last tier: pipelinetier-3. We will trigger a new pipeline for each transition to a higher tier.

Before you set this MR to auto-merge, please check the following:

• You are the last maintainer of this merge request
• The latest pipeline for this merge request is pipelinetier-3 (You can find which tier it is in the pipeline name)
• This pipeline is recent enough (created in the last 8 hours)

If all the criteria above apply, please set auto-merge for this merge request.

See pipeline tiers and merging a merge request for more details.

removed pipeline:run-e2e-omnibus-once label

Hey there , could you please make sure this merge request gets merged?

The merge request is set to auto-merge, but it is not currently mergeable (MR detailed_merge_status is discussions_not_resolved).

This message was generated automatically. You're welcome to improve it.

approved this merge request

resolved all threads

started a merge train

merged

mentioned in commit 09c74835

added workflowstaging-canary label

added workflowcanary label and removed workflowstaging-canary label

added workflowstaging label and removed workflowcanary label

added workflowproduction label and removed workflowstaging label

mentioned in issue #497844 (closed)

mentioned in issue gitlab-org/quality/pipeline-triage#292 (closed)

mentioned in issue gitlab-org/quality/pipeline-triage#293 (closed)

added releasedcandidate label

added releasedpublished label and removed releasedcandidate label