Using the "BENCHMARK_CONFIG" build variable causes the same requested benchmark to be repeated in every test stage of the pipeline.
Context
If a CI build is requested via a curl command with the inclusion of a "variables[BENCHMARK_CONFIG]=<specific_test>" form the requested specific_test is executed multiple times rather than just once (with 3 repeats). The expectation is that the requested specific test should be carried out once with the default 3 repeats on its own with no other benchmark tests.
e.g. https://gitlab.com/BuildStream/benchmarks/pipelines/43886234
Task Description
Broadly the task is as follows:
-
Remove the logic that calls the specific test from the general ci_run.sh. -
Implement a specific pipeline job that carries out bespoke specific tests and skips past the other build job elements if possible.
Acceptance Criteria
Run a bespoke curl command that runs a specific test. The specific test should successfully run once. The results should not be added to the results cache and should only be available via the local build artifact.