Skip a scenario entirely if it doesn't include any tests
Problem
Sometimes scenarios don't actually include any tests. In those cases it's wasteful to wait for a GitLab docker container to start, run 0 tests, and then shut down the container.
E.g., this will happen if a job runs only running quarantined tests, but that particular scenario doesn't have any tests in quarantine at that time.
Here's one case: https://gitlab.com/gitlab-org/quality/nightly/-/jobs/495286497
It lists 18 quarantined tests but it doesn't actually run any of them - they're all skipped because they're not instance_saml
tests.
Solution
One option is to use rspec's --dry-run
before running the scenario itself, parse the output, and if there are 0 examples (excluding skipped examples), don't run the scenario.
Edited by Mark Lapierre