Ability to run CI jobs with some input. E.g. tell it which spec to run, or which tests to run rspec --bisect

Why

Sometimes it is useful to be able to run a job with some specific input, such as a file to lint, a test to re-run, a markdown page to preview or a set of tests to run rspec --bisect on. Currently I'd do this by using gitlab-runner locally after editing the .gitlab-ci.yml file and committing changes. This is awkward to set up and then slows my computer down while it runs.

What

Ability to feed input into the CI job. Perhaps by selecting a file or giving custom input. The CI runner will then set this input as an variable which I could use when writing a CI job.

Related

  • This came up in this week's backend call. @reprazent suggested the ability to run bisect a set of tests to find an order dependant failure.

  • I mentioned something like this in https://gitlab.com/gitlab-org/gitlab-ce/issues/44000#note_62324755:

    I often run rubocop locally with git diff master-green... --name-only | egrep "(\.rb|ruby)$" | grep -v 'schema.rb'| grep -v 'db/fixtures/' | xargs bundle exec rubocop to catch these as soon as possible. I've also sometimes attempted gitlab-runner exec docker rspec-local after editing .gitlab-ci.yml to run just a handful of specified tests, with mixed success.

  • @DouweM mentioned that there was something similar in https://gitlab.com/gitlab-org/gitlab-ee/issues/5281, where we wanted to be able to run tests for files open in the WebUI without running tests for other files.