Related issue https://gitlab.com/gitlab-org/gitlab/-/issues/301061 Runner PoC to use when testing: https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/5600+ ## Problem Users need a better way to configure dynamic values for their CI jobs. They want functionality that is similar to the existing CI inputs feature for configuring dynamic values into CI configurations. User escalation issue requesting a solution for this problem: https://gitlab.com/gitlab-org/gitlab/-/issues/527952+ Today we provide variables for defining dynamic values but users have often found variables challenging to work with. Variables values are overridden in a defined hierarchy that is often hard to understand for users. Variables are also relatively unrestrained, making it hard for us to determine when their usage present a security concern. Instead a job should define what values it can receive at runtime and any other unexpected values should be rejected. ### Retrying CI jobs with new values Our first objective is to provide a solution to users that lets them retry a CI job with new values. ## Proposal Add a new job level keyword `inputs` that configures dynamic values that will be interpolated when the job is passed to the Runner. Here is an example definition: ```yaml test_job: inputs: test_echo: default: "echo me" test_file_path: default: $APP_TEST_PATH script: - echo "${{ job.inputs.test_echo }}" - rspec ${{ job.inputs.test_file_path }} ``` Default values for the inputs are defined in the job configuration. Values can be static or they can be any variable available to the Runner. Those variables are expanded by the Runner _before_ the job script(s) are executed, which will allow users to slowly migrate away from using variables directly in their job definitions. **What will the syntax for defining an input look like?** The syntax for defining a job input will match the syntax for defining a step input, which is also very similar to the syntax for defining a config input. Source of truth for step input definitions: https://gitlab.com/gitlab-org/step-runner/-/blob/0a1f15d2ff01af22cfc5651b386a4373ede6e8ee/schema/v1/spec.json#L39 **What will the syntax for using an inputs look like?** We've decided to use \`${{ job.inputs.test_input }}. This matches the syntax used for Step inputs. Discussion: https://gitlab.com/groups/gitlab-org/-/epics/17833#note_2504356979 Job inputs will be prefixed with a `job` context to make it easier for users to understand when they are using a job input and when they are using a config input. **What keywords will support inputs?** We should support inputs wherever variable expansion is currently supported on the Runner _or_ by the execution shell environment. The Runner will interpolate inputs _before_ passing them to the execution shell. Reference: https://docs.gitlab.com/ci/variables/where_variables_can_be_used/#gitlab-ciyml-file Keyword list: `after_script`, `before_script`, `script` `artifacts:name/paths/exclude` `cache:key/paths/policy` `image` `services` and `services:name` **What types will inputs have?** Job inputs will support all types that step inputs support. **How will inputs work with CI Steps?** Job inputs will be built to be forward compatible with step inputs. Discussion: https://gitlab.com/groups/gitlab-org/-/epics/17833#note_2517651353 Step runner design doc: https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/gitlab_steps/ Runner technical vision architecture: https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/runner_technical_vision/#architecture ### Future vision We could give users the option to restrict jobs so they only have access to their inputs. Any variable values they want to use must be passed in as an input. This will make job definitions easier to debug and more secure. We can also [introduce `outputs`](https://gitlab.com/gitlab-org/gitlab/-/issues/410087), which can be used to pass values between jobs. We could also make it possible to set artifacts as input values. Here's an example of what inputs could look like with outputs and artifacts: ```yaml build_job: script: - export TEST_VAR="test value" - touch test_file.rb artifacts: - name: test_file # we can use artifacts with inputs if we have a name to reference them with paths: - test_file.rb outputs: - value_for_test_job: $TEST_VAR # we can allow outputs to access variables defined in the job script test_job: inputs: project_path: default: $PROJECT_PATH test_echo: output: build_job.value_for_test_job default: "Default is used if output does not exist" test_file: artifact: build_job.test_file default: "Default is used if artifact does not exist" script: - rspec ${{ job.inputs.test_file }} - echo "${{ job.inputs.test_echo }}" - echo "${{ job.inputs.project_path }}" - echo $PROJECT_PATH # nil because it is not passed in as an argument ```