We need to create a compelling demo video showcasing the **CI Job Inputs** feature for the upcoming release blog post. This feature allows users to define and retry CI jobs with configurable input values, providing a more intuitive and secure alternative to CI variables for dynamic job configuration. --- ## 🎯 Why This Feature Matters ### The Problem Today, users face several challenges when trying to run CI jobs with dynamic values: 1. **Complex Variable Hierarchy**: CI variables have a complex override hierarchy that's difficult to understand and debug 2. **Retry Friction**: When retrying jobs, users must manually redefine all variables, which is time-consuming when dealing with jobs that have 10-20 parameters 3. **Lack of Validation**: Variables have no type checking or input validation, and their complex override hierarchy makes it hard to predict which value will be used, creating potential security risks 4. **Limited Validation**: No built-in way to validate or restrict what values a job can receive ### Real User Needs From the original feature request ([#21676](https://gitlab.com/gitlab-org/gitlab/-/issues/21676)): - Developers want to run specific tests (e.g., `rspec --bisect` on a subset of tests) - Teams need to lint specific files without running full pipelines - Users want to preview markdown pages or run targeted debugging - Manual jobs require pre-filled default values to reduce repetitive data entry ([#301061](https://gitlab.com/gitlab-org/gitlab/-/issues/301061)) ### The Solution: Job Inputs Job inputs provide: - **Explicit contracts**: Jobs define exactly what values they can receive; unexpected values are rejected - **Better UX**: Default values can be pre-filled, with easy modification on retry - **Improved security**: Clear boundaries on what data flows into jobs - **Easier debugging**: No complex variable hierarchy to understand - **Forward compatibility**: Built to work seamlessly with CI Steps and future pipeline modularity features --- ## 🎬 Demo Requirements ### Key Use Cases to Demonstrate 1. **Defining Job Inputs** - Show a job with `inputs` keyword defining multiple input types - Demonstrate default values (static and variable-based) - Highlight the `${{ job.inputs.input_name }}` interpolation syntax Example configuration to showcase: ```yaml test_job: inputs: test_echo: default: "echo me" type: string test_file_path: default: $APP_TEST_PATH type: string environment: default: "staging" type: string script: - echo "${{ job.inputs.test_echo }}" - rspec ${{ job.inputs.test_file_path }} - echo "Running in ${{ job.inputs.environment }}" ``` 2. **Retrying Jobs with New Input Values** - Run a pipeline with default input values - Navigate to the retry UI showing pre-populated input fields - Modify one or more input values - Show the job executing with the new values - Display job logs confirming the new inputs were used 3. **Manual Jobs with Inputs** - Demonstrate triggering a manual job - Show the play form with input fields - Modify inputs before triggering - Execute and verify 4. **Supported Keywords** (if time permits) - Show inputs working in: `script`, `before_script`, `after_script` - Demonstrate usage in `artifacts:paths`, `cache:paths`, or `image` --- ## 🔧 Technical Implementation Details ### Architecture Overview **Rails/GitLab Side:** - New `inputs` keyword added to CI configuration ([#547440](https://gitlab.com/gitlab-org/gitlab/-/issues/547440)) - Database table `ci_job_inputs` stores input definitions and values ([#547439](https://gitlab.com/gitlab-org/gitlab/-/issues/547439)) - Input specifications persisted in `Ci::Build` options field ([#557879](https://gitlab.com/gitlab-org/gitlab/-/issues/557879)) - JSON schema validation on input values ([#554208](https://gitlab.com/gitlab-org/gitlab/-/issues/554208)) - Maximum 20 inputs per job enforced ([#554211](https://gitlab.com/gitlab-org/gitlab/-/issues/554211)) - Input size limits and default type handling ([#579453](https://gitlab.com/gitlab-org/gitlab/-/issues/579453)) **API & Endpoints:** - Job retry endpoints enhanced to accept input values ([#547443](https://gitlab.com/gitlab-org/gitlab/-/issues/547443)) - Manual job play endpoints support inputs ([#582076](https://gitlab.com/gitlab-org/gitlab/-/issues/582076)) - GraphQL mutation for playing manual jobs with inputs - Job payload includes inputs for Runner consumption ([#543972](https://gitlab.com/gitlab-org/gitlab/-/issues/543972)) **Runner Side:** - Runner receives unexpanded job inputs from GitLab - Uses step-runner Go library for expression interpolation (ensures forward compatibility with CI Steps) - Interpolation happens at runtime using `${{ job.inputs.* }}` syntax - Runner interpolates inputs **before** passing to execution shell - Supports interpolation in all keywords where variable expansion currently works **Input Types & Validation:** - Supports all types that [step inputs support](https://gitlab.com/gitlab-org/step-runner/-/blob/0a1f15d2ff01af22cfc5651b386a4373ede6e8ee/schema/v1/spec.json#L39) - Type definitions: `string`, `number`, `boolean`, and more - Optional `options` and `regex` validation (similar to CI config inputs) - Default values can be static strings or CI variables **UI Components:** - Retry page with input value form ([#547649](https://gitlab.com/gitlab-org/gitlab/-/issues/547649)) - Manual job play page with inputs ([#585104](https://gitlab.com/gitlab-org/gitlab/-/issues/585104)) - Input fields pre-populated with default or previous values - Form validation before job submission ### Key Design Decisions 1. **Syntax Choice**: `${{ job.inputs.input_name }}` - Uses `${{ }}` for runtime expansion (matches CI Steps) - Prefixed with `job` context to differentiate from config inputs (`$[[ inputs.* ]]`) - Clear indication to users that interpolation happens at runtime, not during YAML parsing 2. **Forward Compatibility with CI Steps** - Input types match step input specification exactly - Runner uses step-runner library for interpolation - When Steps becomes GA, job inputs will seamlessly integrate - No breaking changes required for users when migrating to Steps 3. **Interpolation Location** - Happens in Runner (not Rails) for runtime flexibility - Occurs **before** shell execution - Supports same keywords as variable expansion: `script`, `before_script`, `after_script`, `artifacts`, `cache`, `image`, `services` 4. **Backward Compatibility** - Works with existing `script`, `before_script`, `after_script` keywords - No requirement to use CI Steps `run` syntax - Gradual migration path for users ### Related Technical Work **Completed:** - PoC: Runner interpolation ([gitlab-runner#38952](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/38952), [&19756](https://gitlab.com/groups/gitlab-org/-/work_items/19756)) - Database schema and models ([#547439](https://gitlab.com/gitlab-org/gitlab/-/issues/547439)) - CI config keyword implementation ([#547440](https://gitlab.com/gitlab-org/gitlab/-/issues/547440)) - Input cloning in Build model ([#554204](https://gitlab.com/gitlab-org/gitlab/-/issues/554204)) - Validation and limits ([#554208](https://gitlab.com/gitlab-org/gitlab/-/issues/554208), [#554211](https://gitlab.com/gitlab-org/gitlab/-/issues/554211), [#579453](https://gitlab.com/gitlab-org/gitlab/-/issues/579453)) - Retry endpoint support ([#547443](https://gitlab.com/gitlab-org/gitlab/-/issues/547443)) - Metrics tracking ([#580221](https://gitlab.com/gitlab-org/gitlab/-/issues/580221)) **In Progress:** - Feature flag rollout ([#578838](https://gitlab.com/gitlab-org/gitlab/-/issues/578838)) - Documentation ([#585107](https://gitlab.com/gitlab-org/gitlab/-/issues/585107), [#577833](https://gitlab.com/gitlab-org/gitlab/-/issues/577833)) - UI pages ([#547649](https://gitlab.com/gitlab-org/gitlab/-/issues/547649), [#585104](https://gitlab.com/gitlab-org/gitlab/-/issues/585104)) - Manual job support ([#582076](https://gitlab.com/gitlab-org/gitlab/-/issues/582076)) - Runner features flag ([#588590](https://gitlab.com/gitlab-org/gitlab/-/issues/588590)) --- ## 🎥 Demo Script Suggestions ### Opening (15-20 seconds) **Problem Statement**: "When running CI jobs, teams often need to retry with different parameters or run specific tests. Today, this requires manually updating all variables, which is time-consuming and error-prone. Variables also have a complex override hierarchy that's hard to debug." ### Feature Introduction (30-40 seconds) - Show `.gitlab-ci.yml` with job inputs defined - Highlight the clean syntax with default values - Explain how inputs provide an explicit contract: "Jobs define exactly what values they accept" - Mention security benefit: "Unexpected values are rejected" ### Live Demo (60-90 seconds) 1. **Initial Run**: Show pipeline running with default input values in job logs 2. **Retry with New Values**: - Navigate to completed job - Click retry button - Show the retry form with input fields pre-populated with defaults - Modify 2-3 input values (e.g., change test file path, environment) - Submit and show job running 3. **Verification**: Show job logs displaying the new input values being used 4. **Manual Job** (bonus): Trigger a manual job, modify inputs in play form, execute ### Value Proposition (15-20 seconds) - "Job inputs make it easy to retry jobs with new values" - "No complex variable hierarchy to understand" - "Better security with explicit input contracts" - "Forward compatible with CI Steps for future pipeline modularity" --- ## 🔗 Related Resources - **Epic**: [&17833 CI Job Inputs](https://gitlab.com/groups/gitlab-org/-/work_items/17833) - **Parent Epic**: [&16739 Pipeline Modularity](https://gitlab.com/groups/gitlab-org/-/work_items/16739) - **Original Feature Requests**: - [#21676 - Ability to run CI jobs with input](https://gitlab.com/gitlab-org/gitlab/-/issues/21676) - [#301061 - Prefilled variables for manual jobs](https://gitlab.com/gitlab-org/gitlab/-/issues/301061) - **Step Runner Input Spec**: [spec.json](https://gitlab.com/gitlab-org/step-runner/-/blob/0a1f15d2ff01af22cfc5651b386a4373ede6e8ee/schema/v1/spec.json#L39) - **CI Steps Documentation**: [docs.gitlab.com/ci/steps](https://docs.gitlab.com/ci/steps/)