<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> *This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.* <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> ### Release notes Creating a complex pipeline configuration requires trial and error, and every time you commit a faulty configuration you need to wait for the pipeline to run to realize there's a problem. Currently, it's possible to validate the pipeline syntax statically in the pipeline editor, but we don't evaluate the pipeline logic like `rules` and `needs` job dependencies. In this release, we've added the ability to test the pipeline creation before running the pipeline, which takes into consideration the different rules you've configured on your pipeline. ## 🧩 Problem Creating a complex pipeline configuration requires trial and error, and every time you commit faulty configuration you need to wait for the pipeline to run to realize there's a problem. Currently, it's possible to validate the pipeline syntax statically in the pipeline editor, but we don't evaluate the pipeline logic like `rules` and `needs` job dependencies. The standalone CI Lint tool offers an option to simulate the pipeline for the default branch, but that feature is not discoverable, not well documented, and is difficult to understand what it does from how it's presented in the Linter UI. #### Intended users * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) #### Metrics - We can measure the usage of this feature by tracking the click events on the "Validate pipeline" button. - We will further gather feedback on this feature by linking a [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/346687) in the `?` popover next to the dropdown. :point_up: Feedback issue needs to be refined. <!-- How are you going to track uage of this feature? Think about user behavior and their interaction with the product. What indicates someone is getting value from it? --> #### User experience goal <!-- What is the single user experience workflow this problem addresses? For example, "The user should be able to use the UI/API/.gitlab-ci.yml with GitLab to <perform a specific task>" https://about.gitlab.com/handbook/engineering/ux/ux-research-training/user-story-mapping/ --> The user should be able to validate their pipeline under selected conditions. In the MVC iteration, only one type of condition will be available (a git push event for the default branch). ## ✨ Proposal - Move the existing pipeline simulation functionality to the pipeline editor Validate tab (currently called Lint). - The tab content will be replaced with the simulation functionality which will validate the pipeline on demand (including the syntax and logic). #### Feature acceptance criteria - In order to drive the feature usage and encourage feedback, we will show a `NEW` badge in the Validate tab the first time the user opens the Validate tab after the feature release. The badge should disappear once you close the Validate tab after your first interaction with it upon the feature release. - The `Pipeline source` dropdown with a pre-selected option will be added to the Validate tab. No other choices will be available yet and the button will be disabled. In the future we will add more options and inputs to that controls bar. - The `?` button will open a popover with supplemental content and docs + [feedback issue links.](https://gitlab.com/gitlab-org/gitlab/-/issues/346687) - The simulation empty state will explain the details of what happens during the simulation. - The simulation will run with the current content of the Edit tab input. - While the simulation is running, we will show a loading state. - Simulation can be stopped by clicking on the "Cancel" button. - Closing the page should continue running the simulation in the background. We will only stop the simulation if you cancel it by clicking on the `cancel` button as the simulation is running. - When the simulation completes, we will show either a "success" or "error" alert **with updated UI copy** to indicate whether any errors were found in the simulation. The error messages will appear below the alert in case there are errors. - The continuous syntax validation widget will work unrelated to the results of the simulation and will only show the syntax status. The text of the syntax validation widget will be updated to reflect the scope of that validation (syntax only). - Closing the Editor page will reset content of the Validate tab to the empty state. - If the contents of the configuration or the branches change, we'll show a UI text note next to the Validate button notifying the developer that the config has changed. #### Design specs :link: [Designs](https://gitlab.com/gitlab-org/gitlab/-/issues/337282/designs/Validate_tab_-_badge.png) :link: [Figma specs](https://www.figma.com/file/O1iQx5kEr3UK1cTYu1qFnH/%F0%9F%92%9CPipeline-Editor?node-id=1924%3A69142) ### Further suggested improvements (out of scope in this issue) <details><summary>Click to expand</summary> - Add additional pipeline conditions and variables support - In the future we could reuse the same design in the Visualise tab to allow you to see what the pipeline will _look_ like with certain conditions and/or inputs. - It would be great to make this functionality available as an extension of sort for those GitLab CI users who prefer to use their local editor for writing the configuration. * User would like to know about these logic errors in real time, without having to run the simulation. * Users would like to know more information about the different pipeline conditions. * The errors should be more actionable, making it easy to fix the problem in the pipeline editor. (ex. highlight the place where the error occurs in the editor and link to that place from the error). * User would like to see the output of what worked correctly in the simulation, up until the error occurred, to see a trace of what happened and where the pipeline configuration "broke". * The output formatting needs improvement. * The status of the latest pipeline can be confused for the status of the current pipeline configuration or simulation. </details> ### Permissions and Security According to the current implementation you need to have permissions to push to the the default branch to use the simulation. When you have insufficient access, the current simulation implementation allows you to run the simulation in the UI, but throws an error about insufficient permissions. This is far from ideal experience and needs to be addressed as a follow-up to the MVC. > You must have permissions to create pipelines on this branch to validate with a simulation. <!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)? Consider adding checkboxes and expectations of users with certain levels of membership https://docs.gitlab.com/ee/user/permissions.html * [ ] Add expected impact to members with no access (0) * [ ] Add expected impact to Guest (10) members * [ ] Add expected impact to Reporter (20) members * [ ] Add expected impact to Developer (30) members * [ ] Add expected impact to Maintainer (40) members * [ ] Add expected impact to Owner (50) members --> ### Documentation This feature needs to be documented in detail on the pipeline editor page, as well as any other page in the docs where we talk about pipeline validation/ linting. We need to cover exactly what checks the simulation performs, as well as its limitations. We can also link the feedback issue in the docs to encourage users to provide feedback. We should make our vision clear that this feature will evolve into allowing you to set specific conditions like branch, pipeline source, variables, etc, to give context for this MVC feature. <!-- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/workflow.html#for-a-product-change * Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/workflow.html * If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html --> ### Implementation Table | ~frontend | MR | | ------ | ------ | | Introduce feature flag to replace `Lint` tab with `Validate` tab in the pipeline editor | https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89497 | | Add popover and `NEW` badge to the `Validate` tab | https://gitlab.com/gitlab-org/gitlab/-/merge_requests/90345 | | Add simulation functionality to the `Validate` tab | https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91241 | | Roll out feature flag | https://gitlab.com/gitlab-org/gitlab/-/issues/364257 | ### Availability & Testing <!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning --> ### Available Tier * Free <!-- This section should be used for setting the appropriate tier that this feature will belong to. Pricing can be found here: https://about.gitlab.com/pricing/ * Free * Premium/Silver * Ultimate/Gold -->