## Background - In %"16.6", we’ve announced the Beta release of the CI/CD components :white_check_mark: - In %"16.7", we've announced the Beta release of CI/CD catalog :white_check_mark: Our vision for the catalog is to be a single location where users can access and consume CI/CD related offerings (Components, Steps, etc…) We would like to encourage our users to move away from Templates to Components, today we have \~100 different GL-maintained templates which would need to be converted into components and published ## Process You can watch a video of a refactoring from CI template to CI component: https://www.youtube.com/watch?v=dGCPrIAuBmE 1. Pick a template to convert (from the list below or a dependency of the templates below). 2. Convert the template to components 3. Decide whether the component should have its own project or whether multiple components should be bundled together in the same project. Example: [Code Quality](https://gitlab.com/components/code-quality) is a good example of a project just containing that component. AWS could be a project hosting multiple components from AWS-related templates. 4. Create a project in https://gitlab.com/components or ask `@fabiopitino` to assist if any problems. 5. Follow the [conversion docs](https://docs.gitlab.com/ee/ci/components/#convert-a-cicd-template-to-a-component). 6. Ensure that the component could be eventually reused where currently the template is used (e.g. in the [Auto-Devops](https://gitlab.com/gitlab-org/gitlab/-/blob/143249ee3a6bc93c00e69eebc80698d4cff0aae0/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml#L173) template). No need to replace the template with the component because it would break self-managed instances. We only need to ensure it COULD be replaced eventually, for example that that all required inputs are defined. 7. Get the whole project reviewed by `@fabiopitino` and another member of gitlab~16116820. 8. Release and publish the component project in the catalog. 9. Determine the ownership and maintenance of GitLab-maintained CI Components after creation. 10. In the original template open an MR which adds a comment with a link to the location of the converted component which explains that any change that is made to this templates needs to be duplicated to the associated component 11. Ping the respective team in the MR so they are aware of this change ## List List of popular templates (based on https://app.periscopedata.com/app/gitlab/785953/Pipeline-Authoring-Dashboard?widget=13440051&udv=1204805) | Template | Issue | Migrated by | Done? | Tested by | Owned by | |----------|-------|-------------|-------|-----------|----------| | [ADO](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) | https://gitlab.com/gitlab-org/gitlab/-/issues/433263 | | Convert this at the end in [autodevops](https://gitlab.com/components/autodevops), when all dependencies are done | | gitlab~29601784 | | [SAST](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml) | | @fabiopitino | :white_check_mark: https://gitlab.com/components/sast | gitlab~10690740 | gitlab~10690740 | | [SAST IaC]() | | | We could include it as separate component in [sast](https://gitlab.com/components/sast) project | | ~"group::static analysis" | | [Code-Quality](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/Code-Quality.gitlab-ci.yml) | | @fabiopitino | :white_check_mark: https://gitlab.com/components/code-quality | gitlab~10690740 | gitlab~10690740 | | [Helm-2to3](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/Helm-2to3.gitlab-ci.yml) | https://gitlab.com/gitlab-org/gitlab/-/issues/433262 | | :x: currently not popular | | gitlab~29601784 | | [Code-Intelligence](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/Code-Intelligence.gitlab-ci.yml) | https://gitlab.com/gitlab-org/gitlab/-/issues/433261 | @kisha.mavryck to assign to [team](https://gitlab.com/groups/gitlab-org/-/epics/12289#note_1716008468) | | | gitlab~16934793 | | [ECS](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy/ECS.gitlab-ci.yml) | | @fabiopitino | :warning: https://gitlab.com/components/aws | help needed | gitlab~29601784 | | [EC2](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy/EC2.gitlab-ci.yml) | | @fabiopitino | :warning: https://gitlab.com/components/aws | help needed | gitlab~29601784 | | [Test](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/Test.gitlab-ci.yml) | https://gitlab.com/gitlab-org/gitlab/-/issues/433265 | @f_caplette | :x: No longer supported | No longer supported | gitlab~29601784 | | [Build](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/Build.gitlab-ci.yml) | https://gitlab.com/gitlab-org/gitlab/-/issues/433266 | @lauraX | README needs to be created | help needed | gitlab~29601784 | | [Deploy](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) | | | we can include in the [autodevops](https://gitlab.com/components/autodevops) project | | ~"group::environments" | | [Dependency scanning](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.gitlab-ci.yml) | https://gitlab.com/gitlab-org/gitlab/-/issues/433267 | @Kasia_Misirli | | | gitlab~10690742 | | [license_scanning](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/License-Scanning.gitlab-ci.yml) | https://gitlab.com/gitlab-org/gitlab/-/issues/433268 | | | | gitlab~10690742 | | [DAST](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Security/DAST.gitlab-ci.yml) or [DAST.latest](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Security/DAST.latest.gitlab-ci.yml) understand which to convert | https://gitlab.com/gitlab-org/gitlab/-/issues/433691 | @avielle | | | gitlab~10690741 | | [DAST-Default-Branch-Deploy](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/DAST-Default-Branch-Deploy.gitlab-ci.yml) | https://gitlab.com/gitlab-org/gitlab/-/issues/433269 | @avielle | | | gitlab~10690741 | ### DevRel focus based on https://gitlab.com/groups/gitlab-org/-/epics/12066#note_1654990982 and https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates #### Programming languages See https://gitlab.com/groups/gitlab-org/-/epics/12290 #### Container images, Security, Mobile See https://gitlab.com/groups/gitlab-org/-/epics/12291 ## How to test Each component project should have a `.gitlab-ci.yml` in it that includes the components defined in the project. See an example here: https://gitlab.com/components/aws/-/blob/main/.gitlab-ci.yml?ref_type=heads. - The **basic goal** (required) is to include the components and get the pipeline pass. - Some [pipelines may be failing](https://gitlab.com/components/aws/-/pipelines/1090835013) because of missing pre-requisites. - We need to ensure that jobs included with the component are executed successfully. - We also recommend (whenever possible) to have jobs in `.gitlab-ci.yml` that verify the side-effects introduced by the component. For example: if a component generates Code Quality report, in a subsequent job in the pipeline we check that a report artifact is created correctly.