<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=405007) </details> <!--IssueSummary end--> <!-- This issue template can be used as a great starting point for feature requests. The section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated using the release post item generator: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator. The remaining sections are the backbone for every feature in GitLab. The goal of this template is brevity for quick/smaller iterations. For a more thorough list of considerations for larger features or feature sets, you can leverage the detailed [feature proposal](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md). --> ### Release notes Previously, it was not possible to track a pull-based deployment as part of a GitLab CI/CD pipeline. This made it difficult to have pre- and post-deployment jobs and to track deployments across several environments in a single place. The current GitLab release ships with an experimental version to trigger a Flux reconciliation from the pipeline. We consider this an experiment as we are looking for feedback for the best approach to track the reconciliation of the resources that actually interest our users, not just one resource thoughfully picked by us. If you would like to contribute to this discussion, your insights are welcome in <ISSUE LINK> <!-- What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). " --> ### Problem to solve As a Platform engineer, I want the pull-based deployments to be part of my deployment pipeline. If Flux watches a git ref, the sync will happen - after a new commit arrived to the watched ref - Flux's regular sync loop fires (default: 10mins) This does not allow any workflows to run [pre-](https://gitlab.com/gitlab-org/gitlab/-/issues/288307) and [post-deployment](https://gitlab.com/gitlab-org/gitlab/-/issues/363579). In https://gitlab.com/gitlab-org/gitlab/-/issues/389857+ we looked into alternative approaches where Flux would not watch the default branch, but either another branch or an OCI artifact generated from the default branch. Using an OCI artifact is likely the best approach and is inline with where the industry is going. The idea is to introduce new CI syntax to - easily build an OCI image from a given directory (like `pages` jobs deploy `public`) - push the built OCI image to the GitLab container registry - notify Flux (in the selected cluster) about a new image being available - wait for Flux to sync (or pick up an ongoing sync) - we should wait for a Kustomization ([health checks](https://fluxcd.io/flux/components/kustomize/kustomization/#health-checks), [wait](https://fluxcd.io/flux/components/kustomize/kustomization/#wait)) or Helm release to sync ([wait in Helm](https://fluxcd.io/flux/components/helm/helmreleases/#disabling-resource-waiting)), not just the source - continue the pipeline <!-- What is the user problem you are trying to solve with this issue? --> ### Current workaround 1. trigger a flux sync from the pipeline (see a demo at https://gitlab.com/gitlab-org/ci-cd/deploy-stage/demos/podtato-head-delivery/-/blob/main/.gitlab-ci.yml?ref_type=heads#L63) 1. wait for the flux resource status using `kubectl wait` ([example](https://stackoverflow.com/questions/53536907/kubectl-wait-for-condition-complete-timeout-30s)) ### Proposal The first iteration will consist of a helper script/tool that performs the actions listed above: - Create an OCI artifact from manifests contained in a directory - Push the artifact to GitLab's registry - Wait for Flux to sync the changes (either via notifying Flux explicitly, or waiting for the next cycle) The tool will be packaged into a public Docker image, which can be used by any CI job that requires the functionality. See https://gitlab.com/gitlab-org/terraform-images for a similar example. #### Definition of done - [ ] Script/tool created to package and push OCI artifact. - sync waits for the OCI image only - [ ] Image containing script/tool is available under gitlab-org/cluster-integration - [ ] Example job documented #### Out of scope (iteration 1) - Changes to CI syntax ### Intended users * [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer) ### Feature Usage Metrics - number of times when the job with the given syntax is triggered <!-- How are you going to track usage of this feature? Think about user behavior and their interaction with the product. What indicates someone is getting value from it? Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md --> <!-- Label reminders Use the following resources to find the appropriate labels: - Use only one tier label choosing the lowest tier this is intended for - https://gitlab.com/gitlab-org/gitlab/-/labels - https://about.gitlab.com/handbook/product/categories/features/ --> <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> *This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.* <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> /label ~"workflow::refinement"