<!--The format of the title should be Title should follow the format `Breaking change exception: Description`--> # Breaking Change Exception Request ## Executive summary GitLab chart and GitLab Operator expose GitLab with a bundled NGINX Ingress controller by default. NGINX Ingress was [recently retired](https://kubernetes.io/blog/2025/11/11/ingress-nginx-retirement/) and will receive no further updates after March 2026. As a replacement GitLab chart adopts Gateway API and Envoy Gateway. Gateway API is the official and recommended Ingress replacement upstream, and Envoy is a component that was [planned for adoption in all GitLab install methods](https://gitlab.com/gitlab-org/architecture/auth-architecture/design-doc/-/merge_requests/39) anyway. More detailed reasoning on why GitLab chart choose Envoy (Gateway) as replacement is [documented in the architecture decisions](https://gitlab.com/gitlab-org/charts/gitlab/-/blob/master/doc/architecture/decisions.md#why-bundle-a-gateway-controller). With us being forced to migrate away from NGINX Ingress, we propose to: * Retire support for all bundled Ingress controllers (NGINX Ingress, HAPRoxy, and Traefik) in 19.0. The component will remain to be available, but production deployments should move to the bundled Envoy Gateway. * The retirement of HAProxy and Traefik in this context allows us to focus more on Gatway API and to reduce maintenance overhead. Both components are alternatives to NGINX Ingress, but are rarely used by GitLab chart users. * Default to Envoy Gateway/Gateway API starting with 19.0. NGINX Ingress, HAPRoxy, and Traefik remain available but are deprecated and disabled by default. * Fully drop all deprecated bundled Ingress controllers in 20.0. ### Does your breaking change meet one of these three criteria? 1. The impact of the breaking change has been **fully mitigated via an automated migration** that requires no action from the customer. - Ingresses are very component and environment specific and can't be migrated automatically. 2. The breaking change will have **negligible customer impact** as measured by actual product usage tracking across Self-Managed, GitLab.com, and Dedicated. For instance if it impacts less than 1% of the GitLab customer base. - The amount of impacted users is very high, but we are pushed to this deprecation from upstream. 3. The breaking change is being implemented due to a **significant security risk**- Severity 1 or 2. - After March 2026 we won't be able to receive security updates for NGINX Ingress (chart). - We are currently unable to patch Traefik and HAProxy regulary, because of regular potential breaking changes. If future high severity security issues are identified, we won't be able to patch these components. ## Impact assessment ### How many users and customers are impacted? What Tier/ARR of customer? * Impact: MEDIUM * Impacted install methods: GitLab chart and Operator * Impacted tiers: all Impact assessment: * While almost all chart and Operator instances are impacted we expect to be able to migrate the majority of instances with minimal disruption. * Geo instances, instances with multiple Webservice deployment, or instances which changed the Ingress configuration might need additional manual changes. * Customers with needing to reduce the upgrade downtime or with custom can operate Envoy and NGINX Ingress site by site and slowly migrate traffic over by updating DNS entries one by one. ### Can we get the same outcome without a breaking change? No ### When do you want to make the breaking change? * Change of default and support status in %"19.0". * Gateway API and Envoy Gateway are the new default. * Ingress objects are disabled by default. * The bundled Ingress controllers (NGINX Ingress, Traefik, and HAProxy) are deprecated and disabled by default (NGINX is currently our default). * Users can continue to bring their own Ingress controller or reenable Ingresses and the bundled controller of their choice. * Removal of deprecated Ingress controllers in %20.0. ### What is the alternative for customers to do the same job the change will break? Users can either migrate to a self-managed Ingress controller or change their configuration to use the old defaults until 20.0. ### How difficult is it for customers to migrate to the alternative? Is there a migration plan? The heavily depends on the usage pattern. Basic installs can be migrated with no manual configuration changes by changing the default GitLab chart and Operator configuration. ## Rollout & Communication plan ### Have we handled similar breaking changes in the past? What can we learn from the success or failure of those methods? No ### Are you employing any strategies to lessen the customer or support impact? Proposed rollout/communication: * 18.7: * Start supporting Gateway API via Envoy Gateway * Announce default change for 19.0 and removal for 20.0 * Prepare documentation on how to switch to an external Ingress controller or API Gateway * 19.0: * Default to Gateway API via Envoy Gateway. Disable Ingresses and the bundled Ingress controllers by default. * Officially deprecate the bundled Ingress controllers (NGINX Ingress, Traefik, and HAProxy). * `Ingresses` itself are not being deprecated and continue to be supported going forward (both by GitLab chart and Operator, and by Kubernetes itself). * Goal: Incentivive users to migrate and fresh installs to use the more future proof component. * 20.0: * Drop deprecated Ingress controllers. * `Ingresses` can still be enabled on demand - But require users to bring their own Ingress controller. * Goal: Make sure all instances have migrated off NGINX Ingress. ### Internal Communication - Engineering, Product, Security, Customer Success, and Sales: What questions do you anticipate they will have? What is it important that they know? - Support: What guidance will the support team need to handle customer tickets related to this change? - Once you receive approval to proceed with the breaking change you should create a [Support Preparedness issue](https://gitlab.com/gitlab-com/support/support-team-meta/-/blob/master/.gitlab/issue_templates/Support%20Preparedness.md?ref_type=heads) with instructions on how to handle customer queries and concerns. ### Customer Communication What form will this take? * A breaking change announcement. * Users enabling a bundled Ingress controller will be notified with a custom upgrade note. ## Approval Process * [x] Tag in the appropriate Product Director and Engineering Director for your product area. They may ask for further clarifications. * [x] Approved by **Product Director or GPM**: @fzimmer * [x] Approved by **Engineering Director**: @chsanders * [x] Once you have approval from both Product GPM or Director & Engineering Director, add the label "Breaking Change: Awaiting Support Approval" and remove the prior label. * [x] Share this issue in the [#support_leadership](https://gitlab.enterprise.slack.com/archives/C01F9S37AKT) slack channel and ask for a review from the support perspective. Once you get signoff from 1 Support Senior Leader, add the label "Breaking Change: Awaiting VP Approval" and remove the prior label. https://gitlab.slack.com/archives/C01F9S37AKT/p1769771444258419 * [x] Approved by **Support Stable Counterpart (If Applicable)**: @jessie * [ ] Approved by **Support Senior Leader**: \< Add Name \> * [x] Tag in the appropriate Product and Engineering VPs or Senior Directors for your product area. They may ask for further clarifications. * [x] Approved by **Product VP or Senior Director**: @fzimmer * [x] Approved by **Engineering VP or Senior Director**: @marin * [x] Once you have approval from both the Engineering & Product VP or Senior Director, it will be added to the agenda of the Prod + Eng Weekly Strategy meeting for visibility and a chance for input from other ELT/PLT members. This step is dependent on the "Breaking Change: VP Approved" label being correctly applied. * [x] Seek approval on any customer comms from the **Corporate Communications** and **Customer Success** teams Once you have approval, create a [public deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Deprecations.md) that will serve as a source of truth for customers in regards to the change. Ensure the change is added to the deprecations docs page by following [this guidance](https://docs.gitlab.com/development/deprecation_guidelines/#update-the-deprecations-and-removals-documentation). **Please keep this issue open** until the breaking change has been implemented. This ensures all breaking changes are visible to leadership on [this board](https://gitlab.com/gitlab-com/Product/-/boards/9741901).