Implementation of Breaking changes in .com deployments
NOTE - 13.0 IS SCHEDULED FOR MAY 22 AT THIS TIME (NO CHANGES) - Delivery will be pushing features sooner on .com
Presently GitLab is planning to release our next major release on May 22nd: https://docs.gitlab.com/ee/policy/maintenance.html#versioning
Delivery will be rolling features using CD. So during the month breaking changes could be implemented before May 22nd.
These interruptions could have an impact on our customer experience and perception of inadequate communications.
Release process for breaking changes
Feature flags should be our first option that we consider. If you put a changing behavior behind a feature flag, you can remove it and the possible code in a patch release.
Breaking changes behind a feature flag
- Set feature flag such that the breaking change is enabled by default
- Notify release managers - exact process TBD
- Once the update is published to .com, disable the breaking change
- On the 22nd during release, we re-enable the feature flags to enable the breaking changes
Breaking changes that cannot be behind a feature flag
-
BEFORE the MR changing behavior is merged, include
@gitlab-org/release/managers
group as approvers for the MR, label the MR with "breaking change" label. - Raise an issue at https://gitlab.com/gitlab-org/release/tasks/
- In the issue, provide a link to MR(s) and a short description of impact on GitLab.com users.
- Provide details of how communication to users of GitLab.com will be done.
- Ping
@gitlab-org/release/managers
so they can make plans on whether they can do a controlled release to .com. - Release managers will attempt to provide an estimated timeline, but be advised that due to timezones the timeline for release might be very short.
- It is crucial that all "breaking change" MR's are fully ready, which means reviewed, with passing pipelines NO LATER THAN 2020-05-15. We will not be considering this special process for breaking change past that date. Breaking change MR's past this date will go as regular MR's through our regular process.
Stolen from #897 (closed)
Changes which require a major release
-
Must Have:
Yes
if the change is required for %13.0 due to the impact of waiting another year -
On Track:
✅ if progress is going well,⚠ if at risk,❌ if it will not make it without a delay. -
.com affected:
✅ if it affects .com users,❌ if it does not affect .com users - MR-timeframe: List of MR changing compatibility and approximate timeframe for merge (week).
It would be great if we could update the items in the list by Monday of next week so we can get a sense of where we stand.
✅
Enablement Currently devopsenablement is on track with breaking changes that require %13.0.
DRI | Change (Issue/Epic) | Must Have? | On Track? | .com affected | MR-timeframe | Affect on customers |
---|---|---|---|---|---|---|
@mendeni | PG11 and removal of 9.6 & 10 (gitlab-org/omnibus-gitlab#5099 (closed) gitlab-org&2414 (closed)) | Yes | No | GitLab.com is scheduled to upgrade on May 10th gitlab-com/gl-infra&106 (closed) | >= PG11 is required | |
@mendeni | Puma opt-out (gitlab-org&954 (closed)) | Yes | No | Puma was fully rolled to .com on Feb 19 gitlab-com/gl-infra/production#1684 (closed) gitlab-org/omnibus-gitlab!4141 (merged) gitlab-org/omnibus-gitlab!4165 (closed) | Puma or Unicorn is supported | |
@mendeni | Remove node['gitlab'] monitoring deprecation in next major release(gitlab-org/omnibus-gitlab#4687 (closed)) | No | No | gitlab-org/omnibus-gitlab!4139 (merged) | Old settings are removed | |
@mendeni | Remove protected paths throttle from Omnibus (gitlab-org/omnibus-gitlab#4750 (closed)) | No | No | gitlab-org/omnibus-gitlab!4149 (merged) | Protected paths throttling doc and logic removed | |
@mendeni | Remove support for and references to postgresql['data_dir'] (gitlab-org/omnibus-gitlab#4806 (closed)) |
No | No | Customers need to use postgresql['dir'] instead |
||
@mendeni | Remove Grafana reset logic during upgrade in GitLab 13.0 (gitlab-org/omnibus-gitlab#4891 (closed)) | No | No | gitlab-org/omnibus-gitlab!4155 (merged) | Should be no impact | |
@mendeni | Rename Redis slave to replica (gitlab-org/omnibus-gitlab#5141 (closed)) | No | No | gitlab-org/omnibus-gitlab!4168 (merged) | Will require a redis restart |
Secure stage
Currently devopssecure is on track with breaking changes that require %13.0.
DRI | Change (Issue/Epic) | Must Have? | On Track? | .com affected | MR-timeframe | Affect on customers |
---|---|---|---|---|---|---|
@gonzoyumo | Remove Docker in Docker requirement: gitlab-org/gitlab#37278 (closed) - Dep Notice | Strong Desire | Yes | Need to update CI config when customization is based on job name | ||
@gonzoyumo | Drop support for license_management CI job, artifact and template License-Management.gitlab-ci.yml : gitlab-org/gitlab#8912 (closed)
|
? | Yes | None if already migrated to new template, otherwise need to update CI to point to new file | ||
@twoodham | Transition to rules syntax for Secure's vendored templates: gitlab-org&2300 (closed) | Yes | Yes | Task to merge all our template changes expected week of May 11th | Customers overriding Secure templates AND using only/excludes [will need to migrate to rules](https://docs.gitlab.com/ee/user/application_security/#transitioning-your-onlyexcept-syntax-to-rules)
|
|
@gonzoyumo | Remove deprecated variable PIP_DEPENDENCY_PATH in gemnasium-python: gitlab-org/gitlab#13305 (closed)
|
? | Yes | Use new variable DS_PIP_DEPENDENCY_PATH instead |
||
@gonzoyumo | Remove deprecated proprietary report format (clair scanner) for Container Scanning: gitlab-org/gitlab#35442 (closed) | ? | Yes | Make sure to use new template and new job definition for Container Scanning | ||
@sethgitlab | Remove the legacy way of using DAST: gitlab-org/gitlab#31866 (closed) | Yes | Yes | After 2020-05-22 | Need to update old templates | |
@gonzoyumo | Remove undefined value from Severity and Confidence: gitlab-org/gitlab#202685 (closed)
|
? | Yes | No longer rely on or expect undefined value but use unknown instead |
||
@twoodham | Drop usage of x-y-stable docker images and replace with pining to Major version: gitlab-org/gitlab#207128 (closed) - Dep Notice
|
Strong Desire | No | |||
@gonzoyumo | Remove the legacy cve property from the JSON report: gitlab-org&2791
|
Strong Desire | Yes | No long use the cve property but use id instead and make sure to provide location properties |
Defend stage
Currently ~"devops::defend" is on track with breaking changes that require %13.0. DRI: @whaber
No breaking changes from the defend stage.
deprecation issues for 13.0
Summarized list of Runner
✅
Create:Editor DRI | Change (Issue/Epic) | Must Have? | On Track? | .com affected | MR-timeframe | Affect on customers |
---|---|---|---|---|---|---|
@dsatcher | Remove Snippet Content from search (UI and API breaking change) | Yes | Yes | Merged | Snippet content search API will return 500 |
Release:Progressive Delivery
DRI | Change (Issue/Epic) | Must Have? | On Track? | .com affected | MR-timeframe | Affect on customers |
---|---|---|---|---|---|---|
@csouthard | Legacy Feature Flags Deprecation Plan | No | Yes | Multiple MRs; Targeting 13.1 | Old "V1" Feature Flags will become read-only in 13.4 |
Release:Release Management
DRI | Change (Issue/Epic) | Must Have? | On Track? | .com affected | MR-timeframe | Affect on customers |
---|---|---|---|---|---|---|
@sean_carroll | Deprecate Pages Legacy Architecture | Yes | Yes | MR Closed | Pages architecture will be opt -out so most people will already have the API config enable and disk source will be removed in 14.0 | |
@sean_carroll | Remove deprecated Release Evidence endpoints | Yes | Yes | MR In Review | Removed fields accessed differently in API | |
@sean_carroll | Pages Secrets Files | No | Yes | MR In Review | Won't be able to pass secrets on CLI |
Configure:Configure
DRI | Change (Issue/Epic) | Must Have? | On Track? | .com affected | MR-timeframe | Affect on customers |
---|---|---|---|---|---|---|
@nicholasklick | Default Auto DevOps Postgres install to V2 | Yes | Yes | MR In Review | With gitlab-org/gitlab#211890 (closed) implemented, the impact on pipelines utilizing the legacy option is that the pipeline will refuse to proceed. User will have to additionally opt-in to switch from legacy to new option as this actively deletes the legacy DB. | |
@nicholasklick | Moving Auto DevOps jobs syntax to rules |
Yes | Yes | MR In Review | Impacted users (small percentage?) will see their pipeline fail to start with something like: jobs:<job-name> config key may not be used with rules
|
|
@nicholasklick | Change k8s api endpoints from deprecated beta to apps/v1 |
Yes | Yes | MR In Review | If user is using Kubernetes 1.9 and below, they will need to upgrade their Kubernetes cluster in order to get apps/v1 . Note in https://docs.gitlab.com/ee/topics/autodevops/#requirements, we require Kubernetes 1.12+. If the user's cluster is Kubernetes 1.9 or below, this breaking change will cause them to an error message |
Monitor:APM
DRI | Change (Issue/Epic) | Must Have? | On Track? | .com affected | MR-timeframe | Affect on customers |
---|---|---|---|---|---|---|
@mnohr | Elastic Helm chart deprecation | Yes | No | MRs merged | If a user upgrades to the supported chart, their existing data in ElasticStack will be removed | |
@mnohr | Deprecate influxDB | Yes | No | MRs merged | Only support prometheus going forward |
Verify:Continuous Integration
DRI | Change (Issue/Epic) | Must Have? | On Track? | .com affected | MR-timeframe | Affect on customers |
---|---|---|---|---|---|---|
@cheryl.li | Remove JenkinsDeprecatedService |
Yes | Yes | Merged gitlab-org/gitlab!31607 (merged) | Removes the deprecated CI service, JenkinsDeprecatedService and associated DB records |
✅
Manage: Access DRI | Change (Issue/Epic) | Must Have? | On Track? | .com affected | MR-timeframe | Affect on customers |
---|---|---|---|---|---|---|
@lmcandrew | Remove deprecated /admin/application_settings redirect |
Yes | Yes | Merged | No effect unless a self-managed administrator had the endpoint bookmarked |
cc: @marin @joshlambert