Skip to content

16.11 self-managed installations error when passing `emails_disabled` on groups API

Context

In %16.11, we unintentionally broke the Groups API, leading to an incident on gitlab.com: gitlab-com/gl-infra/production#17775 (closed)

This was fixed on gitlab.com by !148577 (merged), behind a new feature flag: invert_emails_disabled_to_emails_enabled.

The invert_emails_disabled_to_emails_enabled flag was enabled globally on gitlab.com, which fixed the issue immediately.

The invert_emails_disabled_to_emails_enabled flag was removed in %17.0, and replaced by a longer-term fix: !151065 (merged).

Problem

Reported initially in !127899 (comment 1947504617).

Self-managed customers upgrading to %16.11 will experience the same incident as gitlab.com did, and receive 500s when using the Groups API and specifying emails_disabled.

To fix the issue it, they must either:

  1. Enable the invert_emails_disabled_to_emails_enabled feature flag, since it is disabled by default in %16.11.
  2. OR upgrade to %17.0

Options

  1. Option 1: Document that invert_emails_disabled_to_emails_enabled should be enabled when upgrading to %16.11: https://docs.gitlab.com/16.11/ee/api/groups.html
  2. Option 2: Backport a fix to %16.11 which will default the flag to true: https://gitlab.com/gitlab-org/gitlab/-/blob/v16.11.4-ee/config/feature_flags/gitlab_com_derisk/invert_emails_disabled_to_emails_enabled.yml#L9

Considerations

%16.11 is the last version before a major upgrade (%17.0), so customers might stay on %16.11 longer until they've accommodated any breaking changes from %17.0.

Decision

Going with option 2, see #467690 (closed).

Edited by François Rosé