Handle incoming service desk emails with a custom project_key

Summary

Service Desk supports a custom project_key so that an incoming address such as incoming+<full-path-slug>-<project_key>@... routes an email to a specific project. This identification form was deferred when extracting email identification into the gitlab-email_handler gem and routing email to cells, because the value it needs is not yet claimed in the Topology Service.

This issue tracks claiming the routable value so that custom-project_key Service Desk emails can be routed directly to the owning cell, like all other incoming email.

Background: why this was deferred

The other incoming email formats carry something that is already (or can be made) globally unique and resolvable offline (no DB):

  • project id keys → resolved by project_id
  • partitioned reply keys → namespace id decoded offline from the key
  • legacy project paths → resolved by the top-level namespace (a claimed route)
  • custom Service Desk emails → resolved via a Topology Service Classify lookup on the custom email address (itself globally unique)

The custom project_key form was simply not claimed yet, so the gem's :service_desk matcher returns no routable target for it.

Key finding: the address is unambiguously splittable offline

The raw string after incoming+ for this form is <full_path_slug>-<project_key>:

  • full_path_slug = Gitlab::Utils.slugify(full_path) → [a-z0-9-] (slugify turns every non-alphanumeric, including the / between group and project, into -), so the slug contains hyphens.
  • project_key = [a-z0-9_]+ → never contains a hyphen, so it is always the final hyphen-delimited segment.

Because the key is hyphen-free and the match is end-anchored, the existing pattern recovers slug and key deterministically with no DB:

/\A(?<slug>.+)-(?<key>[a-z0-9_]+)\z/

The greedy slug consumes up to the last hyphen; everything after it is the key. This split is also injective: two different (slug, key) pairs can only produce the same concatenated string if both the slug and the (hyphen-free) key are identical. So "#{full_path_slug}-#{project_key}" is a collision-free routable value, and a hyphen separator is as good as any other.

(The app currently calls the DB via Project.with_service_desk_key — that is for existence resolution, i.e. confirming such a project exists, not for disambiguating the split. mail_room does not need that for routing; it only needs the Topology claim to exist.)

Proposed approach

Claim the single concatenated string "#{full_path_slug}-#{project_key}" as one globally-unique claim. mail_room passes the raw address string straight to Classify with no splitting, and the Topology Service matches it verbatim — the same pattern already used for custom emails (claim the exact routable value).

The app computes the claim value from the project (slug) and the setting (project_key) at claim/verification time, where the DB is available.

High-level steps:

  1. Topology Service: add a new bucket/subject/source type for the service desk custom-key claim in proto/claims/v1/messages.proto and matching validation in validation.go. Update the Topology Service client gem in GitLab afterwards (scripts/update-topology-service-gem.sh).
  2. Make the model claimable. Include Cells::Claimable and declare a cells_claims_attribute whose value is "#{full_path_slug}-#{project_key}", gated by cells_unique_claims + a model-specific cells_claims_* flag, plus the verification-worker flag for backfilling. Use if: to skip rows without a project_key.
  3. Maintain the claim value on path changes. The claim value is derived from full_path (via the slug), so it changes when a project is renamed or moved to another namespace. The new column/claim value must be recomputed and re-issued whenever the path changes (hook into the existing route-rename / move machinery and the verification worker; confirm coverage during implementation). Decide handling for in-flight emails addressed to the old slug.
  4. Backfill existing records via Cells::Claims::VerificationService once claiming is live. Existing service_desk_settings rows with a project_key set must be claimed. Expect a backfill/post-deploy rollout, not an inline migration.
  5. Identify and route. Extend the gem's :service_desk matcher to emit a target carrying the raw <slug>-<project_key> string, and have the mail_room service resolve that target through Classify and forward to the owning cell. Unmatched/unknown keys continue to drop or fall back to the default cell per the routing policy.

Open questions

  • Is custom-project_key routing in scope for the Cells Service Desk MVC, or can these emails fall back to a default cell until the claim exists?
  • What is the behavior for in-flight emails addressed to the old slug right after a rename/move, before the claim is re-issued?
  • How many existing service_desk_settings rows have a project_key set (backfill sizing)?
  • The composite string should live in its own bucket type so it cannot collide with routes or custom email claims.

Out of scope

  • The gem extraction and handler refactor (done in #604167 (closed)).
  • Routing of all other incoming email formats, which are already handled.
Edited by Mario Celi