[FF] instance_ssh_certificates -- DB-backed instance-level SSH certificate authorities

Summary

Roll out the instance_ssh_certificates feature flag, which gates DB-backed instance-level SSH certificate authority (CA) support.

GitLab already supports SSH certificate authentication, but instance-level CAs are configured on the filesystem (config.yml / gitlab.rb). GitLab Dedicated admins have no filesystem access, so they cannot configure instance CAs at all. This work makes instance CAs DB-backed and manageable through an Admin API and UI.

  • Feature work item: #611272 (closed)
  • Parent epic: &22743
  • DRI: unassigned — see #g_create_source_code
  • Team Slack channel: #g_create_source_code

Flag type: beta, default_enabled: false, instance-level actor. Beta was chosen so the flag is documented and self-managed admins can opt in — this is a self-managed/Dedicated feature, not a GitLab.com feature.

Work item #611272 (closed) ships only the foundation: the instance_ssh_certificates table, the InstanceSshCertificate model, this flag, and the availability guard. No user-facing behaviour ships behind the flag yet. Later issues add /authorized_certs resolution (#611274 (closed)), the Admin REST API (#611314), the Admin UI, and audit events.

Important

This is not a GitLab.com percentage rollout.

The availability guard is:

def self.available?
  !Gitlab.com? && Feature.enabled?(:instance_ssh_certificates, :instance)
end

!Gitlab.com? makes the feature permanently unavailable on GitLab.com regardless of flag state. That is a product requirement: there is no instance Admin Area on .com, and .com already has group-level CAs for Enterprise Users. So --actors percentage rollouts on .com do not apply here, and neither does --project / --group / --user targeting — the flag is checked with the :instance actor.

The audience is self-managed and GitLab Dedicated. Self-managed admins opt in themselves. GitLab Dedicated tenants cannot flip feature flags — no gitlab.rb access, no chatops. So the flag must be defaulted on (and then removed) before the feature counts as shipped for Dedicated. That flip is the main gate this issue tracks.

Feature flag controls · Feature flag lifecycle

What could go wrong?

  • Blast radius: this table is a chain-of-trust root. Any CA it trusts can mint certificates that authenticate to the entire instance. A bad or attacker-controlled row is an instance-wide authentication bypass.
  • FIPS. On FIPS-enabled instances (FedRAMP), a CA public key using a non-approved algorithm or an insufficient key size is rejected at insert time by SshKeyValidator. Model specs cover this. It needs verifying on a real FIPS-enabled instance before default-on.
  • Audit events are not implemented yet (deferred to the Admin API issue, #611314). Adding or removing a CA is currently unaudited. Do not default this on for Dedicated until it is audited.
  • Guard omission. Every future caller — API endpoints, GraphQL, UI — must check available?. A surface that skips it could expose CA management where it shouldn't exist, including on .com.
  • No data-loss risk while there is no write path. The foundation adds a table and a model only.

Non-production validation

Instance-level flag, so no --actors and no project/group/user targeting.

/chatops gitlab run feature set instance_ssh_certificates true --dev --pre --staging --staging-ref

Self-managed admins (and local GDK) enable it from a Rails console:

Feature.enable(:instance_ssh_certificates)
  • Enabled on --dev --pre --staging --staging-ref
  • Verified on a self-managed-like instance (not .com) that InstanceSshCertificate.available? returns true
  • Verified available? returns false on a .com-like instance even with the flag on

Before defaulting on

  • #611274 (closed) merged — /authorized_certs resolves DB-backed instance CAs
  • #611314 merged — Admin REST API for adding/removing CAs
  • Admin UI merged
  • Audit events implemented for CA add and remove
  • Every surface (REST, GraphQL, UI) honours InstanceSshCertificate.available?
  • FIPS key validation verified on a FIPS-enabled instance
  • Docs + version history written, including that the feature is unavailable on GitLab.com

Default on, then remove

This is the finish line for Dedicated. Dedicated tenants cannot flip the flag, so they only get the feature once the flag defaults on — and properly, once it is gone.

  • Flip default_enabled: true in config/feature_flags/beta/instance_ssh_certificates.yml
  • Ship one release with the flag defaulted on
  • Remove all Feature.enabled?(:instance_ssh_certificates, :instance) references, leaving the !Gitlab.com? guard in place
  • Delete config/feature_flags/beta/instance_ssh_certificates.yml
  • Update docs to drop the flag caveat
  • Run the cleanup chatops:
/chatops gitlab run release check <merge-request-url> <milestone>
/chatops gitlab run feature delete instance_ssh_certificates --dev --pre --staging --staging-ref --production

Rollback

/chatops gitlab run feature set instance_ssh_certificates false --dev --pre --staging --staging-ref
/chatops gitlab run feature delete instance_ssh_certificates --dev --pre --staging --staging-ref --production

Self-managed rollback is Feature.disable(:instance_ssh_certificates) in a Rails console.