Send SSH certificate provenance in Git audit events

What does this MR do and why?

Addresses "Gap 1" in https://gitlab.com/gitlab-org/gitlab/-/work_items/611349.

Rails half MR: gitlab!259751 (merged)

When a Git operation over SSH is authenticated with an SSH certificate through gitlab-sshd, the Git audit event sent to Rails now includes the certificate's provenance:

JSON field	Value
ca_fingerprint	ssh.FingerprintSHA256(cert.SignatureKey): the signing CA's fingerprint, with the SHA256: prefix
certificate_identity	cert.KeyId, exactly as presented (can differ from the resolved username, for example an email address for group certificates)
certificate_trust_source	file, group, or instance: the authentication branch that accepted the certificate

For any other kind of authentication, the three keys are left out of the request (omitempty).

Related to https://gitlab.com/gitlab-org/gitlab/-/work_items/611349. The Rails receiver accepts these as optional parameters on POST /api/v4/internal/shellhorse/git_audit_event. Older Rails versions ignore unknown parameters, so this is safe to deploy first.

How it works

• grantCertificate records the CA fingerprint, raw KeyId, and trust source in the authenticated connection's Permissions.Extensions. The trust source is passed in explicitly, because the existing certificate_scope log field can't tell a local CA apart from an API instance match.
• sshd.go reads those extensions into the session as *commandargs.CertificateMetadata, which reaches the command through NewWithUsername as commandargs.Shell.Certificate.
• The metadata deliberately does not travel in sshenv.Env, so it stays out of session log lines.
• gitauditevent.Client.Audit copies it into the request. AuditParams and the upload-pack and receive-pack call sites are unchanged.

Authentication, actor resolution, namespace restrictions, NeedAudit gating, and non-fatal audit failures are unchanged. No certificate or key material is sent, and no extra API calls are made.

How to set up and validate locally

1. Authenticate to gitlab-sshd with a CA-signed SSH certificate.
2. Clone a repository.
3. Verify that the streamed repository_git_operation event contains the certificate metadata.

Prerequisites

• A working GDK running GitLab EE with an Ultimate development license.
• GDK's gitlab-sshd service enabled, listening on 127.0.0.1:2222.
• An existing, non-empty private project that the root user can clone.
• Clean working trees in GDK's active gitlab/ and gitlab-shell/ checkouts.
• Python 3 for a small local HTTP receiver.

Replace group/project below with the test project's full path. Adjust the SSH host and port if your GDK uses different settings.

1. Check out and build both MRs

Run the following from your GDK root directory. Keep this terminal open for subsequent shell commands:

export GDK_ROOT="$PWD"
export CERT_DIR="$GDK_ROOT/tmp/cert-audit"
export PROJECT_PATH="group/project"

cd "$GDK_ROOT/gitlab"
git fetch https://gitlab.com/gitlab-org/gitlab.git merge-requests/259751/head
git switch --detach FETCH_HEAD
bundle install

cd "$GDK_ROOT/gitlab-shell"
git fetch https://gitlab.com/gitlab-org/gitlab-shell.git merge-requests/1580/head
git switch --detach FETCH_HEAD
make compile

cd "$GDK_ROOT"
gdk restart rails-web sidekiq sshd

These must be the checkouts used by GDK's services. Building a separate worktree alone does not update the running services.

make compile builds both GitLab Shell and gitlab-sshd. The Rails MR changes Ruby code, so installing dependencies and restarting Rails and Sidekiq loads the changes. This MR requires no frontend build or database migration.

2. Enable Git audit events for the project

Open a Rails console:

cd "$GDK_ROOT"
gdk rails console

Run the following, replacing group/project:

project = Project.find_by_full_path('group/project')
user = User.find_by_username('root')
raise 'Project or user not found' unless project && user

puts "Previous flag state: #{Feature.enabled?(:log_git_streaming_audit_events, project)}"
Feature.enable(:log_git_streaming_audit_events, project)

settings = Gitlab::CurrentSettings
previous_allowlist = settings.outbound_local_requests_whitelist
puts "Previous outbound allowlist: #{previous_allowlist.inspect}"

settings.update!(
  outbound_local_requests_whitelist: (previous_allowlist + ['127.0.0.1']).uniq
)

puts "Expected author_id: #{user.id}; project ID: #{project.id}"

Record the previous settings for cleanup, then exit the console.

3. Start an audit-event receiver

In a separate terminal, run:

python3 -u - <<'PY'
import json
from http.server import BaseHTTPRequestHandler, HTTPServer

class Receiver(BaseHTTPRequestHandler):
    def do_POST(self):
        body = self.rfile.read(int(self.headers["Content-Length"]))
        payload = json.loads(body)
        events = payload if isinstance(payload, list) else [payload]

        for event in events:
            if event.get("event_type") == "repository_git_operation":
                print(json.dumps(event, indent=2), flush=True)

        self.send_response(200)
        self.end_headers()

print("Listening on http://127.0.0.1:9898", flush=True)
HTTPServer(("127.0.0.1", 9898), Receiver).serve_forever()
PY

In GitLab:

1. Go to Admin > Monitoring > Audit events > Streams.
2. Select Add streaming destination > HTTP endpoint.
3. Set the name to Certificate audit smoke test.
4. Set the destination URL to http://127.0.0.1:9898.
5. Ensure the destination is active and its event filters allow repository_git_operation. Leaving event filters unset also allows this event.

Leave the receiver running.

4. Create and trust an SSH certificate

In the original terminal, generate a fresh CA and client key:

mkdir -p "$CERT_DIR"

ssh-keygen -t ed25519 -N '' -f "$CERT_DIR/ca"
ssh-keygen -t ed25519 -N '' -f "$CERT_DIR/client"

ssh-keygen \
  -s "$CERT_DIR/ca" \
  -I root \
  -n git \
  -V '-5m:+1h' \
  "$CERT_DIR/client.pub"

ssh-keygen -lf "$CERT_DIR/ca.pub" -E sha256

Record the CA fingerprint from the last command, including its SHA256: prefix.

Print the absolute CA public-key path:

echo "$CERT_DIR/ca.pub"

In $GDK_ROOT/gitlab-shell/config.yml, add trusted_user_ca_keys beneath the existing sshd: section. Use the exact absolute path printed above:

sshd:
  # Retain the existing SSH settings.
  trusted_user_ca_keys:
    - /absolute/path/to/gdk/tmp/cert-audit/ca.pub

If trusted_user_ca_keys already exists, append the test CA to its list.

Verify the file exists, restart SSH, and confirm that it is listening:

ls -l "$CERT_DIR/ca.pub"

cd "$GDK_ROOT"
gdk restart sshd
gdk status sshd
lsof -nP -iTCP:2222 -sTCP:LISTEN

Expect sshd to be running and gitlab-sshd to be listening on port 2222.

This exercises the file trust source and does not require FF_GITLAB_SHELL_SSH_CERTIFICATES.

5. Clone using the certificate

Do not register the fresh client public key as a GitLab SSH key.

GIT_SSH_COMMAND="ssh -F /dev/null \
  -i '$CERT_DIR/client' \
  -o CertificateFile='$CERT_DIR/client-cert.pub' \
  -o IdentitiesOnly=yes \
  -o IdentityAgent=none \
  -o BatchMode=yes \
  -o StrictHostKeyChecking=accept-new" \
git clone \
  "ssh://git@127.0.0.1:2222/$PROJECT_PATH.git" \
  "$CERT_DIR/checkout"

The clone should succeed. This command uses the test certificate and avoids reusing keys from your SSH agent.

6. Verify the event

Allow Sidekiq a few seconds to deliver the event to the receiver.

Confirm that at least one received event has:

• event_type: repository_git_operation.
• The expected author and project IDs.
• The following values under details.custom_message:

{
  "protocol": "ssh",
  "action": "git-upload-pack",
  "verb": "clone",
  "ca_fingerprint": "SHA256:<the CA fingerprint recorded above>",
  "certificate_identity": "root",
  "certificate_trust_source": "file"
}

Additional fields are expected. The CA fingerprint must match exactly, including the SHA256: prefix.

• A successful certificate-authenticated clone produces a streamed Git audit event containing all three correct certificate-provenance fields.

Cleanup

• Remove the test CA entry from gitlab-shell/config.yml and restart sshd.
• Remove the temporary streaming destination.
• Restore the previous outbound allowlist and project feature-flag configuration.
• Stop the HTTP receiver.
• Delete the temporary certificate directory and clone.
• Return both repositories to their previous branches when finished.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist.