Add GitHub OAuth webhook management service

Carla Dragorequested to merge
630401-oauth-webhook-mgmt-service into master

What does this MR do and why?

Part of the GitHub OAuth path being evaluated as an alternative to the GitHub App approach for continuous sync. Continuous sync needs a webhook on the GitHub repository so deliveries reach GitLab.

  • Adds create_hook and delete_hook to Gitlab::GithubImport::Client, wrapped in with_rate_limit like the client's other single-request methods.
  • Adds Import::Sync::Github::Oauth::WebhookManagementService (EE) with enable / disable:
    • Inputs: the Import::Sync::Repository, an OAuth-authenticated client:, and the per-project callback_url:. Taking the client and URL as inputs keeps this MR independent of two open decisions: where the OAuth token is stored (under discussion in the issue), and the receiver route, which #630402 (closed) adds as POST /api/v4/projects/:id/github/webhooks.
    • Identifies the GitHub repository by its numeric id (provider_repository_xid), so it keeps working if the repository is renamed or transferred.
    • enable: no-op if a webhook is already recorded; generates a random secret (SecureRandom.hex(32)), creates the hook on GitHub subscribed to push and pull_request, then saves the hook id and secret. There's no DB transaction around the GitHub call. If saving fails, it deletes the just-created hook (logging if that delete also fails) and re-raises, so no orphaned webhook is left that GitLab can't verify. A GitHub 422 (hook already exists) is raised, not handled.
    • disable: deletes the hook on GitHub, then clears the local columns. If GitHub returns a 404, 401, or 403 (the hook was removed manually, or the token was revoked), it logs the failure and still clears the columns; the user removes any leftover hook on GitHub themselves. Other errors, like a GitHub 5xx or rate limit, are raised with the columns kept, so the caller can retry instead of losing track of a hook that probably still exists.
    • Both methods run inside a per-repository Redis lock (in_lock, retries: 0), so two concurrent calls can't create duplicate hooks. A concurrent call fails fast with FailedToObtainLockError. The record is reloaded inside the lock, so a stale copy can't act on outdated state. No DB transaction is held open during the GitHub call.
    • Never logs the secret or the token.
  • Storage: adds provider_webhook_xid (bigint, nullable, check constraint > 0 when present, matching the table's other provider_*_xid columns) and webhook_secret (jsonb, nullable, Rails encrypts, prevent_from_serialization, max length 510) to import_sync_repositories. This replaces the separate import_sync_github_oauth_webhooks table from the original issue description, per the issue author's suggestion.
  • No feature flag in this MR: nothing calls the service yet. The callers (#630402 (closed) and the sync enable flow) will be behind the flag.
  • Known dependency, out of scope: import_sync_repositories.provider_installation_xid is NOT NULL with a > 0 check (the table was built for the GitHub App path), so an OAuth-connected repository can't have a row yet. Raised in the issue; the service works on an existing row, and specs use the existing factory.

Database review

Migrations (regular, db/migrate, milestone 19.5):

  • 20260925090000_add_webhook_columns_to_import_sync_repositories.rb: two nullable add_columns.
  • 20260925090001_add_provider_webhook_xid_check_to_import_sync_repositories.rb: add_check_constraint (provider_webhook_xid IS NULL OR provider_webhook_xid > 0), disable_ddl_transaction!.

import_sync_repositories is a small table (table_size: small in db/docs), gitlab_main_org, not replicated by Siphon, so no ClickHouse change.

No new index: every lookup goes through the repository row (unique project_id).

Queries

The only new queries are two single-row UPDATEs by primary key, run when enable / disable save. Rails doesn't re-run the model's project_id uniqueness check on these saves because project_id doesn't change.

Enable:

UPDATE "import_sync_repositories" SET "updated_at" = now(), "provider_webhook_xid" = 42, "webhook_secret" = '{}'::jsonb WHERE "import_sync_repositories"."id" = <id>;

 Update on public.import_sync_repositories  (cost=0.15..3.17 rows=0 width=0) (actual time=0.030..0.031 rows=0 loops=1)
   Buffers: shared hit=5
   I/O Timings: read=0.000 write=0.000
   ->  Index Scan using import_sync_repositories_pkey on public.import_sync_repositories  (cost=0.15..3.17 rows=1 width=54) (actual time=0.029..0.029 rows=0 loops=1)
         Index Cond: (import_sync_repositories.id = 1)
         Buffers: shared hit=5
         I/O Timings: read=0.000 write=0.000
Settings: work_mem = '230MB', jit = 'off', random_page_cost = '1.5', seq_page_cost = '4', effective_cache_size = '472585MB'
Query ID: 9010157457090215228

Time: 0.536 ms  
  - planning: 0.443 ms  
  - execution: 0.093 ms  
    - I/O read: 0.000 ms  
    - I/O write: 0.000 ms  
  
Shared buffers:  
  - hits: 5 (~40.00 KiB) from the buffer pool  
  - reads: 0 from the OS file cache, including disk I/O  
  - dirtied: 0  
  - writes: 0

Query plan: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/58626/commands/163611

Disable:

UPDATE "import_sync_repositories" SET "updated_at" = now(), "provider_webhook_xid" = NULL, "webhook_secret" = NULL WHERE "import_sync_repositories"."id" = <id>;

Update on public.import_sync_repositories  (cost=0.15..3.17 rows=0 width=0) (actual time=0.026..0.027 rows=0 loops=1)
   Buffers: shared hit=5
   I/O Timings: read=0.000 write=0.000
   ->  Index Scan using import_sync_repositories_pkey on public.import_sync_repositories  (cost=0.15..3.17 rows=1 width=54) (actual time=0.025..0.025 rows=0 loops=1)
         Index Cond: (import_sync_repositories.id = 1)
         Buffers: shared hit=5
         I/O Timings: read=0.000 write=0.000
Settings: effective_cache_size = '472585MB', work_mem = '230MB', jit = 'off', random_page_cost = '1.5', seq_page_cost = '4'
Query ID: 256068181402178415

Time: 0.553 ms  
  - planning: 0.455 ms  
  - execution: 0.098 ms  
    - I/O read: 0.000 ms  
    - I/O write: 0.000 ms  
  
Shared buffers:  
  - hits: 5 (~40.00 KiB) from the buffer pool  
  - reads: 0 from the OS file cache, including disk I/O  
  - dirtied: 0  
  - writes: 0

Query plan: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/58626/commands/163614

Both use the primary key; webhook_secret holds the encrypted payload in production (shown here as '{}'::jsonb).

References

Screenshots or screen recordings

Not applicable. This MR is backend-only and adds no UI.

How to set up and validate locally

  1. Run migrations.
  2. In a Rails console, find or create an Import::Sync::Repository whose provider_repository_xid is the numeric id of a GitHub test repository you can administer.
  3. Build a client with a GitHub OAuth token that has the repo scope: client = Gitlab::GithubImport::Client.new('', parallel: true) (the service rejects non-parallel clients).
  4. service = Import::Sync::Github::Oauth::WebhookManagementService.new(repository, client: client, callback_url: 'https://example.com/webhook')
  5. service.enable, then check the hook appears under the repository's Settings > Webhooks on GitHub, and repository.reload.provider_webhook_xid is set.
  6. service.disable, then check the hook is gone and both columns are nil.
  7. Specs: bundle exec rspec spec/lib/gitlab/github_import/client_spec.rb ee/spec/models/import/sync/repository_spec.rb ee/spec/services/import/sync/github/oauth/webhook_management_service_spec.rb

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #630401 (closed)

Edited by Carla Drago

Merge request reports