Draft: POC - Import GitHub repositories with continuous sync

Config instructions

⚠️ Installation instructions can be found in https://gitlab.com/gitlab-org/gitlab/-/work_items/622060

Summary

This feature-flagged POC extends the existing GitHub importer with an optional Keep projects updated from GitHub after import mode.

Users can keep GitHub as the system of record while evaluating a read-only GitLab copy. When they are ready to complete the migration, Stop updates performs a fenced final check and restores ordinary GitLab-local editing.

Ongoing access uses short-lived, read-only GitHub App installation credentials and does not depend on the user who initiated the import.

Scope

Included:

• Existing GitHub importer for initial project creation and bootstrap.
• GitHub App user authorization, installation enrollment, exact missing-permission feedback, and short-lived installation credentials.
• GitHub-to-GitLab updates for:
  • Branches and tags.
  • Git LFS objects reachable from synchronized repository references.
  • Issues.
  • Pull requests represented as merge requests.
  • New and edited issue and merge request conversation comments.
• Signed webhook hints, immediate catch-up, manual Update now, and scheduled missed-event repair.
• Provider-derived repository and collaboration content remains read-only in GitLab while GitHub retains authority.
• Connection health, reconnect behavior, and guarded final Stop updates.
• A first-class GitHub updates settings surface with per-domain health.
• Exclusion of conflicting legacy pull and push mirrors while GitHub remains the provider of record.
• A deterministic GDK test provider for offline development.

Not included:

• GitLab-to-GitHub writes or bidirectional editing.
• GitHub Actions or CI continuity.
• Ongoing reviews, inline review comments, approvals, reactions, or deleted-comment propagation.
• A zero-RPO atomic snapshot across every GitHub domain.
• Production-grade multi-primary or failover fencing.

Contract references

• The feature is disabled by default through config/feature_flags/wip/github_continuous_import.yml.
• Updated-versus-imported-once coverage is owned by lib/gitlab/github_import/settings.rb.
• Ordinary imports retain the existing importer path when connected mode is absent or false.
• A connected project stays read-only until the final stop transition succeeds.
• GitHub updates use Continuity records rather than GitLab's legacy mirror storage. Both systems mutate the same Git repository, so they cannot run concurrently.

Change details

• Reuses the current GitHub repository picker, importer stages, representations, transformations, progress, cancellation, and re-import behavior.
• Adds durable connection, repository, synchronization, webhook-delivery, import grant, and external-object records.
• Uses same-App server-side enrollment when the current App authorization already proves installation and repository access.
• Falls back to the bound App setup flow when installation or repository access still requires user action.
• Names missing Metadata, Contents, Issues, Pull requests, or Administration access during setup and reconnect instead of requiring users to guess.
• Reconciles provider state into the existing project without a second provisioning or transformation stack, including durable work-item mappings and counter-cache refreshes after import and later updates.
• Fences importer, reconciliation, lifecycle, and stop transitions to prevent local writes from racing provider application.
• Disables and server-guards legacy mirror creation, reconfiguration, scheduling, and synchronization until bootstrap is canceled or Stop updates succeeds.

Validation evidence

• An initial 4,165-example backend sweep covered the 107 Ruby spec files then present, with six expected pending examples. Its one timing-dependent fixture failure was corrected and rerun successfully.
• Later hardening added 29 Ruby spec files and one frontend suite, with focused coverage for lifecycle cleanup, work-item reconciliation, reconnect permissions, repository settings, LFS continuation, and mirror exclusion.
• The latest mirror coverage passed 174 service and worker examples, 16 scheduler examples, six exact scheduling regressions, and two repository-settings feature examples. A follow-up initializer regression passed separately.
• The initial 11 frontend suites passed 262 tests. Subsequent reconnect and settings changes passed their focused frontend suites.
• The feature-flag definition suite passed, and RuboCop, ESLint, Prettier, HAML lint, gettext lint, Ruby syntax, and whitespace checks passed for their changed files.
• Live GDK GitHub authorization, App-scoped repository discovery, connected import, GitHub issue convergence, work-item counts, and the GitHub updates settings surface were verified.
• A complete live missed-webhook repair, reconnect, and final Stop updates walkthrough remains before this draft is marked ready.

Risks and follow-ups

• The PostgreSQL advisory barriers and renewable leases are appropriate for the bounded single-primary POC. Production hardening requires durable execution and retirement fencing across failover.
• The final stop check is fenced but is not one atomic provider snapshot; a GitHub change made during the final check might not be included.
• Full-fidelity pull-request collaboration, outbound commands, and CI continuity remain separate future work.
• The App must remain installed and authorized for the selected repositories.
• GitHub updates and legacy repository mirrors both mutate the same Git repository, so pull and push mirrors remain unavailable until bootstrap is canceled or Stop updates completes successfully.
• Repositories with more than 10,000 reachable Git LFS objects fail closed in this POC rather than running an unbounded reconciliation.
• The MR is intentionally large because it carries a complete vertical POC behind a disabled feature flag.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.