### Problem to solve When sync or verification fails, we want to group errors by type and surface metadata (severity, title, troubleshooting links) in the [Geo Troubleshooting Dashboard](https://gitlab.com/groups/gitlab-org/-/work_items/16555#note_2535504140). Currently, many Geo errors are raised as plain strings (e.g., `raise "No valid connection to primary registry"`), or are caught as generic `StandardError` and stored as free-text messages. This makes reliable grouping impossible. We need typed error classes that can carry dashboard metadata and serve as stable grouping keys when stored in the `sync_error_class` / `verification_error_class` registry columns (#548529). ### Proposal The approach separates two concerns: - `Geo::Errors::ErrorType` (Fixed Items Model): The metadata catalog with stable IDs, severity, title, description, and troubleshooting links - `Geo::Errors::*Error` (Exception classes): The raise/rescue hierarchy, where each class links to its `ErrorType` via `error_type_identifier` 1. New **`Geo::Errors::ErrorType` class** using the Fixed Item Model so the downstream API (#548541) can find metadata from a stored class name. 2. **New error classes** for error paths that currently use plain strings, have no error object, or catch external errors. 3. **Replace string raises** in Geo code with the new typed classes. 4. **Wrap external errors** that commonly cause sync/verification failures (e.g., `GRPC::Unavailable`, `Gitlab::Shell::Error`) in Geo error classes at the catch sites, so all stored error class names are `Geo::Errors::*` with full metadata. ### Implementation plan #### 1. Add new error classes for string-based error paths These cover error paths that currently construct messages from strings without an error object: | Class | Replaces | Severity | Location | |---|---|---|---| | `Geo::Errors::SyncTimeoutError` | `"Sync timed out after ..."` | `:warning` | `ReplicableRegistry.fail_sync_timeouts` | | `Geo::Errors::VerificationTimeoutError` | `"Verification timed out after ..."` | `:warning` | `VerificationState.fail_verification_timeouts` | | `Geo::Errors::ChecksumMismatchError` | `"Checksum does not match the primary checksum"` | `:critical` | `VerifiableRegistry.verification_failed_due_to_mismatch!` | | `Geo::Errors::PrimaryRegistryConnectionError` | `"No valid connection to primary registry"` | `:critical` | `ContainerRepositorySync#execute` | #### 2. Replace string raises in Geo code Search for `raise "<message>"` patterns in `ee/app/services/geo/` and `ee/app/models/concerns/geo/` and replace with the appropriate error class. Known candidates: - `ee/app/services/geo/container_repository_sync.rb`: `raise "No valid connection to primary registry"` -> `raise Geo::Errors::PrimaryRegistryConnectionError` Note: `raise NotImplementedError` and `raise ArgumentError` in abstract base classes and replicator plumbing should **not** be replaced. These are programming errors, not sync/verification failures. #### 3. Wrap external errors at catch sites External errors like `GRPC::Unavailable` or `Gitlab::Shell::Error` commonly cause sync/verification failures. Rather than storing third-party class names (which can't carry metadata), wrap them in Geo error classes at the existing `rescue` sites. For example: | Geo wrapper class | Wraps | Severity | Catch site | |---|---|---|---| | `Geo::Errors::GitalyConnectionError` | `GRPC::Unavailable`, `GRPC::DeadlineExceeded` | `:critical` | `FrameworkRepositorySyncService#sync_repository` | | `Geo::Errors::RepositoryNotFoundError` | `Gitlab::Git::Repository::NoRepository` | `:warning` | `FrameworkRepositorySyncService#sync_repository` | | `Geo::Errors::GitError` | `Gitlab::Shell::Error`, `Gitlab::Git::BaseError` | `:warning` | `FrameworkRepositorySyncService#sync_repository` | | `Geo::Errors::BlobDownloadError` | `StandardError` in blob download | `:warning` | `BlobDownloadService#execute` | Each wrapper preserves the original error as `cause` (via `raise Geo::Errors::GitalyConnectionError.new(...), cause: e` or Ruby's implicit chaining) so the original message is still available for logging and the `last_sync_failure` text column. #### 4. Create the `Geo::Errors::ErrorType` Fixed Items Model Add class-level methods with sensible defaults that subclasses override: ```ruby # ee/app/models/geo/errors/error_type.rb module Geo module Errors class ErrorType include ActiveRecord::FixedItemsModel::Model ITEMS = [ { id: 1, name: 'unknown', severity: :warning, title: 'Unknown error', description: 'An unclassified error occurred', troubleshooting_links: [] }, { id: 2, name: 'sync_timeout', severity: :warning, title: 'Sync timeout', description: 'Sync timed out', troubleshooting_links: [] }, { id: 3, name: 'verification_timeout', severity: :warning, title: 'Verification timeout', description: 'Verification timed out', troubleshooting_links: [] }, { id: 4, name: 'checksum_mismatch', severity: :critical, title: 'Checksum mismatch', description: 'Checksum does not match the primary', troubleshooting_links: [] }, { id: 5, name: 'primary_registry_connection', severity: :critical, title: 'Primary registry connection error', description: 'No valid connection to primary registry', troubleshooting_links: [] }, { id: 6, name: 'replicable_does_not_exist', severity: :warning, title: 'Replicable does not exist', description: 'File does not exist on disk', troubleshooting_links: [] }, { id: 7, name: 'replicable_excluded_from_verification', severity: :info, title: 'Excluded from verification', description: 'Replicable is excluded from verification', troubleshooting_links: [] }, { id: 8, name: 'gitaly_connection', severity: :critical, title: 'Gitaly connection error', description: 'Gitaly is unavailable or timed out', troubleshooting_links: [] }, { id: 9, name: 'repository_not_found', severity: :warning, title: 'Repository not found', description: 'Git repository does not exist', troubleshooting_links: [] }, { id: 10, name: 'git_error', severity: :warning, title: 'Git error', description: 'A Git or shell error occurred', troubleshooting_links: [] }, { id: 11, name: 'blob_download', severity: :warning, title: 'Blob download error', description: 'Failed to download blob from primary', troubleshooting_links: [] } ].freeze attribute :name, :string attribute :severity, :string # :critical, :warning, :info attribute :title, :string attribute :description, :string attribute :troubleshooting_links # array of { title:, url: } hashes end end end ``` Fill up this array with severity and troubleshooting values that make sense. #### 5. Update `Geo::Errors::BaseError` to link to `ErrorType` Each exception class declares its `error_type_identifier`, linking it to the metadata catalog: ```ruby # ee/app/models/geo/errors.rb module Geo module Errors class BaseError < StandardError # Each subclass overrides this to return its ErrorType id def self.error_type_identifier 1 # default: unknown end def self.error_type ErrorType.find(error_type_identifier) end end end end ``` This cleanly separates the exception hierarchy (for raise/rescue) from the metadata catalog (for the dashboard). The metadata lives in one place (`ErrorType` ITEMS) and is version-controlled with stable IDs. Update the existing subclasses (`ReplicableDoesNotExistError`, `ReplicableExcludedFromVerificationError`, `UnknownSelectiveSyncType`, `StatusTimeoutError`) to override the `error_type_identifier` with the correct ID. This ensures every stored `sync_error_class` / `verification_error_class` value is a `Geo::Errors::*` class with full dashboard metadata, and the API (#548541) can resolve metadata with a simple `error_class_name.constantize.error_type`. #### 6. Update specs * Add `ee/spec/models/geo/errors/error_type_spec.rb` to test the Fixed Items Model * Update `ee/spec/models/geo/errors_spec.rb` to cover `error_type_identifier` and the linkage * Update any specs that assert on the old string messages (e.g., `blob_replicator_strategy_shared_examples.rb`)