Skip to content
Snippets Groups Projects
  1. Feb 24, 2025
  2. Feb 21, 2025
  3. Feb 20, 2025
  4. Feb 19, 2025
    • 🤖 GitLab Bot 🤖's avatar
    • Karthik Nayak's avatar
      migration: Pass metadata from the request's context · 1d47a36e
      Karthik Nayak authored
      The migration manager uses two context:
      1. request context: This is the context provided by the incoming gRPC
      request which initiated the migration manager. We use this request to
      check if a migration is enabled/disabled.
      2. manager's internal context: This context is used to propagate the
      partition manager status within the manager. This is the context which
      is passed to individual migrations.
      
      Since the manager's internal context doesn't have access to the
      featureflags, this is extended to the migrations. A repercussion of
      which is that the latest Git version which is behind a feature flag is
      not utilized by the migrations since we don't have the required
      information in the context.
      
      Let's copy over metadata from the request's context to the manager's
      context to solve this issues.
      Unverified
      1d47a36e
    • Karthik Nayak's avatar
      migration: Close the dry-run manager when the partition is closed · 912bf59d
      Karthik Nayak authored
      When the partition's `Close()` method is called, we should also close
      the dry-run migration manager. Since `combinedMigrationPartition` has an
      embedded `storage.Partition` field, any `Close()` operation would
      directly be called on that field. We should add an override to ensure
      that the dry-run's `Close()` is also called. This would ensure that the
      context for the dry-run manager is cancelled and any ongoing operations
      are stopped as necessary.
      Unverified
      912bf59d
    • Sami Hiltunen's avatar
      Merge branch '6607-relative-path' into 'master' · 73eac5af
      Sami Hiltunen authored
      recovery: Run recovery commands with relative path
      
      Closes #6607
      
      See merge request !7621
      
      
      
      Merged-by: Sami Hiltunen's avatarSami Hiltunen <shiltunen@gitlab.com>
      Approved-by: default avatarJames Fargher <jfargher@gitlab.com>
      Approved-by: Sami Hiltunen's avatarSami Hiltunen <shiltunen@gitlab.com>
      Reviewed-by: default avatarJames Fargher <jfargher@gitlab.com>
      Co-authored-by: default avatarMustafa Bayar <mbayar@gitlab.com>
      Verified
      73eac5af
    • Quang-Minh Nguyen's avatar
    • GitLab Release Tools Bot's avatar
      Update changelog for 17.9.0 · c39d57f8
      GitLab Release Tools Bot authored
      [ci skip]
      Verified
      c39d57f8
    • Quang-Minh Nguyen's avatar
      raft: Implement Raft storage component · c53a7d59
      Quang-Minh Nguyen authored
      A Raft-based cluster requires persistent storage to maintain consensus
      data across restarts. This commit implements the Raft storage component
      that integrates with Gitaly's write-ahead while providing additional
      Raft-specific data persistence.
      
      The storage implementation serves several purposes:
      - Implements etcd/raft's Storage interface for log entry access
      - Adds persistence of Raft states (hard state, config state)
      - Tracks and acknowledges committed entries and snapshots
      - Manages log entry metadata in both WAL and KV store
      
      Key design considerations:
      - Reuses existing WAL infrastructure (log.Manager) for local log entry
        persistence.
      - Uses KV store for Raft-specific states. A new internal LSN
        (committedLSN) was introduced. This index tracks the latest acked LSN
        by the Raft group. It is used to delete persisted but not acked log
        entries after a restart. This pruning is important because we don't
        the primary node to apply non-acked entries.
      - Extends log entry format to include Raft metadata (term, index)
      
      Directory structure for log entries:
      Without Raft:             With Raft:
      └── MANIFEST             └── MANIFEST
      └── 1                    └── RAFT
      └── 2                    └── 1
      └── ...                  └── 2
                              └── ...
      c53a7d59
    • Quang-Minh Nguyen's avatar
      raft: Add doc for Gitaly multi-raft architecture · d1cc9f2c
      Quang-Minh Nguyen authored
      Most important components of raftmgr will be added in some later
      commits. This commit adds a doc for Gitaly multi-raft architecture to
      `doc/raft`. This doc will guide the implementation of those components.
      d1cc9f2c
    • Quang-Minh Nguyen's avatar
      storage: Add DeleteLogEntriesFrom for trailing log removal · edec4a8f
      Quang-Minh Nguyen authored
      In Raft, when log entries with a higher term need to replace previously
      appended entries, it is necessary to efficiently remove the tail of the
      WAL. This commit introduces a new method, DeleteLogEntriesFrom, that
      deletes all log entries starting from a given LSN (inclusive) to the end
      of the log.
      
      The method ensures:
      - No deletion is performed if a consumer is present, preventing
        premature notifications to an active consumer.
      - Deletion is refused if the requested starting LSN is below the low
        water mark, thereby avoiding removal of entries still in use.
      - Multiple trailing entries are removed efficiently by renaming them to
        a "to_delete" folder, with only one fsync after the batch operation.
      - Proper concurrency is maintained by acquiring both the main and
        pruning mutexes, ensuring that no conflicting operations occur during
        deletion.
      edec4a8f
  5. Feb 18, 2025
  6. Feb 17, 2025
  7. Feb 14, 2025
Loading