- Feb 24, 2025
-
-
Olivier Campeau authored
Bundle-URI: Auto-generation of bundle See merge request gitlab-org/gitaly!7588 Merged-by:
Olivier Campeau <ocampeau@gitlab.com> Approved-by:
Mustafa Bayar <mbayar@gitlab.com> Reviewed-by:
Sami Hiltunen <shiltunen@gitlab.com> Reviewed-by:
Mustafa Bayar <mbayar@gitlab.com> Reviewed-by:
Olivier Campeau <ocampeau@gitlab.com>
-
Olivier Campeau authored
This commit removes the ProgressTracker files. The ProgressTracker is no longer used. It was removed by a previous commit.
-
Olivier Campeau authored
A previous commit added a Strategy interface to handle the logic that determines whether a bundle should be generated or not. Although that previous commit did introduce that interface, it wasn't used. This commit updates the bundleURI manager to use this interface, using the OccurrencesStrategy implementation. It also updates the various tests that needed to be changed.
-
Olivier Campeau authored
When generating a bundle, the manager increment a progress tracker and compares the number of `in progress` requests with a given threshold. If the threshold is reached, a bundle is generated. That bundle is generated into it's own goroutine. Once the comparison is done, the progress tracker is decremented. Those successive operations are happening one after another: 1. Increment counter 2. Compare 3. Decrement The fast pace at which those operations are executed makes it very unlikely that a bundle will be generated. The real issue is that the decrement of the counter happens right after the comparison. In such an implementation, a repository would need to be cloned `threshold` number of times in a matter of milliseconds or less in order for the counter to reach the threshold. This commit introduces a new `Strategy` interface that could be used by the manager to delegate the decision whether a bundle should be generated or not. It also introduces an implementation of that `Strategy` interface that tracks the number of occurrences for which a clone was requested for a repository. If the number of occurrences reaches a given threshold within an given interval of time, then a bundle is generated. It also keeps track of the time a bundle was generated for a repository, to avoid creating a bundle that was generated not so long ago. Finally, it limits the number of concurrent bundle generation that can occur at the same time.
-
Divya Rani authored
tools/golangci-lint: Update module golang.org/x/tools to v0.29.0 See merge request !7545 Merged-by:
Divya Rani <drani@gitlab.com> Approved-by:
John Cai <jcai@gitlab.com> Co-authored-by:
GitLab Renovate Bot <gitlab-bot@gitlab.com>
-
Quang-Minh Nguyen authored
Fix typo in RawPatchRequest documentation See merge request !7636 Merged-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com> Approved-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com> Co-authored-by:
Vasilii Iakliushin <viakliushin@gitlab.com>
-
James Liu authored
tools/goimports: Update module golang.org/x/tools to v0.30.0 See merge request !7557 Merged-by:
James Liu <jliu@gitlab.com> Approved-by:
Divya Rani <drani@gitlab.com> Co-authored-by:
GitLab Renovate Bot <gitlab-bot@gitlab.com>
-
- Feb 21, 2025
-
-
GitLab Release Tools Bot authored
[ci skip]
-
GitLab Release Tools Bot authored
[ci skip]
-
Divya Rani authored
config: Ensure pack-objects cache doesn't point to storage root See merge request !7631 Merged-by:
Divya Rani <drani@gitlab.com> Approved-by:
Emily Chui <echui@gitlab.com> Approved-by:
Divya Rani <drani@gitlab.com> Co-authored-by:
James Liu <jliu@gitlab.com>
-
Vasilii Iakliushin authored
-
Quang-Minh Nguyen authored
Makefile: Fix order of bundled Git rules with Make v3.81 and older See merge request !7626 Merged-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com> Approved-by:
Karthik Nayak <knayak@gitlab.com> Approved-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com> Co-authored-by:
Patrick Steinhardt <psteinhardt@gitlab.com>
-
Divya Rani authored
raft: Skip transport tests if Praefect is enabled See merge request !7635 Merged-by:
Divya Rani <drani@gitlab.com> Approved-by:
Divya Rani <drani@gitlab.com> Co-authored-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com>
-
Quang-Minh Nguyen authored
-
-
James Liu authored
test: Fix git-repack(1) loose object assertion See merge request !7633 Merged-by:
James Liu <jliu@gitlab.com> Approved-by:
James Fargher <jfargher@gitlab.com>
-
James Liu authored
We have had a case recently [1] where the pack_objects_cache.dir value was set to the same path as the Gitaly storage path. The pack-objects cache is a filesystem cache and is evicted after a customisable age. This misconfiguration caused repositories to be deleted from the filesystem when the eviction was triggered. Validate that the absolute path of pack_objects_cache.dir doesn't match the absolute path of any configured storage directories, or any of its parent directories. [1] #6611
-
- Feb 20, 2025
-
-
John Cai authored
go: Update module github.com/go-enry/go-enry/v2 to v2.9.2 See merge request !7625 Merged-by:
John Cai <jcai@gitlab.com> Approved-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com> Approved-by:
John Cai <jcai@gitlab.com> Co-authored-by:
GitLab Renovate Bot <gitlab-bot@gitlab.com>
-
Sami Hiltunen authored
migration: Pass metadata from the request's context See merge request !7629 Merged-by:
Sami Hiltunen <shiltunen@gitlab.com> Approved-by:
Sami Hiltunen <shiltunen@gitlab.com> Approved-by:
Divya Rani <drani@gitlab.com> Reviewed-by:
Sami Hiltunen <shiltunen@gitlab.com> Reviewed-by:
Karthik Nayak <knayak@gitlab.com> Co-authored-by:
Karthik Nayak <knayak@gitlab.com>
-
Emily Chui authored
go: Update module golang.org/x/crypto to v0.33.0 See merge request !7619 Merged-by:
Emily Chui <echui@gitlab.com> Approved-by:
Emily Chui <echui@gitlab.com> Co-authored-by:
GitLab Renovate Bot <gitlab-bot@gitlab.com>
-
- Feb 19, 2025
-
-
🤖 GitLab Bot 🤖 authored
-
Karthik Nayak authored
The migration manager uses two context: 1. request context: This is the context provided by the incoming gRPC request which initiated the migration manager. We use this request to check if a migration is enabled/disabled. 2. manager's internal context: This context is used to propagate the partition manager status within the manager. This is the context which is passed to individual migrations. Since the manager's internal context doesn't have access to the featureflags, this is extended to the migrations. A repercussion of which is that the latest Git version which is behind a feature flag is not utilized by the migrations since we don't have the required information in the context. Let's copy over metadata from the request's context to the manager's context to solve this issues.
-
Karthik Nayak authored
When the partition's `Close()` method is called, we should also close the dry-run migration manager. Since `combinedMigrationPartition` has an embedded `storage.Partition` field, any `Close()` operation would directly be called on that field. We should add an override to ensure that the dry-run's `Close()` is also called. This would ensure that the context for the dry-run manager is cancelled and any ongoing operations are stopped as necessary.
-
Sami Hiltunen authored
recovery: Run recovery commands with relative path Closes #6607 See merge request !7621 Merged-by:
Sami Hiltunen <shiltunen@gitlab.com> Approved-by:
James Fargher <jfargher@gitlab.com> Approved-by:
Sami Hiltunen <shiltunen@gitlab.com> Reviewed-by:
James Fargher <jfargher@gitlab.com> Co-authored-by:
Mustafa Bayar <mbayar@gitlab.com>
-
Quang-Minh Nguyen authored
Implement Raft storage component See merge request !7584 Merged-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com> Approved-by:
Emily Chui <echui@gitlab.com> Reviewed-by:
Sami Hiltunen <shiltunen@gitlab.com> Reviewed-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com> Reviewed-by:
James Liu <jliu@gitlab.com> Reviewed-by:
Emily Chui <echui@gitlab.com> Reviewed-by:
Divya Rani <drani@gitlab.com>
-
GitLab Release Tools Bot authored
[ci skip]
-
Quang-Minh Nguyen authored
A Raft-based cluster requires persistent storage to maintain consensus data across restarts. This commit implements the Raft storage component that integrates with Gitaly's write-ahead while providing additional Raft-specific data persistence. The storage implementation serves several purposes: - Implements etcd/raft's Storage interface for log entry access - Adds persistence of Raft states (hard state, config state) - Tracks and acknowledges committed entries and snapshots - Manages log entry metadata in both WAL and KV store Key design considerations: - Reuses existing WAL infrastructure (log.Manager) for local log entry persistence. - Uses KV store for Raft-specific states. A new internal LSN (committedLSN) was introduced. This index tracks the latest acked LSN by the Raft group. It is used to delete persisted but not acked log entries after a restart. This pruning is important because we don't the primary node to apply non-acked entries. - Extends log entry format to include Raft metadata (term, index) Directory structure for log entries: Without Raft: With Raft: └── MANIFEST └── MANIFEST └── 1 └── RAFT └── 2 └── 1 └── ... └── 2 └── ...
-
Quang-Minh Nguyen authored
Most important components of raftmgr will be added in some later commits. This commit adds a doc for Gitaly multi-raft architecture to `doc/raft`. This doc will guide the implementation of those components.
-
Quang-Minh Nguyen authored
In Raft, when log entries with a higher term need to replace previously appended entries, it is necessary to efficiently remove the tail of the WAL. This commit introduces a new method, DeleteLogEntriesFrom, that deletes all log entries starting from a given LSN (inclusive) to the end of the log. The method ensures: - No deletion is performed if a consumer is present, preventing premature notifications to an active consumer. - Deletion is refused if the requested starting LSN is below the low water mark, thereby avoiding removal of entries still in use. - Multiple trailing entries are removed efficiently by renaming them to a "to_delete" folder, with only one fsync after the batch operation. - Proper concurrency is maintained by acquiring both the main and pruning mutexes, ensuring that no conflicting operations occur during deletion.
-
- Feb 18, 2025
-
-
Olivier Campeau authored
Goroutine leaks: buffering a channel to avoid blocking See merge request !7623 Merged-by:
Olivier Campeau <ocampeau@gitlab.com> Approved-by:
Sami Hiltunen <shiltunen@gitlab.com> Approved-by:
Mustafa Bayar <mbayar@gitlab.com>
-
Olivier Campeau authored
When the rate limiter returns an error but the request context was already cancelled, the `errs` channel has no listener, which blocks the goroutine that want to write to it. By buffering the channel with 1 slot, we can be sure the error will be written to the channel, which will then be garbage collected. Relevant resources: * #6618
-
James Liu authored
Revert "Merge branch 'freinink/476241-default-committer-info-with-gpg-signing' into 'master'" See merge request !7628 Merged-by:
James Liu <jliu@gitlab.com> Approved-by:
James Liu <jliu@gitlab.com> Co-authored-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com>
-
Quang-Minh Nguyen authored
This reverts merge request !7620
-
- Feb 17, 2025
-
-
🤖 GitLab Bot 🤖 authored
-
- Feb 14, 2025
-
-
Mustafa Bayar authored
tools/protoc-gen-go: Update module google.golang.org/protobuf to v1.36.5 See merge request !7578 Merged-by:
Mustafa Bayar <mbayar@gitlab.com> Approved-by:
Mustafa Bayar <mbayar@gitlab.com> Co-authored-by:
GitLab Renovate Bot <gitlab-bot@gitlab.com>
-
Quang-Minh Nguyen authored
Provide a default Git Committer Name and Email when GPGSigning is enabled See merge request !7620 Merged-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com> Approved-by:
John Cai <jcai@gitlab.com> Approved-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com> Co-authored-by:
Fred Reinink <freinink@gitlab.com>
-
Patrick Steinhardt authored
Introduce both Meson and Ninja as tool versions so that asdf knows to pick the correct version.
-
Patrick Steinhardt authored
In the GNU Make commit f5891a26 (Implement the shortest stem first search order for pattern-specific variables and pattern rules., 2009-09-28), the way how pattern rules are being matched has changed in a backwards-compatible way: The pattern-specific variables and pattern rules are now applied in the shortest stem first order instead of the definition order (variables and rules with the same stem length are still applied in the definition order). This produces the usually-desired behavior where more specific patterns are preferred. To detect this feature search for 'shortest-stem' in the .FEATURES special variable. Before that commit, pattern rules were evaluated so that the first matching rule picked. After the change, this was adapted so that the rule with the shortest stem was matched. We rely on this behaviour with how we build bundled Git: we've got two rules for building Git, one that matches "git-%/build/...
-
Patrick Steinhardt authored
This reverts commit 79e4d8fa.
-