Commits on Source (38)
-
🤖 GitLab Bot 🤖 authored
-
GitLab Release Tools Bot authored
[ci skip]
-
GitLab Release Tools Bot authored
[ci skip]
-
GitLab Release Tools Bot authored
[ci skip]
-
GitLab Release Tools Bot authored
-
Justin Tobler authored
In Git versions prior to 2.46.0, performing git-clone(1) on a bundle did not perform fsck checks even when `transfer.fsckObjects` was enabled. Add a test case exercising an fsck error being caught when cloning from a bundle.
-
Justin Tobler authored
Starting in Git version 2.46.0, executing git-clone(1) on a bundle performs fsck checks when `transfer.fsckObjects` is enabled. Prior to this, this configuration was always ignored and fsck checks were not run. Unfortunately, fsck message severity configuration is ignored by Git only for bundle clones. This issue is resolved in Git version 2.48.0, but until this gets rolled out to GitLab, disable `transfer.fsckObjects` so bundles containing fsck errors can continue to be cloned. This matches behavior prior to Git version 2.46.0.
-
James Liu authored
localrepo: Exercise `CloneBundle()` fsck configuration in test See merge request !7579 Merged-by:
James Liu <jliu@gitlab.com> Approved-by:
Emily Chui <echui@gitlab.com> Approved-by:
James Liu <jliu@gitlab.com> Co-authored-by:
Justin Tobler <jtobler@gitlab.com>
-
Eric Ju authored
The offloading storage is a cloud object store where we can save our offloaded object to. As part of repository offloading, interacting with the offloading storage is required for operations such as downloading, uploading, deleting, and listing. This commit introduces an new sink implementation to serve as an abstraction where we can interact with the offloading storage.
-
Eric Ju authored
We have retry and cancel support in the offloading sink. In order test those, we have to mock a bucket that can simulate delay and error. In this commit, we add a mock bucket. The bucket is based on fileblob bucket but intercept its response with cuostizable delay and error. With such simulation, we can write test cases to examine the offloading sink's behavior under long delay and frequent error.
-
Eric Ju authored
This commit adds tests to ensure proper configuration of sink options when creating a bucket sink. The tests also validate that: - Default values are correctly set. - Special cases, such as WithNoRetry(), properly configure the configuration as expected.
-
Eric Ju authored
To make sure offloading sink can working correctly and is capable to deal with retry and cancellation under long delay, various test cases are added for each each operation.
-
Eric Ju authored
housekeeping: Add offloading bucket sink Closes #6513 See merge request !7488 Merged-by:
Eric Ju <eju@gitlab.com> Approved-by:
Olivier Campeau <ocampeau@gitlab.com> Reviewed-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com> Reviewed-by:
Eric Ju <eju@gitlab.com> Reviewed-by:
Olivier Campeau <ocampeau@gitlab.com>
-
Mustafa Bayar authored
We download the archived WAL entry, extract it to a temporary dir and then finally move it to the partition's log directory. Currently this results in repeated LSN number in the final path such as: "/.../+gitaly/partitions/d4/73/2/wal/0000000000005/0000000000005/". This happens because we are extracting the tar into a path such as "/tempDirectory/0000000000005" but the tar file itself also contains the directory information such as: ➜ tar -tvf 0000000000005.tar drwxrwsrwx 0 git git 0 Jan 22 12:28 0000000000005/ -rw-rw-rw- 0 git git 41 Jan 22 12:28 0000000000005/1 -rw-rw-rw- 0 git git 830 Jan 22 12:28 0000000000005/MANIFEST And then finally we move it to its final destination in log with path like "../wal/0000000000005" using os.Rename(srcPath, destPath), which then results in final path with repeated LSN number. Changes in this MR: - Extracted the tar into a staging directory and then validated that the log entry exists at the correct path. Afterwards passed the path of the log entry directly to the log manager which fixes the issue. - Update unit tests to use realistic tar file structure.
-
Olivier Campeau authored
The name of the bundle URI manager is inconsistent across the program and in some part not very descriptive. The bundle URI manager is now renamed to `bundleURIManager`. References: This commit addresses the following comment: !7542 (comment 2298693589)
-
Olivier Campeau authored
Rename bundleManager to bundleURIManager See merge request !7573 Merged-by:
Olivier Campeau <ocampeau@gitlab.com> Approved-by:
Toon Claes <toon@gitlab.com> Approved-by:
Eric Ju <eju@gitlab.com>
-
Olivier Campeau authored
When a client performs a git-clone (SSH or Smarthttp) Gitaly instructs the `git-upload-pack` command to systematically advertise the bundle-uri capability. This situation causes the Git client to request a bundle-uri even when one does not exists, thus adding unnecessary latency to the overall `clone` operation. It also makes it hard to properly monitor the usage of bundle-uri by clients; we can't correlate a request for bundle-uri with an actual usage of it. The proposed changes have the `git-upload-pack` command advertise bundle-uri only when a bundle exists for the given repository. More context: For SSH, this change does not provide much benefit since the whole packfile negociation flow is performed on a single connection. In other words, the flow starts when the client starts the upload-pack service, and it ends when the client received all the objects it needed. For SmartHTTP, however, this is different. Each request/command sent by the client (`ls-refs`, `bundle-uri`, `fetch`) is sent on different, stateless, RPC calls. There is no way to know what command the client is sending to the git-upload-pack process until that process is actually started, which means that the Git config injected into the process must be computed in advance for every request; this implies computing the SignedURL of the bundle if it exist. By advertising `bundle-uri` capability only when a bundle exists for the given repository, we make sure the client won't send a `bundle-uri` command if no bundle exist. For SmartHTTP it not only reduces by 1 the number of round-trip request and the number of Git config computation, but it also makes it easier to monitor the use of `bundle-uri` feature, since we can be sure that when a client sends the `bundle-uri` command, it is because: 1. A bundle exists 2. The server advertised the capability 3. The client support the capability References: #6572
-
Nao Hashizume authored
-
Divya Rani authored
Let's remove no-op transport as we will implement grpc transport in subsequent commits.
-
Divya Rani authored
The SendMessage RPC doesn't need to be transactionalized so it safely bypass the transaction middleware.
-
Divya Rani authored
This commit introduces a gRPC transport interface for handling Raft communication. It implements the logic for sending and receiving Raft messages, including packing and unpacking log data. Additionally, the RaftManagerRegistry facilitates Raft manager registration and partition lookup. For the address lookup, we introduce a static routing table for now. In future, the table can be dynamic so that the addresses are propagated through gossiping.
-
Divya Rani authored
Add SendMessage RPC which leverages GrpcTransport.Receive to process incoming Raft messages. The Receive method is resposible for routing messages to the appropriate partition manager, unpacking log data and advancing the Raft state machine.
-
Divya Rani authored
This commit introduces a mock raft server and node for testing and adds tests to ensure the transport implementation is functioning correctly.
-
Marc Shaw authored
Change patch-id generation type from stable to verbatim to treat whitespace changes as meaningful changes. MR: gitlab.com/gitlab-org/gitaly/-/merge_requests/7587 Changelog: added
-
James Liu authored
go: Update module github.com/rubenv/sql-migrate to v1.7.1 See merge request !7518 Merged-by:
James Liu <jliu@gitlab.com> Approved-by:
Divya Rani <drani@gitlab.com> Approved-by:
James Liu <jliu@gitlab.com> Co-authored-by:
GitLab Renovate Bot <gitlab-bot@gitlab.com>
-
James Liu authored
recovery: Fix WAL entry tar extraction path Closes #6605 See merge request !7582 Merged-by:
James Liu <jliu@gitlab.com> Approved-by:
James Fargher <jfargher@gitlab.com> Approved-by:
James Liu <jliu@gitlab.com> Reviewed-by:
James Fargher <jfargher@gitlab.com> Reviewed-by:
James Liu <jliu@gitlab.com> Co-authored-by:
Mustafa Bayar <mbayar@gitlab.com>
-
Mustafa Bayar authored
Bundle-uri capability: Only advertise when bundle exist See merge request !7572 Merged-by:
Mustafa Bayar <mbayar@gitlab.com> Approved-by:
Toon Claes <toon@gitlab.com> Approved-by:
Mustafa Bayar <mbayar@gitlab.com> Reviewed-by:
Toon Claes <toon@gitlab.com> Reviewed-by:
Mustafa Bayar <mbayar@gitlab.com> Co-authored-by:
Olivier Campeau <ocampeau@gitlab.com>
-
🤖 GitLab Bot 🤖 authored
-
Justin Tobler authored
By default, Git uses SHA1DC for all SHA1 hashing operations. This implementation includes collision detection to protect against collision attacks, but comes with a performance penalty. Not all hashing operations performed by Git are vulnerable to collision attacks though. In Git version 2.47.0, Git can be built with `OPENSSL_SHA1_UNSAFE` set to use a non-collision-detecting SHA1 implementation in non-cryptographic scenarios. This improves performance without compromising security. Update the Gitaly Makefile to build the bundled Git v2.48 with this option set.
-
Mustafa Bayar authored
Enable verbatim_patch_id feature flag by default See merge request !7587 Merged-by:
Mustafa Bayar <mbayar@gitlab.com> Approved-by:
Mustafa Bayar <mbayar@gitlab.com> Approved-by:
Timothy Schumacher <tschumacher@gitlab.com> Co-authored-by:
Marc Shaw <mshaw@gitlab.com>
-
Divya Rani authored
Implement networking handlers for Raft Closes #6401 See merge request !7544 Merged-by:
Divya Rani <drani@gitlab.com> Approved-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com> Reviewed-by:
Quang-Minh Nguyen <qmnguyen@gitlab.com> Reviewed-by:
James Liu <jliu@gitlab.com>
-
Divya Rani authored
Update dependency git to v2.47.2 See merge request !7591 Merged-by:
Divya Rani <drani@gitlab.com> Approved-by:
Mustafa Bayar <mbayar@gitlab.com> Approved-by:
Divya Rani <drani@gitlab.com> Co-authored-by:
GitLab Renovate Bot <gitlab-bot@gitlab.com>
-
Patrick Steinhardt authored
Makefile: Build Git with non-collision-detecting SHA1 See merge request gitlab-org/gitaly!7575 Merged-by:
Patrick Steinhardt <psteinhardt@gitlab.com> Approved-by:
Emily Chui <echui@gitlab.com> Approved-by:
Toon Claes <toon@gitlab.com> Co-authored-by:
Justin Tobler <jtobler@gitlab.com>
-
Divya Rani authored
-
Patrick Steinhardt authored
The `test-with-reftable` CI job executes the "test-go" target, where the only difference to our normal test target is that it also exports the "GIT_DEFAULT_REF_FORMAT" environment variable. This environment variable gets exported into all dependencies, which causes us to initialize repos other than the test repositories created by our actual unit tests with the "reftable" format. This includes for example the bundled Git repos. This causes an issue with CI caches: it can happen that a job gets a CI cache that uses the "files" format for the bundled Git repos, but the default ref format is "reftable". This causes the following error when reinitializing the directoy: fatal: could not open '/builds/gitlab-org/gitaly/_build/deps/git-v2.48/.git/refs/heads' for writing: Is a directory The issue here is that git-init(1) tries to reinitialize the repository with the wrong ref format. When doing so it wants to write "refs/heads" as a file containing garbage to keep other Git clients which aren't yet aware of the "reftable" format from reading the repository. But because the repo uses the "files" format the path is a directory, and thus writing the file fails. The issue can be trivially reproduced with an up-to-date Git version: $ git init --ref-format=files repo Initialized empty Git repository in /tmp/repo/.git/ $ GIT_DEFAULT_REF_FORMAT=reftable git init repo fatal: could not open '/tmp/repo/.git/refs/heads' for writing: Is a directory Even though this is an upstream bug, it still doesn't make sense for us to propagate the `GIT_DEFAULT_REF_FORMAT` environment variable into all build prerequisites. Fix this by using a Gitaly-specific environment variable instead and setting it up when configuring our tests.
-
John Cai authored
Bump development Ruby version to 3.3.7 See merge request !7585 Merged-by:
John Cai <jcai@gitlab.com> Approved-by:
John Cai <jcai@gitlab.com> Co-authored-by:
Nao Hashizume <nhashizume@gitlab.com>
-
Justin Tobler authored
Makefile: fix default ref format propagating into build prereqs See merge request !7596 Merged-by:
Justin Tobler <jtobler@gitlab.com> Approved-by:
Justin Tobler <jtobler@gitlab.com> Approved-by:
John Cai <jcai@gitlab.com> Reviewed-by:
Justin Tobler <jtobler@gitlab.com> Co-authored-by:
Patrick Steinhardt <psteinhardt@gitlab.com>
-
Emily Chui authored
chore: Ignore GO-2025-3408 vulnerability See merge request !7595 Merged-by:
Emily Chui <echui@gitlab.com> Approved-by:
Emily Chui <echui@gitlab.com> Co-authored-by:
Divya Rani <drani@gitlab.com>
Showing
- .ruby-version 1 addition, 1 deletion.ruby-version
- .tool-versions 1 addition, 1 deletion.tool-versions
- CHANGELOG.md 12 additions, 0 deletionsCHANGELOG.md
- Makefile 8 additions, 2 deletionsMakefile
- go.mod 2 additions, 2 deletionsgo.mod
- go.sum 4 additions, 4 deletionsgo.sum
- internal/bundleuri/git_config.go 24 additions, 39 deletionsinternal/bundleuri/git_config.go
- internal/bundleuri/git_config_test.go 83 additions, 119 deletionsinternal/bundleuri/git_config_test.go
- internal/bundleuri/manager.go 17 additions, 0 deletionsinternal/bundleuri/manager.go
- internal/cli/gitaly/serve.go 30 additions, 22 deletionsinternal/cli/gitaly/serve.go
- internal/cli/gitaly/subcmd_recovery.go 22 additions, 4 deletionsinternal/cli/gitaly/subcmd_recovery.go
- internal/cli/gitaly/subcmd_recovery_test.go 29 additions, 13 deletionsinternal/cli/gitaly/subcmd_recovery_test.go
- internal/featureflag/ff_gitaly_verbatim_patch_id.go 1 addition, 1 deletioninternal/featureflag/ff_gitaly_verbatim_patch_id.go
- internal/git/gittest/reference_backend.go 1 addition, 1 deletioninternal/git/gittest/reference_backend.go
- internal/git/localrepo/bundle.go 7 additions, 0 deletionsinternal/git/localrepo/bundle.go
- internal/git/localrepo/bundle_test.go 34 additions, 2 deletionsinternal/git/localrepo/bundle_test.go
- internal/gitaly/service/dependencies.go 34 additions, 27 deletionsinternal/gitaly/service/dependencies.go
- internal/gitaly/service/raft/send_message.go 49 additions, 0 deletionsinternal/gitaly/service/raft/send_message.go
- internal/gitaly/service/raft/send_message_test.go 136 additions, 0 deletionsinternal/gitaly/service/raft/send_message_test.go
- internal/gitaly/service/raft/server.go 26 additions, 0 deletionsinternal/gitaly/service/raft/server.go
internal/gitaly/service/raft/send_message.go
0 → 100644
internal/gitaly/service/raft/server.go
0 → 100644