- Sep 09, 2021
-
-
Sami Hiltunen authored
Derive virtual storage's filesystem id from its name (14.1) See merge request !3835
-
- Sep 03, 2021
-
-
Sami Hiltunen authored
Rails tests configure Praefect in front of the tests that exercise the Rugged direct git access code. As Praefect is now deriving the filesystem IDs from the names of the virtual storages, the filesystem id checks fail and thus the test fail. This is not a problem in practice as one wouldn't use rugged in a real-world setup with Praefect. This commit worksaround the tests by returning the filesystem ID from the Gitaly node if a virtual storage has only one Gitaly node configured. This matches the setup the tests use and thus pass them. The workaround and the filesystem ID code can be removed in 15.0 once the rugged patches and NFS support are dropped.
-
Sami Hiltunen authored
Gitaly storages contain a UUID filesystem ID that is generated by the Gitaly for each of its storages. The ID is used to determine which storages can be accessed by Rails directly when rugged patches are enabled and to see whether two different storages point to the same directory when doing repository moves. When repository moves are performed, the worker first checks whether the repository's destination and source storage are the same. If they are, the move is not performed. The check is performed by comparing the filesystem IDs of the storages'. As Praefect is currently routing the server info RPC to a random Gitaly node, the filesystem ID can differ between calls as each of the Gitalys have their own ID. This causes the repository moving worker to occasionally delete repositories from the virtual storage as it receives two different IDs on sequential calls. The filesystem ID can identify cases when two storages refer to the same directory on a Gitaly node as the id is stored in a file in the storage. This is not really possible with Praefect. The storage's are only identified by the virtual storage's name. If the name changes, we can't really correlate the ID between the different names as Praefect would consider them different storages. Praefect also supports multiple virtual storages so it's not possible to generate a single ID and use it for all of the virtual storages. Given this, the approach taken here is to derive a stable filesystem ID from the virtual storage's name. This guarantees calls to a given virtual storage always return the same filesystem ID. Configuring two storages that point to the same filesystem should be considered an invalid configuration anyway. Historically, there's been cases when that has been done for plain Gitalys. This is not done for Praefect and wouldn't work as Praefect wouldn't find the repositories with an alternative virtual storage name. With that in mind, we don't have to consider the case where two virtual storages of different names point to the same backing Gitaly storages. The use cases for the filesystem ID seem to be limited and we may be able to remove it in the future once the rugged patches are removed. Changelog: fixed
-
- Sep 02, 2021
-
-
GitLab Release Tools Bot authored
[ci skip]
-
GitLab Release Tools Bot authored
[ci skip]
-
Henri Philipps authored
Backport improved replication logic (v14.1) See merge request !3823
-
- Sep 01, 2021
-
-
Patrick Steinhardt authored
When finalizing a transaction, we always schedule replication jobs in case the primary has returned an error. Given that there are many RPCs which are expected to return errors in a controlled way, e.g. if a commit is missing, this causes us to create replication in many contexts where it's not necessary at all. Thinking about the issue, what we really care for is not whether an RPC failed or not. It's that primary and secondary nodes behaved the same. If both primary and secondaries succeeded, we're good. But if both failed with the same error, then we're good to as long as all transactions have been committed: quorum was reached on all votes and nodes failed in the same way, so we can assume that nodes did indeed perform the same changes. This commit thus relaxes the error condition to not schedule replication jobs anymore in case the primary failed, but to only schedule replication jobs to any node which has a different error than the primary. This has both the advantage that we only need to selectively schedule jobs for disagreeing nodes instead of targeting all secondaries and it avoids scheduling jobs in many cases where we do hit errors. Changelog: performance (cherry picked from commit 73839029)
-
- Aug 31, 2021
-
-
GitLab Release Tools Bot authored
-
Sami Hiltunen authored
[14.1] Only activate Git pack-objects hook if cache is enabled See merge request !3814
-
GitLab Release Tools Bot authored
[ci skip]
-
GitLab Release Tools Bot authored
[ci skip]
-
Stan Hu authored
In !3301, we dropped the `upload_pack_gitaly_hooks` feature flag because it was confusing to have to enable this feature flag on top of the pack objects cache setting in `config.toml`. However, we have found that spawning the gitaly-hooks environment can add significant CPU load due to overhead from transferring data over gRPC and spawning gitaly-hooks processes. We now only enable this hook if the pack objects cache is enabled. Relates to #3754 Changelog: performance
-
- Aug 17, 2021
-
-
GitLab Release Tools Bot authored
[ci skip]
-
GitLab Release Tools Bot authored
[ci skip]
-
- Aug 03, 2021
-
-
GitLab Release Tools Bot authored
-
GitLab Release Tools Bot authored
[ci skip]
-
GitLab Release Tools Bot authored
[ci skip]
-
- Aug 02, 2021
-
-
GitLab Release Tools Bot authored
Allow parsing of long git commit headers See merge request gitlab-org/security/gitaly!40
-
- Jul 28, 2021
-
-
GitLab Release Tools Bot authored
[ci skip]
-
GitLab Release Tools Bot authored
[ci skip]
-
- Jul 22, 2021
-
-
James Fargher authored
`bufio.Scanner` can only handle lines of a certain length. So to enable parsing of very large git commit headers the parser implementation was switched to use `bufio.Reader.ReadString`. Changelog: security
-
- Jul 21, 2021
-
-
GitLab Release Tools Bot authored
[ci skip]
-
GitLab Release Tools Bot authored
[ci skip]
-
- Jul 20, 2021
-
-
GitLab Release Tools Bot authored
[ci skip]
-
GitLab Release Tools Bot authored
[ci skip]
-
- Jul 15, 2021
-
-
Toon Claes authored
Update ffi gem to 1.15.3 See merge request !3664
-
James Fargher authored
Set default Prometheus buckets for Gitalys RPC instrumentation Closes #3431 See merge request !3669
-
- Jul 14, 2021
-
-
Sami Hiltunen authored
Gitaly doesn't set default buckets for RPC latency instrumentation which leads to the instrumentation being disabled by default. This commit adds default buckets to the configuration which is used if the buckets are not explicilty configured. Changelog: changed
-
Sami Hiltunen authored
Add StreamRPC library code See merge request !3601
-
Zeger-Jan van de Weg authored
Support lazy failovers in `praefect dataloss` See merge request !3549
-
- Jul 13, 2021
-
-
Zeger-Jan van de Weg authored
Update read-only repository count metric to account for lazy failover See merge request !3548
-
James Fargher authored
Remove feature gitaly_fetch_internal_remote_errors Closes #3588 See merge request !3647
-
- Jul 12, 2021
-
-
James Fargher authored
Since FetchInternalRemote has been inlined into ReplicateRepository we no longer need to make this RPC errors more verbose.
-
Zeger-Jan van de Weg authored
Fix various static lint issues See merge request !3666
-
Zeger-Jan van de Weg authored
Perform failovers lazily Closes #3207 See merge request !3543
-
Jacob Vosmaer authored
Changelog: other
-
Pavlo Strokov authored
It is not common to use snake case names for packages and package aliases in Go. The change renames aliases to a preferred single word name. We also use 'gitaly' prefix for the project-defined packages that clashes with standard or 3-rd party package names.
-
Pavlo Strokov authored
Some functions, types, fields and other variables are not used. There is no reason to keep them and support. Some of them became redundant starting from declaration and some after the code changes.
-
Pavlo Strokov authored
The done variable is never assigned any value, so the condition always evaluates into true.
-
Pavlo Strokov authored
If struct has a list of fields declared the values for those fields could be assigned during struct instance creation by providing values in the same order as the fields are declared or in any order if field names are used to assign the values. The preferred way is to use a field name assigment as it is less error prone if you define a new field in the middle of the struct and as well more readable as you see the list of the initialized fields.
-