Protobuf specifications and client libraries for Gitaly
Gitaly is part of GitLab. It is a server application that uses its own gRPC protocol to communicate with its clients. This repository contains the protocol definition and automatically generated wrapper code for Go and Ruby.
The .proto files define the remote procedure calls for interacting with Gitaly. We keep auto-generated client libraries for Ruby and Go in their respective subdirectories.
_support/generate-from-proto script from the root of the
repository to regenerate the client libraries after updating .proto
See developers.google.com for documentation of the 'proto3' Protocul buffer specification language.
The core Protobuf concepts we use are rpc, service and message. We use these to define the Gitaly protocol.
- rpc a function that can be called from the client and that gets executed on the server. Belongs to a service. Can have one of four request/response signatures: message/message (example: get metadata for commit xxx), message/stream (example: get contents of blob xxx), stream/message (example: create new blob with contents xxx), stream/stream (example: git SSH session).
- service a logical group of RPC's.
- message like a JSON object except it has pre-defined types.
- stream an unbounded sequence of messages. In the Ruby clients this looks like an Enumerator.
gRPC provides an implementation framework based on these Protobuf concepts.
- A gRPC server implements one or more services behind a network listener. Example: the Gitaly server application.
- The gRPC toolchain automatically generates client libraries that handle serialization and connection management. Example: the Go client package and Ruby gem in this repository.
- gRPC clients use the client libraries to make remote procedure calls. These clients must decide what network address to reach their gRPC servers on and handle connection reuse: it is possible to spread different gRPC services over multiple connections to the same gRPC server.
- Officially a gRPC connection is called a channel. In the Go gRPC library these channels are called client connections because 'channel' is already a concept in Go itself. In Ruby a gRPC channel is an intance of GRPC::Core::Channel. We use the word 'connection' in this document. The underlying transport of gRPC, HTTP/2, allows multiple remote procedure calls to happen at the same time on a single connection to a gRPC server. In principle, a multi-threaded gRPC client needs only one connection to a gRPC server.
- In Gitaly's case there is one server application https://gitlab.com/gitlab-org/gitaly which implements all services in the protocol.
- Gitaly clients use one HTTP/2 connection per Gitaly server they interact with. All services used by the client share the same connection.
- Currently each Gitaly client interacts with exactly 1 Gitaly server, on the same host, via a Unix domain socket. In the future each Gitaly client will interact with many different Gitaly servers (one per GitLab storage shard) via TLS connections.
- Gitaly clients will 'cache' their gRPC connections to avoid
connection setup overhead on each RPC call. Pitfall: the Ruby 'grpc'
gem creates convenience methods that let you establish many new
connections if you are not careful. To avoid this we will always use
Stub.new(nil, credentials, channel_override: the_channel)invocation pattern instead of
Stub.new('some://address', credentials)in Ruby Gitaly clients.
- Gitaly uses grpc.Errorf to return meaningful errors to its clients.
- Each RPC
FooBarhas its own
FooBarResponsemessage types. Try to keep the structure of these messages as flat as possible. Only add abstractions when they have a practical benefit.
- We never make backwards incompatible changes to an RPC that is already implemented on either the client side or server side. Instead we just create a new RPC call and start a deprecation procedure (see below) for the old one.
The CI at https://gitlab.com/gitlab-org/gitaly-proto regenerates the
client libraries to guard against the mistake of updating the .proto
files but not the client libraries. This check uses
git diff to look
for changes. Some of the code in the Go client libraries is sensitive
to implementation details of the Go standard library (specifically,
the ouput of gzip). Use the same Go version as .gitlab-ci.yml (Go
1.8) when generating new client libraries for a merge request.
After you change or add a .proto file you need to re-generate the Go and Ruby libraries before committing your change.
# Install dependencies _support/install-protoc # Re-generate Go and Ruby libraries _support/generate-from-proto
How to deprecate an RPC call