Discussion: RPC level caching
In an effort to reduce Gitaly disk IO, it may be possible to utilize a cache to speed up small but frequent RPC's. This also may be possible to do in a transparent way such that the current Gitaly server design is not complicated with the details of such a cache strategy. Many of the parts that could make this possible are already being designed and implemented for Gitaly HA:
- Ability to detect which RPC's modify a repo in an RPC's stream (#1496 (closed))
- Ability to detect which repository an RPC is modifying in an RPC's stream (#1588 (closed))
A simple design might involve detecting when a repo's cache is invalidated by a mutating RPC, and otherwise providing a cached response.
Also, a simple design might require whitelisting specific RPC's for cache by annotating protobuf service definitions:
rpc HasLocalBranches(HasLocalBranchesRequest) returns (HasLocalBranchesResponse) {
option (op_type).op = ACCESSOR;
option (op_type).cache = MEMORY; // otherwise, cache defaults to NONE
}
How this is ultimately implemented might involve writing gRPC server-side unary/stream interceptors in Gitaly. Requests that are cached can be returned without touching the RPC specific code, while non-cached responses proceed as usual.
Relates to &289 (closed) and &827