This MR removes the use of the feature flag for benchmarking Redis
pipelines in GitLab Rails.

Changelog: other
MR: !118884

Sylvester Chin authored 1 year ago and Bob Van Landuyt committed 1 year ago

This MR adds Gitlab::Redis::CrossSlot module to wrap over .pipelined
calls for relevant call-sites.

Changelog: performance

Sylvester Chin authored 1 year ago and Sincheol (David) Kim committed 1 year ago

This is to benchmark the effects of decomposing multi-key commands into
pipelines to ensure Redis Cluster compatibility.

Using pipeline will improve write performance and enable Redis
Cluster compatibility. Transactions cannot be cross-slot whereas
pipelines can so long as the server contains all required hash-slots.

Kerri Miller authored 4 years ago and Douglas Barbosa Alexandre committed 4 years ago

This module holds common logic for compression and decompression of
strings via ActiveSupport::Gzip. The destination of these compressed
strings is usually Redis, so encoding is handled, as is compression
detection when decompressing.

Sean McGivern authored 4 years ago and Bob Van Landuyt committed 4 years ago

If we want to use Redis Cluster, we won't be able to use a multi-key
command with keys that hash to different slots. For instance, a simple:

    MGET foo bar

Won't work because 'foo' and 'bar' hash to different slots, and
therefore could be on different shards of the cluster.

This is a client-side validator to ensure that we can annotate existing
cross-slot commands easily, as part of the data gathering for whether or
not we use Redis Cluster.

Runs the Gitlab/Json cop autocorrector over the lib directory.

It extracts a method to be reused by all diff caches,
which ensures the rich_text of diff lines are
html safe. Otherwise we'd see a wrong state in
the diff changes tab.

This commit handles note diffs caching, which considerably improves
the performance on merge requests with lots of comments.
Important to note that the caching approach taken here is different
from `Gitlab::Diff::HighlightCache`. We do not reset the whole cache
when a new push is sent or anything else. That's because discussions
diffs are persisted and do not change.