Skip to content

Get a new connection after every retry, use jitter instead of exponential backoff

Jeremiah Bonney requested to merge jbonney/redis-reconnection into master

Before raising this MR, consider whether the following are required, and complete if so:

  • Unit tests
  • Metrics
  • Documentation update(s)

If not required, please explain in brief why not.

Description

This PR makes a couple of adjustments to how Redis connections and retries are handled.

  • Update socket timeout to 1 second instead of 0.2 seconds. We leverage pipelines in several places which may result in slightly longer requests that we don't want to drop on the floor.
  • Update _connect() to only create a new Sentinel object the first time. The topology of Sentinels should change much less than the underlying cache nodes, so we should only need to get new connections instead of doing the entire Sentinel resolution song and dance.
  • Get a new connection on every error by handling retries ourselves. We previously would retry 3 times first and only then get a new connection.
  • Switch the retry strategy to EqualJitterBackoff instead of ExponentialBackoff and use the default (much lower) base/cap values.

With these changes redis connections should be more prompt to reconnect and better able to handle longer-running commands.

Merge request reports