Skip to content

Keep track of scan cursor used by RedisIndex delete_n_bytes

Jeremiah Bonney requested to merge jbonney/redis-cleanup-cache-cursor into master

Before raising this MR, consider whether the following are required, and complete if so:

  • Unit tests
  • Metrics
  • Documentation update(s)

If not required, please explain in brief why not.

Description

SCAN returns a stateless cursor indicating the current scan position, starting and ending at 0. For delete_n_bytes we currently always start with a new cursor, which means that we are starting at the beginning each time the method is called. Since we batch calls to this method in bgd-cleanup, this means we are continuously iterating over potentially previously scanned keys instead of getting keys which haven't been looked at yet.

By keeping track of the last known cursor position between method calls we can pick up the scan where it left off and are more likely to scan keys which haven't been scanned yet.

Changes proposed in this merge request:

  • Keep track of the cursor used in delete_n_bytes between invocations.

Validation

Cleanup should be faster as it won't need to start scanning with a fresh cursor every time.

Merge request reports