Skip to content

Only adjust total_size if keys are new when writing / exist when deleting

Jeremiah Bonney requested to merge jbonney/redis-index-total-size-fixes into master

Before raising this MR, consider whether the following are required, and complete if so:

  • Unit tests - Existing storage tests provide some basic coverage

Description

This PR adjusts how the RedisIndex handles the total_size key when it comes to writes/deletes. Previously total_size would be incremented/decremented for writes/deletes regardless of whether the key already existed or didn't exist. This can lead to total_size getting out of sync if there are cases where the same digest is written to storage multiple times or deleted multiple times. This PR addresses this by doing some checking of the result of writes/deletes to make sure that writes only increment total_size if the key didn't exist previously and deletes only decrement it if the key already existed.

The downside of this is that the adjustments of total_size need to be made in a separate request and can't be part of the same pipeline which incurs an additional round trip to Redis.

Merge request reports