Migrate redis keys from binary to ascii
Background
Redis itself does not care about whether keys containe binary data or not.
However, we have various tools for diagnosing issues with redis that rely on analyzing the key space, and we often use standard unix tools like grep and awk for filtering. In order to effectively aggregate and visualize these values without the need for developing custom tools, it is strongly preferable to have ascii keys.
The most recent analysis tool requires special handling for binary data, and so do many of the other tools.
Problem
KAS makes use of binary data in order to be more memory efficient. We do indeed have a trade-off here between ease of analysis (tooling) and memory space.
Proposal
However, the current Redis memory footprint of KAS keys is less than 6 MB. Even if this change were to double the memory footprint, it is still a tiny fraction of the current 7.5 GiB memory usage, and an even smaller fraction of the 30GB+ budget.
Thus it's probably worth trading a little bit of extra space for much easier ad-hoc analyses.
As such, this is a proposal to move from packed binary keys to ascii.
- Audit all places where Redis keys are constructed.
- Change the places where binary keys are used.
- Make sure to change the key prefix when changing the key format to ensure forward/backward compatibility.