Generate Redis interaction map systematically

In #1405 (closed), we put a significant amount of time and effort to capture the current state of interactions between application and Redis. The result is the list of components, how they interacts with Redis, and a detailed interaction map:

image

The data is valuable for the parent project. It is not likely that we can replicate this action again in the future, due to the high cost. There should be another way of accomplishing this task in the future. I don't have a precise solution at the moment. There are some approaches we can explore:

  • Implement a client-side instrumentation layer. This layer takes sample from the outgoing requests to Redis. The samples should be sanitized and normalized. As the instrumentation resides at the application layer, we know in advance the identity of clients. Pointing out the source component is a bit tricky. It may need to look at the current thread stack trace, which is expensive. There are also some concerns about the performance impact. The instrumentation couldn't capture the applications who don't attach the library.
  • Analyze the network traffic in Redis server. This approach is applied in some places like redis-load-test. It captures all requests from all clients. The analytics script is capable of pulling the detailed command and parameters from each client. It's still uncertain about how we can identify the client sources. Extracting the component information is also impossible. One potential solution is to add a thin layer to attach client information to the Redis request. Postgres marginallia is a good show case.

I'm not even sure whether the cost is worth the benefits the component list brings us. So, I'll leave this issue open. We can triage this when we have time.