Investigate high memory use in redis connections
The following discussion from !91853 (merged) should be addressed:
-
@engwan started a discussion: (+2 comments) This is now in staging canary so I compared a request with canary enabled and disabled.
https://staging.gitlab.com/gitlab-org/gitlab has 45k branches so I tested an issue in this project.
duration_s
is down from 2.937 to 0.175.mem_bytes
is down from 1.2g to 6mb.I'm still confused why this uses that much memory even in the old code.
redis_cache_read_bytes
is "just" 1.6mb. Is the Ruby socket library that inefficient? Or is it the allocation of 45k+ string objects after we get that from the Redis set?🤔 Maybe it's worth taking another look at
hiredis
: !25850 (closed). The problem experienced there was also large SMEMBERS results. So I wonder if it also reduces memory consumption.I'll try to take a look when I have time.