Redis outage: 2018-03-15 22:40 - 22:43 UTC
It looks like the Linux kernel killed redis-01 due to OOM:
[5855861.140467] Out of memory: Kill process 97185 (redis-server) score 970 or sacrifice child
[5855861.145816] Killed process 97185 (redis-server) total-vm:115079856kB, anon-rss:111855536kB, file-rss:0kB, shmem-rss:0kB
Yet we have the LRU settings set here:
# Memory
used_memory:107911646936
used_memory_human:100.50G
used_memory_rss:111681261568
used_memory_rss_human:104.01G
used_memory_peak:107914051632
used_memory_peak_human:100.50G
total_system_memory:118272950272
total_system_memory_human:110.15G
used_memory_lua:54272
used_memory_lua_human:53.00K
maxmemory:107374182400
maxmemory_human:100.00G
maxmemory_policy:allkeys-lru
mem_fragmentation_ratio:1.03
mem_allocator:jemalloc-4.0.3