Caddy healthcheck is leaking memory
Describe the problem
In docker-compose.yml
(the one with separate services), the Caddy service's healthcheck
implementation is causing a memory leak.
Steps to reproduce
- Deploy
docker-compose.yml
to a host - Observe the host's memory consumption
Actual result
Over a timeframe where the service is not used, constantly increasing memory consumption on the host is observed
Expected result
Over a timeframe where the service is not used, a flat, consistent memory consumption on the host is observed
Environment
- Operating System: Ubuntu Server 20.04
-
Deployment Type: Current
docker-compose.yml
file implementation
Notes
After deploying docker-compose.yml
, I spent some time troubleshooting why my host server had an apparent memory leak.
It appears the way docker-compose.yml
implements the Caddy service's healthcheck
causes this.
I traced it down and discussed it with the Caddy maintainers in their project tracker. That conversation included more background and yielded some informative conversation about the issue. This is located here: https://github.com/caddyserver/caddy-docker/issues/276
As well, I tested removing the healthcheck
from docker-compose.yml
entirely to check if it corrected the issue. Indeed, removing it stopped the memory leak.
Here is an actual graph from CloudWatch that tracks % memory used on the host. The timeframe where memory consumption is gradually increasing was when the host was running the current healthcheck
implementation. The timeframe where memory consumption is flat is when I removed the healthcheck
implementation.