DLE accumulates postgres logs - we store them infinitely. Is this a problem? (Was: add log rotation)
Goal
DLE accumulates PostgreSQL logs:
$ sudo ls -la /var/lib/dblab2/dblab_pool2/data/log | wc -l
25411
$ sudo du --max-depth 1 -h /var/lib/dblab2/dblab_pool2/data/log
3.3G /var/lib/dblab2/dblab_pool2/data/log
Consider adding the log rotation.
We cannot rely on PostgreSQL logging_collector for log rotation since:
The name of log files must contain a creation date and be uniquely ordered by time, otherwise, the idleness check will be broken
We also know that:
This may be a big issue only in the case of physical mode. For logical, we perform full refreshes, destroying PGDATA and log directory anyway (
forceInit: true
).
TODO / How to implement
-
Implement logic on the DLE side to remove logs from the log directory (
poolManager.mountDir.dataSubDir/log
, example/var/lib/dblab/dblab_pool/data/log
) older than N (for example, older than one week). -
Before creating a new snapshot, delete all logs from the
clone_pre_xxx
data/log directory.
Acceptance criteria
- logs in the directory
dataSubDir/log
are no longer accumulated - new snapshots are created without old logs
Edited by Vitaliy Kukharik