svcrec errors when running out of disk space
We have noticed in several environments that svcrec will incorrectly start to report on different services, when the system is running out of disk space. Example below:
Correct reporting of mongodb service
1599724595:nexetic-mongo05 service_nrpe:2.69:nexetic-mongo05 service:EFHK:NRPE(5666):1
1599724655:nexetic-mongo05 service_mongodb:1.06:nexetic-mongo05 service:EFHK:MONGODB(27017):1
and immediately we see some new record, which is wrong:
1599724655:nexeti1599725975:nexetic-mongo05 service_mongodb:0.31:nexetic-mongo05 service:EFHK:MONGODB(27017):1
1599725975:nexetic-mongo05 service_nrpe:0.25:nexetic-mongo05 service:EFHK:NRPE(5666):1
1599726035:nexetic-mongo05 service_mongodb:1.37:nexetic-mongo05 service:EFHK:MONGODB(27017):1
1599726035:nexetic-mongo05 service_nrpe:0.91:nexetic-mongo05 service:EFHK:NRPE(5666):1
1599726095:nexetic-mongo05 service_mongodb:0.30:nexetic-mongo05 service:EFHK:MONGODB(27017):1
1599726095:nexetic-mongo05 service_nrpe:0.25:nexetic-mongo05 service:EFHK:NRPE(5666):1
This can be most likely be reproduced on a system which runs out of disk space.