Node Uncaught Exception Out of Memory
This happened today:
May 12 10:23:56 - validator.peer: Worker terminated [validator-peer_249139]
May 12 10:24:00 - validator.peer: Worker terminated [validator-peer_249167]
May 12 10:24:05 - validator.peer: Worker terminated [validator-peer_247987]
May 12 10:24:08 - validator.peer: Worker started for NetXdQprcVkpa:idrTxjkFTZN6
May 12 10:24:09 - validator.peer: Worker started for NetXdQprcVkpa:idqfRjMCuSwy
May 12 10:24:10 - validator.peer: Worker started for NetXdQprcVkpa:idrK62PJcqsJ
May 12 10:24:38 - validator.block: Block BLTEEbR9jwZjNyCdmcsN2Ti56XRWmv4EKLELSGKeTgXbaifnQQk successfully validated
May 12 10:24:38 - validator.block: Request pushed on 2020-05-12T10:24:33-00:00, treated in 4.173ms, completed in 5.162s
May 12 10:24:47 - prevalidator.NetXdQprcVkpa.PsCARTHAGazK: switching to new head BLTEEbR9jwZjNyCdmcsN2Ti56XRWmv4EKLELSGKeTgXbaifnQQk
May 12 10:24:47 - prevalidator.NetXdQprcVkpa.PsCARTHAGazK: Request pushed on 2020-05-12T10:24:38-00:00, treated in 175ms, completed in 8.644s
Uncaught (asynchronous) exception:
Out of memory
Raised at file "format.ml" (inlined), line 242, characters 35-52
Called from file "format.ml", line 469, characters 8-33
Called from file "format.ml", line 484, characters 6-24
Uncaught (asynchronous) exception:
Out of memory
Raised at file "format.ml" (inlined), line 242, characters 35-52
Called from file "format.ml", line 469, characters 8-33
Called from file "format.ml", line 484, characters 6-24
Uncaught (asynchronous) exception:
Out of memory
Raised at file "format.ml" (inlined), line 242, characters 35-52
Called from file "format.ml", line 469, characters 8-33
Called from file "format.ml", line 484, characters 6-24
Uncaught (asynchronous) exception:
Out of memory
Raised at file "format.ml" (inlined), line 242, characters 35-52
Called from file "format.ml", line 469, characters 8-33
Called from file "format.ml", line 484, characters 6-24
May 12 10:24:48 - validator.chain: Update current head to BLTEEbR9jwZjNyCdmcsN2Ti56XRWmv4EKLELSGKeTgXbaifnQQk (fitness 01::0000000000047b36), same branch
May 12 10:24:48 - validator.chain: Request pushed on 2020-05-12T10:24:38-00:00, treated in 297ms, completed in 9.110s
May 12 10:24:54 - validator.peer: Worker terminated [validator-peer_249188]
May 12 10:24:57 - validator.peer: Worker terminated [validator-peer_249190]
May 12 10:25:19 - validator.peer: Worker started for NetXdQprcVkpa:idtEiNw63djk
May 12 10:25:42 - validator.block: Block BKixbr2jdkLvyUvDxBBL1dhXoDojfg1TUSa4VbJ8hFMyxR5q7BC successfully validated
Subsequently, my baker and endorser both stopped as well. "Lost connection to node".
The process 'tezos-node' is also pegged 1 CPU core at 100%. I've been watching top
for the past several minutes, and the node is 99-100% constantly.