Baker client is waiting too long for operations
Environment (Alphanet, build from source, or both)
Latest mainnet commit hash 06398944
Summary
Baker client waited too long for endorsements before producing a block, and eventually aborted. The priority 2 baker eventually produced the block at that height (829846).
Expected behavior
The baker client should have produced a block, even without all 32 endorsements, before the deadline.
Actual behavior
The baker client timed out.
Steps to reproduce
It is difficult to reliably reproduce this issue, as it depends on network conditions.
History mode
archive
Logs
First, the baker client produced this message:
Feb 18 13:25:30 - 005-PsBabyM1.baking.forge: New baking slot found (level 829846, priority 0) at 2020-02-18T13:25:50-00:00 for cb-baker after BMLLjbHhSSoN.
Then, a couple minutes later, the node produced:
Feb 18 13:27:23 - validator.block: Block BLDZXAWCnYaqtHKFT6NR2ELwnVo9t2ctRKZBcjzmGu6HFX3N8HG successfully validated
Feb 18 13:27:23 - prevalidator.NetXdQprcVkpa.PsBabyM1eUXZ: switching to new head BLDZXAWCnYaqtHKFT6NR2ELwnVo9t2ctRKZBcjzmGu6HFX3N8HG
Feb 18 13:27:23 - prevalidator.NetXdQprcVkpa.PsBabyM1eUXZ: Request pushed on 2020-02-18T13:27:23-00:00, treated in 715us, completed in 301ms
Feb 18 13:27:23 - validator.chain: Request pushed on 2020-02-18T13:27:23-00:00, treated in 1.95ms, completed in 308ms
Finally, the baking client aborted:
Feb 18 13:27:23 - 005-PsBabyM1.baking.forge: Received a new head while waiting for operations. Aborting this block.
There is a CPU and disk utilization spike right before the first log message at 13:24:20: