[backport-1.5]: CI breaks due to per unit timeout for 5m

A solution for issue #2863 (closed) needs to be backported to 1.5.x