Investigate review-performance job failure - Follow-up from "Retry review apps jobs in all cases"

The following discussion from !114444 (merged) should be addressed: