Populate Prod DB Failed After 5 Hr Timeout

The populate_prod_db job failed twice for 500 errors and then timed out after 5 hours. I do not have the ability to retry even though it looks like developers should based on https://docs.gitlab.com/ee/user/permissions.html#project-members-permissions 🤔

Perhaps there was some performance degradation I'm missing as we had a number of failures on 2018-08-30.