Fix errors with resource deletion in cleanup scripts and fail at the end
Retry on 502 response errors to prevent errors in parse_body
in wait_for_resource_deletion
so it doesn't stop the cleanup script in the middle of a run.
Don't log failure if resource returns 404 when trying to delete as a 404 is the end goal of deletion.
We then need to fail the job after the script is done if there were any errors. This allows for automatic retry as well.
Example error of script stopping in the middle: https://ops.gitlab.net/gitlab-org/quality/staging/-/jobs/13826859
Example error of 404 even though we want the resource to be deleted: https://ops.gitlab.net/gitlab-org/quality/staging/-/jobs/13860896
Example error of job succeeding even though there were errors: https://ops.gitlab.net/gitlab-org/quality/staging/-/jobs/13859985
Example error: https://ops.gitlab.net/gitlab-org/quality/staging/-/jobs/13826859