Prevent empty manifest from getting created during GCS outage
GCS went down on 3/13/19 and during the outage images were pushed but with empty manifests for some reason. Once GCS recovered, our users were still impacted because when they tried to pull those images that got pushed with empty manifests they were getting errors: Invalid end of JSON format
. Solving this issue required simply rebuilding and pushing the images. This incident was tracked in gitlab-com/gl-infra/production#723 (closed).
I am opening this issue to see if there is anything we can do to prevent such empty manifests from getting created if a similar GCS outage occurs so that users don't end up in this state.