Cannot forcibly push a "lost" LFS object
Summary
It appears to be impossible to forcible re-push a Git LFS object that has been "lost" from the oject store
Steps to reproduce
On a self-hosted gitlab instance installed from the helm chart on GKE:
create new project: "lfs-fail"
clone new project: git clone git@gitlab.example.com:user/lfs-fail
cd lfs-fail
git lfs install
git lfs track \*.gif
git add -A
git ci -m 'track GIFs with LFS'
curl -sL -o picard.gif https://www.reactiongifs.com/r/facepalm.gif
git add -A
git ci -m 'add picard'
git push origin
simulate a bucket failure by manually deleting the object from the bucket.
visit project page and click on "picard.gif", notice the broken image icon and the image not loading
$ git lfs push --all origin
Uploading LFS objects: 100% (1/1), 1.8 MB | 0 B/s, done.
observe, that the object is not uploaded to the bucket.
$ git lfs ls-files -l
2266d502078d53e11986afd92d36a5bca117cc9df1b42e8ce1b89d642a3836e2 * picard.gif
$ git lfs push --object-id origin 2266d502078d53e11986afd92d36a5bca117cc9df1b42e8ce1b89d642a3836e2
Uploading LFS objects: 100% (1/1), 1.8 MB | 0 B/s, done.
it is still not uploaded to the bucket.
What is the current bug behavior?
Gitlab refuses to re-upload the forcibly pushed objects.
From the symptoms it looks like gitlab is telling the client that the objects are already uploaded without actually verifying the presence in the object store.
What is the expected correct behavior?
Gitlab accepting the missing objects.
Output of checks
Gitlab v12.10.3-ee, helm chart v3.3.3 on GKE with object storage in GCS buckets.