Fix critical in UpdateMetadata

Christopher Schinnerlrequested to merge
chris/update-metadata-critical into master

MERGE REQUEST

MR Guidelines

Overview

This MR fixes a critical that frequently appears on portals. It seems to happen within UpdateMetadata on siafiles when the host pubkeytable is pruned.

Commit c3b6896c1 contains the failing regression test which reproduces the problem.

[cschinnerl:.../gitlab.com/SkynetLabs/skyd]$ make test-long pkgs=./skymodules/renter/filesystem/siafile run=TestUpdateMetadataPrune                                                               (c3b6896c1) 
rm -rf cover doc/whitepaper.aux doc/whitepaper.log doc/whitepaper.pdf docker fullcover release
gofmt -s -l -w ./skymodules/renter/filesystem/siafile
go vet ./skymodules/renter/filesystem/siafile
git ls-files "*.md" :\!:"vendor/**" | xargs codespell 
golangci-lint run -c .golangci.yml ./...
analyze -lockcheck=false -- ./skymodules/renter/filesystem/siafile
analyze -lockcheck -- ./benchmark ./build ./cmd/skyc ./cmd/skyd ./cmd/skynet-benchmark ./compatibility ./fixtures ./node ./node/api ./node/api/client ./node/api/server ./profile ./siatest ./siatest/accounting ./siatest/daemon ./siatest/dependencies ./siatest/renter ./siatest/renter/contractor ./siatest/renter/hostdb ./siatest/renterhost ./siatest/skynet ./skykey ./skymodules ./skymodules/accounting ./skymodules/renter/filesystem ./skymodules/renter/filesystem/siadir ./skymodules/renter/filesystem/siafile ./skymodules/renter/hostdb ./skymodules/renter/hostdb/hosttree ./skymodules/renter/skynetblocklist ./skymodules/renter/skynetportals 
GORACE='history_size=3 halt_on_error=1 atexit_sleep_ms=2000' go test -race --coverprofile='./cover/cover.out' -v -failfast -tags='testing debug netgo' -timeout=3600s ./skymodules/renter/filesystem/siafile -run=TestUpdateMetadataPrune -count=1
=== RUN   TestUpdateMetadataPruneHosts
=== PAUSE TestUpdateMetadataPruneHosts
=== CONT  TestUpdateMetadataPruneHosts
--- FAIL: TestUpdateMetadataPruneHosts (0.15s)
panic: Critical error: (unknown v?.?.?, Release: testing) piece.HostTableOffset 4  >= len(sf.pubKeyTable) 3 false
Please submit a bug report here: https://gitlab.com/NebulousLabs/Sia/issues
 [recovered]
        panic: Critical error: (unknown v?.?.?, Release: testing) piece.HostTableOffset 4  >= len(sf.pubKeyTable) 3 false
Please submit a bug report here: https://gitlab.com/NebulousLabs/Sia/issues


goroutine 19 [running]:
testing.tRunner.func1.2(0x14779c0, 0xc0000aa180)
        /usr/local/go/src/testing/testing.go:1144 +0x49f
testing.tRunner.func1(0xc000102900)
        /usr/local/go/src/testing/testing.go:1147 +0x695
panic(0x14779c0, 0xc0000aa180)
        /usr/local/go/src/runtime/panic.go:971 +0x499
gitlab.com/SkynetLabs/skyd/build.Critical(0xc000386578, 0x5, 0x5)
        /Users/cschinnerl/go/src/gitlab.com/SkynetLabs/skyd/build/critical.go:24 +0x16b
gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile.(*SiaFile).hostKey(0xc000184d80, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /Users/cschinnerl/go/src/gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile/siafile.go:1116 +0x205
gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile.(*SiaFile).goodPieces(0xc000184d80, 0x0, 0x0, 0x0, 0xc0000e2e40, 0x2, 0x2, 0x0, 0xc000387c08, 0xc000387c08, ...)
        /Users/cschinnerl/go/src/gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile/siafile.go:1204 +0x116
gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile.(*SiaFile).redundancy.func2(0x0, 0x0, 0x0, 0xc0000e2e40, 0x2, 0x2, 0x0, 0xc0000e2e40, 0x2)
        /Users/cschinnerl/go/src/gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile/siafile.go:834 +0x176
gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile.(*SiaFile).iterateChunksReadonly(0xc000184d80, 0xc000386a70, 0x0, 0x0)
        /Users/cschinnerl/go/src/gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile/persist.go:480 +0x4fc
gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile.(*SiaFile).redundancy(0xc000184d80, 0xc000387c08, 0xc000387c08, 0x0, 0x0, 0x0, 0x0)
        /Users/cschinnerl/go/src/gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile/siafile.go:831 +0x3e5
gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile.(*SiaFile).updateMetadata(0xc000184d80, 0xc000387c08, 0xc000387c08, 0xc000387bd8, 0xc0001abce0, 0x3, 0x3, 0x0, 0x0)
        /Users/cschinnerl/go/src/gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile/siafile.go:997 +0x3d0
gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile.(*SiaFile).UpdateMetadata(0xc000184d80, 0xc0001abc08, 0xc0001abc08, 0xc0001abbd8, 0xc0001abce0, 0x3, 0x3, 0x0, 0x0)
        /Users/cschinnerl/go/src/gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile/siafile.go:970 +0x127
gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile.TestUpdateMetadataPruneHosts(0xc000102900)
        /Users/cschinnerl/go/src/gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile/siafile_test.go:1617 +0x17f2
testing.tRunner(0xc000102900, 0x1507318)
        /usr/local/go/src/testing/testing.go:1194 +0x203
created by testing.(*T).Run
        /usr/local/go/src/testing/testing.go:1239 +0x5d8
FAIL    gitlab.com/SkynetLabs/skyd/skymodules/renter/filesystem/siafile 0.779s
FAIL
make: *** [test-long] Error 1

Checklist

Review and complete the checklist to ensure that the MR is complete before assigned to an approver.

  • All new methods or updated methods have clear docstrings
  • Testing added or updated for new methods
  • Any new packages are added to Makefile and .gitlab-ci.yml
  • API documentation updated for API updates
  • Module README.md updated for changes to workflow
  • Changelog File Created

Issues Closed

Edited by Christopher Schinnerl

Merge request reports