Skip to content

Increase pre-allocated random bytes for storage driver tests

João Pereira requested to merge fix-storage-driver-tests-panic into release/2.7-gitlab

Context

Two panic errors on the S3 storage driver test suite were reported in #1 (closed).

To reproduce the issue run:

go test -v github.com/docker/distribution/registry/storage/driver/s3-aws -args -check.v -check.b -check.f "DriverSuite.*1GBFiles"
----------------------------------------------------------------------
PANIC: /Users/jpereira/go/src/github.com/docker/distribution/registry/storage/driver/testsuites/testsuites.go:1035: DriverSuite.BenchmarkPutGet1GBFiles

... Panic: runtime error: slice bounds out of range (PC=0x102F0D4)

/usr/local/Cellar/go@1.12/1.12.13/libexec/src/runtime/panic.go:522
  in gopanic
/usr/local/Cellar/go@1.12/1.12.13/libexec/src/runtime/panic.go:54
  in panicslice
/Users/jpereira/go/src/github.com/docker/distribution/registry/storage/driver/testsuites/testsuites.go:1357
  in randomContents
/Users/jpereira/go/src/github.com/docker/distribution/registry/storage/driver/testsuites/testsuites.go:1049
  in DriverSuite.benchmarkPutGetFiles
/Users/jpereira/go/src/github.com/docker/distribution/registry/storage/driver/testsuites/testsuites.go:1036
  in DriverSuite.BenchmarkPutGet1GBFiles
/usr/local/Cellar/go@1.12/1.12.13/libexec/src/reflect/value.go:308
  in Value.Call
/usr/local/Cellar/go@1.12/1.12.13/libexec/src/runtime/asm_amd64.s:1337
  in goexit

----------------------------------------------------------------------
…

----------------------------------------------------------------------
PANIC: /Users/jpereira/go/src/github.com/docker/distribution/registry/storage/driver/testsuites/testsuites.go:1077: DriverSuite.BenchmarkStream1GBFiles

... Panic: runtime error: slice bounds out of range (PC=0x102F0D4)

/usr/local/Cellar/go@1.12/1.12.13/libexec/src/runtime/panic.go:522
  in gopanic
/usr/local/Cellar/go@1.12/1.12.13/libexec/src/runtime/panic.go:54
  in panicslice
/Users/jpereira/go/src/github.com/docker/distribution/registry/storage/driver/testsuites/testsuites.go:1357
  in randomContents
/Users/jpereira/go/src/github.com/docker/distribution/registry/storage/driver/testsuites/testsuites.go:1107
  in DriverSuite.benchmarkStreamFiles
/Users/jpereira/go/src/github.com/docker/distribution/registry/storage/driver/testsuites/testsuites.go:1078
  in DriverSuite.BenchmarkStream1GBFiles
/usr/local/Cellar/go@1.12/1.12.13/libexec/src/reflect/value.go:308
  in Value.Call
/usr/local/Cellar/go@1.12/1.12.13/libexec/src/runtime/asm_amd64.s:1337
  in goexit

Problem

We are trying to allocate 1024*1024*1024 bytes for 1GB file benchmarks while the pre-allocated random bytes are just 128<<20 in capacity.

Solution

We need to bump the pre-allocated random bytes capacity to 128<<23 in order to cope with the 1GB test requirements.

Although the integration tests fail (there is still one non-panic error to be solved, reported in #1 (closed)), the previously failing DriverSuite.BenchmarkPutGet1GBFiles and DriverSuite.BenchmarkStream1GBFiles tests are fixed with this change.

Edited by João Pereira

Merge request reports