Add missing tests coverage to the s3-aws driver
We are currently missing tests coverage for the following cases:
-
parameter parsing: -
chunksize -
multipartcopychunksize -
multipartcopymaxconcurrency -
multipartcopythresholdsize -
maxrequestspersecond -
maxretries -
object ACL parameter should not be set when object ownership is enabled
-
-
getParameterAsInt64: -
conversion from string, int64, int, []byte, -
min and max
-
-
Writer: -
s3Path(path) != *multi.Key - existing multi-part upload?this is a bigger issue, we should re-think whether we should be resuming multipart uploads at all.
-
-
Stat: -
fi.IsDir = true- never reached as we do not set Delimiter field
-
-
Move: -
lastByte = fileInfo.Size() - 1 - adjust lastbyte size
-
-
URLFor -
method != GET and method != HEAD (Post?)
-
-
Walk is not covered at all -
WaltPararell: -
fallback to Walk because parallel is not supported -
path == "" -
empty bucket
-
-
Writer.Write: -
old file is bigger than minChunkSize
-
-
during multipart upload, if the last part is smaller than minpartsize, the file is re-uploaded, but what if the file is larger than 5GiB (i.e. max part size)? -
during flush, the pending and ready parts both have w.driver.ChunkSize size, and we do Commit() plus Close() - I check if here is going to be a data corruption here -
use multi-error in copy() func for pararell multi-part upload. -
resp, err := d.S3.ListObjectsV2WithContext(ctx, listObjectsV2Input)in the Delete() call - it should not treat error equal toresp.Contents == 0 -
Stat call and all other that use ListObjectsV2Input should be precise when checking the output of the list as this only does a prefix match, and it should be matching on up to delimiter for the paths received. I have already added DirPartialPrefix test for the Stat() call -
chunk size should be smaller than 2x chunk size because of how flushPart() method works - if if int64(len(w.pendingPart)) < w.driver.ChunkSize {to wtedy pending i ready parts are concatenated which could result in double the size (well, almost)
Edited by Pawel Rozlach