Fix skyfile fanout encoding

MERGE REQUEST

MR Guidelines

Overview

There was a bug in the design of the skyfile upload code and how the fanout bytes were encoded for the basesector.

The fanout encoding only checked for empty hashes for the piece roots for the special case of unencrypted 1-of-N skyfiles where only 1 piece root was needed. This made it possible for encrypted skyfiles to have encoded fanouts that contained empty hashes for the piece roots. While this bug would also affects N-of-M files, since N-of-M skyfiles are not supported yet, no existing files would be impacted.

Since the empty piece roots come from the fact that not all the parity pieces had been uploaded, the solution is to either wait for the upload to be fully complete or to manually create the piece roots from the data source. Since blocking until the upload is 100% complete is not a good UX for the user, this MR implements the second solution of creating the fanout bytes from the data source.

This MR does the following:

1. Adds some protective build.Criticals with unit testing to ensure we do not make future changes that would allow for empty hashes getting into the fanout.
2. Adds a regression test in the siatest package.
3. Updates the SkyfileUploadReader to have the underlying reader either be, or be created from, an io.TeeReader. This allows for adding a fanoutReader that can be used to generate the encoded fanout.

Example for Visual Changes

N/A

Checklist

Review and complete the checklist to ensure that the MR is complete before assigned to an approver.

• All new methods or updated methods have clear docstrings
• Testing added or updated for new methods
• Any new packages are added to Makefile and .gitlab-ci.yml
• API documentation updated for API updates
• Module README.md updated for changes to workflow
• Issue added to Sia-UI repo for new supporting features
• Changelog File Created

Issues Closed