New Renter File Management
FEATURE REQUEST
Description of Request
Currently the renter's files
field is a map of *siafiles
. If we change this to a struct that contains a *siafile
then we can do a lot of interesting things. If we add a counter to the struct we can keep track of threads that are using the siafile
and remove them when not in use to reduce memory usage. We can also use this struct to do lazy loading which would improve start up time and reduce memory usage.
Reason or Need for Feature
The repair by folder changes introduce a new locking case where we need to be thread safe between the siafiles in memory and the siafiles being read from disk, as well as being thread safe for the sia directory metadata files that are being read from disk.
Design / Proposal
The proposal is to implement a locking structure like the following
Renter {
files siaFileSet
}
type (
siaFileSet struct {
mu
fileMap map[string]*siaFileSetEntry
}
siaFileSetEntry struct {
counter int
file *siafile.SiaFile // No need to have a lock since the SiaFile already handles locking.
}
)
func (s *siaFileSet) Get(key string) *siafile.SiaFile {
// TODO implement
}
func (s *siaFileSet) Insert(sf *siafile.SiaFile) {
// TODO implement
}
func (s *siaFileSet) Return(sf *siafile.SiaFile) {
// TODO implement
}
func (s *siaFileSet) Move(oldKey, newKey string) {
// TODO implement
}
The initial implementation would be to just move the *siafile
into the struct. Then future merge requests can add fields and interface methods to improve functionality as needed.