portmod breaks in-progress downloads when squelching spaces
To avoid the issue of spaces in download files (e.g. from NexusMods), which break the space-separated language used for specifying sources, we replace all spaces in files in the download cache with underscores. The issue is that if you're in the middle of downloading a file, this renaming will break the download, resulting in an incomplete download and a corrupt file.
There are two options that occur to me for handling this.
The first would be to use percent encoding, encoding spaces as %20
. I don't really like this option, as while it would be fine if, say, copying a download URL, the fact is that most filenames with spaces come from NexusMods, and with the current system we need to preserve the original filename (more or less) to get manual downloads to be recognized. NexusMods, unhelpful as they are, do not supply download links (or at least, ones that can be used by everyone), so if we were to use percent encoding we would need to manually replace spaces with %20
when copying the archive names including them in the pybuild, just like we currently have to replace them with underscores, except it's three times the characters.
The second option, which I like more, would be to create an import directory adjacent to the download cache. Manually fetched downloads would go in the import directory, and if a download is not found is not found in the download cache, we start hashing files in the import directory until we find it, then move it into the download directory with the correct name. Partially downloaded files would obviously not have the correct hash, so they would be ignored, and it would remove the need to preserve the names of files that have to be manually fetched.
There are two issues with the second option though.
- This could become quite slow if many files start to accumulate in the import cache. Of course, this will only slow down manual downloads, and this could be mitigated somewhat by warning the user if more than a certain number of files are in the import cache (files shouldn't be there longterm).
- This adds an extra step to creating pybuilds for mods that require manual downloads, as you would have to manually rename the file the first time. Importmod could help with this though, as we could have it rename the source files as part of the import procedure. Unfortunately importmod doesn't currently work for updating existing pybuilds, so whenever a mod updates you would still need to manually rename the source file (until importmod#2 (closed) is completed).