When saving datasets, hard-link files and not copy them
https://docs.python.org/3/library/os.html#os.link
That should then be faster and also consume less space. Because datasets and files should be immutable, we can do that.
Edited by Mitar
https://docs.python.org/3/library/os.html#os.link
That should then be faster and also consume less space. Because datasets and files should be immutable, we can do that.