Add file deduplication to build steps
There's duplicates.py
in the data dir, but it's only run manually. This should be part of the build, just before updating the count and committing to Git.
There's duplicates.py
in the data dir, but it's only run manually. This should be part of the build, just before updating the count and committing to Git.