S3 unfreeze all
This Merge Request tries to provide a way to unfreeze all files in S3 Glacier storage at once to reduce the possible waiting time.
It seems that botos recover() method already is blocking, so I implemented a
'pre_process_download_batch' method that actually calls 'pre_process_download' for
each file to restore in a separate thread.
It is working well for my use case. If there are files to unfreeze from Glacier the unfreezing process is started for all of them. The actual 'pre_process_download' is still blocking until the file it tries to preprocess is available in S3. If all files already are in S3 the restore works normally without additional waiting time.
Due to my poor python knowledge and the unfamiliar codebase this might not be the best solution and I might have done some stupid mistakes, but it is a nicely isolated addition.