Skip to content

Update/refactor for performance

pcseanmckayrequested to merge
pcseanmckay/pypi-downloader:update/refactor-for-performance into master

This refactor is designed to maximize use of threading at all times to speed up performance. As a demonstration of the performance gain, here are some stats that were compiled using the refactor versus previous versions:

Note these were executed while downloading the following packages: mxnet-cu80 lalsuite frida tensorflow-io-nightly tensorflow-gpu cupy-cuda91 cupy-cuda90

  1. Using current version with parallel utility with a thread count of 1: 462m47
  2. Using current version with a thread count of 15: 401m4
  3. Using this refactor with a thread count of 15: 326m7

Merge request reports