Speed up skynet ls

Matthew Seveyrequested to merge
sevey/speed-up-skynet-ls into master

MERGE REQUEST

MR Guidelines

Overview

This MR aims to make skyc skynet ls usable :-)

  1. Created some helper functions to refactor skyc renter ls so that it was easier to compare skyc renter ls to skyc skynet ls. Additionally there was some overlap so this DRY'd things up a bit.
  2. Update logic in skyc skynet ls to utilize the skynet stats in the directory metadata.
  3. Refactor printing of skynet directories and files into helper functions.

skyc skynet ls should now be just as fast as skyc renter ls. What makes both calls slow is when getDir is called in a recursive way. This is because it does a synchronous API call for each sub directory. So on our portals that is 65k blocking API calls.

I will do a follow up that tries to speed that up a bit.

Example for Visual Changes

https://asciinema.org/a/cI41tcypPE96cJ79Ib9ud2eEB

Checklist

Review and complete the checklist to ensure that the MR is complete before assigned to an approver.

  • All new methods or updated methods have clear docstrings
  • Testing added or updated for new methods
  • Any new packages are added to Makefile and .gitlab-ci.yml
  • API documentation updated for API updates
  • Module README.md updated for changes to workflow
  • Issue added to Sia-UI repo for new supporting features
  • Changelog File Created

Issues Closed

Edited by Matthew Sevey

Merge request reports