Sevey/bubble scheduler
MERGE REQUEST
Overview
This MR adds a new struct bubbleScheduler. The bubbleScheduler manages bubble updates for the renter. The problem that this solves is that with large filesystems, an increasing number of bubbles get called on the top level directories. Since this bubbles are called in a go routine when the child directory bubble update is finished, the node can be overloaded with go routines and these root directories get stuck in a state of bubblePending but bubbles are never executed because of all the bubbles occurring on the child directories.
This branch was tested on the m1 skynet server. As a result of the testing some additional updates and refactoring of the bubble updates were made to reduce bottlenecks in the bubble and health loop code. On the worst maintenance node us-east-upload this branch brought the AggregateLastHealthCheck time from 7 days in the past to 7 hours in the past. Further debugging found that the main bottle neck on the maintenance server now is fsyncs.
Example for Visual Changes
Checklist
Review and complete the checklist to ensure that the MR is complete before assigned to an approver.
-
All new methods or updated methods have clear docstrings -
Testing added or updated for new methods -
Any new packages are added to Makefile and .gitlab-ci.yml -
API documentation updated for API updates -
Module README.md updated for changes to workflow -
Issue added to Sia-UI repo for new supporting features -
Changelog File Created