Monitor: Adjust storage monitor so that imminent out of storage issues are noticed sooner
User Story
As a poor fellow I want Scalyr to notice when an instance is going to run out of storage space before it does so that downtime is avoided.
Background Documents
- First issue was the threshold was mismatched - Scalyr was looking at a set number, ElasticSearch was looking at a percentage (95%). This issue still exists as the threshold has simply been raised for the current default storage size. Ideally we'd match by percentage (scalyr powerquery? custom monitor?)
- Second issue is that we find out about high write operations too late - storage is full before the triggered alert can do any good (weekend, timezones, etc). Perhaps over-monitor test instance for high-rates to find the edge cases before promotion?
Acceptance Criteria
(Describe additional criteria that must be met for the User Story to be complete)
e.g.
Whenever the user clicks cancel or tries to go forward or back in the browser they should be asked to confirm before they lose changes. If the user clicks Save any input should be validated so they can't make a mistake
Definition of Ready
-
User Story can be broken down no further -
Acceptance Criteria and Story Points are agreed upon by development team and PO
Definition of Done
-
If required, UI Mockups are created and reviewed -
Code is written with automated unit tests, reviewed, merged to master branch, all tests pass -
All UI texts are multilingual -
Manual tests in test environment are successfully performed -
Test Plan is updated -
Documentation is updated