Closed
Milestone
May 15, 2024–May 23, 2024
Data-Availability Layer (DAL): GC skip list cells
Context
The DAL node stores skip list cells (used in case of a refutation game) for each level. However they are never removed which leads, in the medium/long term, for unacceptably high data storage (in terms of overall space used and number of files used by the store).
The shards and slots for observer nodes are kept for about 3 months, precisely for use in refutation games. We need to do the same here.
Here the number of files and their total size, assuming 8KiB per file (we have 6208 bytes for a cell file, and 5248 bytes for a hash file) and 33 files (32 cell files, and 1 hash file) per level:
Block time | Files per Day | Size per Day (GiB) | Files per 90 Days | Size per 90 Days (GiB) |
---|---|---|---|---|
8 sec (Q) | 356,400 | 2.714 | 32,076,000 | 244.26 |
10 sec (Paris) | 288,000 | 2.197 | 25,920,000 | 197.73 |
Outcomes of the project
- a DAL node GC-ing non-relevant skip list cells
Participants
Work breakdown
-
quantify the problem (how much data and how many files are stored per day or month) -
recall and confirm from which level GC cells are not needed anymore -
GC irrelevant skip list cells (!13331 (merged))