Loading
server: negotiate zstd compression on the gRPC services
I saw this was in the client-pathfinding work originally via the demo, but it actually turned out to be completely orthogonal to that branch. However, I still feel it might be good to introduce this compression as it's beneficial to the upcoming recovery feature.
zstd compression savings in VTXO mailbox payloads
Measured on the committed VTXO fixtures (lib/src/test_util/vectors.rs) at
zstd level 3, the default level tonic uses).
Single VTXO
| VTXO | raw | zstd (lvl 3) | saved |
|---|---|---|---|
| board | 298 B | 250 B | ~16% |
| arkoor_htlc_out | 659 B | 567 B | ~14% |
| arkoor2 | 948 B | 802 B | ~15% |
| round1 | 1136 B | 905 B | ~20% |
| round2 | 1174 B | 944 B | ~20% |
| arkoor3 | 1418 B | 1139 B | ~20% |
Batched into one frame
| batch | raw | zstd (lvl 3) | saved |
|---|---|---|---|
| round1 + round2 (siblings) | 2310 B | 1162 B | ~50% |
| all 6 distinct VTXOs | 5633 B | 2304 B | ~59% |
| 100 msgs (50× sibling pair) | 115.5 KB | 1.2 KB | ~99% |
| 100× board (identical) | 29.8 KB | 255 B | ~99% |
Lone VTXOs compress ~15-20%; batched mailbox reads compress ~50-60% because VTXOs from the same tree share large identical prefixes (99% under heavy duplication).
Compute cost for compression/decompression
This MR only introduces decompression for clients which is asymmetrically much cheaper than compression which the server does. I've specifically left client sending compression out of scope as the savings there are minimal and incur somewhat of a noticeable cost for lower compute clients.
I'd like to get some hard numbers here, but in general decode is quite cheap.
Edited by Duncan Dean