feat: Integration with persy using background ops

Hi,

This is a rework of the Integration with Persy and replace !107 (closed), depending on the last improvements in the master branch of Persy regarding background fsync operations, that got quite faster and more stable, this allowed me to simplify a lot the integration removing the need of a write cache, now the operation when possible are 1 to 1 to Persy operations, in terms of durability of this integration is the same of Normal of sqlite (means not data loss on program crash, potential loss of recent transactions on OS crash).

Regarding performances I ran the rjbench 15 times against both Persy and Sqlite here a short version of the results:

Persy

1st run:
real	0m19.621s
user	0m0.407s
sys	0m0.112s

15th run:
real	0m30.735s
user	0m0.497s
sys	0m0.131s

Sqlite

1st run:
real	0m21.644s
user	0m0.315s
sys	0m0.190s

15th run:
real	0m25.904s
user	0m0.399s
sys	0m0.146s

From this number the performance are comparable, Persy seems a bit slower on the long run but does a bit more early cleaning of journal that do not produce big pause, as well I'm in continue toning of Persy right now.

In attachment the full log of the run

persy_run.log sqlite_run.log

I also ran the rjbench against persy 100 times just to see the speed drop with data grow, this is the cleaned log of the 100 runs : speed_100.log

I checked the database size after this runs and was around 750 Mbs the RAM used at the end was ~ 1.5GB with the database cache configured at 250MB

In terms of release of the relative Persy improvements I'm expecting to release them around January with the 1.2 release.

That's all for now.


  • I agree to release my code and all other changes of this MR under the Apache-2.0 license

Merge request reports

Loading