refactor and fix db supervision in multi store API
this tickets supersedes #3 (closed), #6 (closed) & #7 (closed) .
goals
-
make sure that an error in the transaction don't crash the database process without closing the database handler #3 (closed) -
on database deletion, make sure to close the process correctly -
if the database has not been closed (happen during a VM crash) then repair it on startup if possible -
distinct db not_found
errors from docnot_found
error
implementation
current
[barrel_store_sup]. [barrel_dbs_sup]
| |
| |
| |
| ---------------------------
| | |
barrel_store --(m)-- barrel_db barrel_db ...
The current implementation is the following:
- databases are created via the
barrel_store
process that serialize the operations and maintain the configuration. - on creattion a database process is created. It is supervised by
barrel_dbs_sup
and monitored by barrel_store - once created the database process is registered in an ETS table owned by the
barrel_store
process - the
barrel_db
process maintains the nif resource to rocksdb - all write operations are done in the
barrel_db
process - reads are done by fetching the NIF resource in ETS.
- the barrel_store process monitor the database
- dbs are never closed: they can only be created or deleted
this design has the following pros/cons
Pros:
- simple
- read are distinct from write
Cons:
- no garaty on the perfomance promises: writes can be a bottleneck or too much read processes can co-exists
- if something happen on write it can crash
- barrel_store is not a supervisor
new implementation
The new implementation will give the following changes:
- maintain then if resource in its own process independently from the read and writes process. It will makes things easier when it's about to close the db
- operation in multi store should be asynchrony to allow concurrent creation/deletion
- instead of using an ETS we will put everything in processes. Writes and Reads process will be distinct and can be scaled depending on the performance promise
- on restart a database can be repaired.
- add a maintenance mode for a db to allows any background operation. During this operation a database can be reopened.
Edited by benoît chesneau