Synchronize access to tables and fields during an export/duplication/snapshot
The duplicate application job read all the data in a REPEATABLE READ
transaction isolation level and the snapshots feature will use the same logic.
This isolation level permits to ignore all the changes committed after the beginning of the job transaction, but do not prevent inconsistencies in the case of any non-MVCC friendly commands like ALTER_TABLE
.
To guarantee that a time-expensive operation like an export/duplication/snapshot works as expected, we could create an advisory lock that need to be taken exclusively from the long running job
or shared between all the requests that requires to alter tables.
The idea is:
-
every
ALTER_TABLE
requests try to acquire a shared advisory lock withwait=False
. This because if someone has taken it exclusively, it could be a long running task and could requires a long time to release it, so it's better to just return an error to the user. -
every long-running job try to acquire the advisory lock in an exclusive way, waiting for it if needed.
NOTES: how to handle concurrency between multiple long-running jobs?