Backup applied log entries
The WAL provides us a deterministic way to restore a repository to a known state. To use this for disaster recovery, we need to store the applied entries externally before they are pruned.
Add a new LogEntryArchiver
which will be notified of newly available
entries, tar
the entry directory, and write it to a backup.Sink
,
which may be either a local directory or object storage. It has a
configurable number of worker goroutines to perform the backups.
Failures will typically share a common cause, such as connectivity issues with object storage or overtaxed system resources. The archiver will wait with an exponential backoff for subsequent failures before continuing on. There is no limit to the number of retries for an entry.
Each log entry is received from the LogManager
and acknowledged to it
in sequence, but a given entry may not be backed up in that order due to
retries or goroutine scheduling. In this case, the entry will be stored
by the archiver in a backlog until its LSN is reached, at which point
it will be acknowledged.
Should a successfully backed up entry be resubmitted to the
LogEntryArchiver
, for example if the TransactionManager
shutdown
before reading the AcknowledgeTransaction
response, the archiver will
acknowledge the transaction a second time without creating a new backup.
Entries are stored by <PARTITION_ID>/, which allows access to the entries of a given repository or fork cluster based on their partitionID.