Move WAL state from the repository to an external directory
TransactionManager currently stores some write-ahead log state in the repository in the wal
subdirectory. We need to move this state out of the repository for two reasons:
- The WAL state should exist independently of the repository. Deleting the repository from the disk shouldn't be tied to deleting the write-ahead log's state.
- We'll soon have to support multiple repositories in a partition for object pools. The WAL state is shared between the partition and shouldn't exist in a single repository that may be deleted.
Let's move the state from the repo to <storage>/partitions/<ab>/<cd>/<partition_id>/wal
.
ab
and cd
are the first four hex digits from the hash of the partitions id similar to hashed storage. This serves to keep the number of directory entries manageable regardless of the number of partitions in the storage.
Initially for now, the partition ID can be the relative path of the repository with the directory separators removed.
In future, we'll have to support assigning multiple repositories to a given partition. At that point, the partition ID has to be decoupled from the actual repository path. We'll figure out the details at the later point but we'll likely use an uint64
for this with a lookup table that associates a given relative path with a partition.