Generate backup manifest files
Since manifest files essentially mean that the backup layout is not important, we can put them in their own directory tree. That way, it wont affect existing layouts and when we later go to query for what backups are available in object-storage, we only need to query manifest files and no other backup files.
Proposed manifest file layout:
<object_storage_root>/
manifests/
<storage_name>/
<relative_path>/
<manifest_backup_id>.toml|yaml|json
<manifest_backup_id>.toml|yaml|json
will be the serialised contents of the Backup
struct:
// Backup represents all the information needed to restore a backup for a repository
type Backup struct {
// Steps are the ordered list of steps required to restore this backup
Steps []Step
// ObjectFormat is the name of the object hash used by the repository.
ObjectFormat string
}
// Step represents an incremental step that makes up a complete backup for a repository
type Step struct {
// BundlePath is the path of the bundle
BundlePath string
// RefPath is the path of the ref file
RefPath string
// PreviousRefPath is the path of the previous ref file
PreviousRefPath string
// CustomHooksPath is the path of the custom hooks archive
CustomHooksPath string
}
Right now the backup ID given to pointer layout is the backup ID of the full backup. This wont work for manifest files because it will force us to overwrite existing manifest files. Instead introduce a new "manifest backup id". This will allow us to temporarily accept both backup IDs for smooth transition.
If there is no file written, then the corresponding path, e.g. BundlePath
, should be an empty string. This will mean that on restore we wont have to query object-storage only to find that files don't exist. It is important for WORM that what exists in the manifest file is exactly what we use for restore.