feat: automated data retrieval - physical and logical restoring (#93)
Description
- design data retrieval process
- define the main stages and jobs
- implement initialization jobs (physical and logical restoring)
Related issue
Examples
...
retrieval:
stages:
- initialize
- snapshot
spec:
initialize:
jobs:
# - name: logical-restore
# options:
# containerName: retriever_logical_restore
# dumpFile: /tmp/db.dump
# forceInit: false
# dbName: test
# partial:
# tables:
# - test
- name: physical-restore
options:
containerName: retriever_physical_restore
tool: walg
dockerImage: "postgresai/sync-instance:12"
envs:
WALG_GS_PREFIX: "gs://{BUCKET}/{SCOPE}"
walg:
storage: gcs
backupName: LATEST
credentialsFile: /tmp/sa.json # optional
snapshot:
jobs:
- name: masking
- name: snapshotting
options:
schedule: "0 */6 * * *"
retention:
maxSnapshotCount: 4
Checklist
-
the MR description has been reviewed -
this MR contains text changes and they have been reviewed OR there are no texts changes -
this MR contains GUI/CLI changes and they have been reviewed OR there are no GUI/CLI changes -
this MR contains API changes, specifications reflect those changes and they have been reviewed OR there are no API changes
Closes #93 (closed)
Edited by Artyom Kartasov