Skip to content

feat: automated data retrieval - physical and logical restoring (#93)

Artyom Kartasov requested to merge 93-data-retrieval into master

Description

  • design data retrieval process
  • define the main stages and jobs
  • implement initialization jobs (physical and logical restoring)

Related issue

#93 (closed)

Examples

...

retrieval:
  stages:
    - initialize
    - snapshot

  spec:
    initialize:
      jobs:
#        - name: logical-restore
#          options:
#            containerName: retriever_logical_restore
#            dumpFile: /tmp/db.dump
#            forceInit: false
#            dbName: test
#            partial:
#              tables:
#                - test
        - name: physical-restore
          options:
            containerName: retriever_physical_restore
            tool: walg
            dockerImage: "postgresai/sync-instance:12"
            envs:
              WALG_GS_PREFIX: "gs://{BUCKET}/{SCOPE}"
            walg:
              storage: gcs
              backupName: LATEST
              credentialsFile: /tmp/sa.json # optional

    snapshot:
      jobs:
        - name: masking
        - name: snapshotting
          options:
            schedule: "0 */6 * * *"
            retention:
              maxSnapshotCount: 4

Checklist

  • the MR description has been reviewed
  • this MR contains text changes and they have been reviewed OR there are no texts changes
  • this MR contains GUI/CLI changes and they have been reviewed OR there are no GUI/CLI changes
  • this MR contains API changes, specifications reflect those changes and they have been reviewed OR there are no API changes

Closes #93 (closed)

Edited by Artyom Kartasov

Merge request reports