fix: clean up data and manage runs during full data refresh in logical mode. New config option to skip full refresh during startup time: skipStartRefresh
Description
- Remove all snapshots and PGDATA during a full refresh of data in a logical mode
- Remove
--clean
and--if-exists
flags for thepg_restore
since PGDATA is going to be empty - Remove the
forceInit
options from logical configuration files, add a warning if PGDATA still contains any files or directories. - Add a new
skipStartRefresh
option for a logical mode to skip data refreshing
Will not be implemented:
- db_name for pg_restore because we need to specify an existing database that we can connect to. Otherwise we will get the error:
[CONTAINER dblab_server]: 2023/03/06 17:01:01 restore.go:511: [INFO] Running restore command for test
[pg_restore --username postgres --dbname test --create --jobs 2 /var/lib/test/dblab_mount/test_dblab_pool/dump/test]
pg_restore: [archiver (db)] connection to database "test" failed:
connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: database "test" does not exist
Related issue
Examples
forceInit
deprecated
logicalRestore:
options:
<<: *db_container
dumpLocation: "/var/lib/dblab/dblab_pool/dump"
parallelJobs: 4
ignoreErrors: false
<<: *db_configs
A new skipStartRefresh
option:
retrieval:
refresh:
# Timetable is to be defined in crontab format: https://en.wikipedia.org/wiki/Cron#Overview
timetable: "0 0 * * 1"
# Skip data refresh while the retrieval starts.
skipStartRefresh: false
Checklist
-
MR description has been reviewed -
MR changes are functionally tested -
MR does NOT have API/CLI changes OR there are API/CLI changes and they have been reviewed & DOCS ARE ADJUSTED (reference doc, etc) -
MR does NOT have UI changes OR there are UI changes and they have been reviewed & UX IS REVIEWED
Edited by Nikolay Samokhvalov