Support queryPreprocessing in logicalRestore + allow to use inline SQL in queryPreprocessing
Goal
During database restore, an error occurred when trying to create a policy because there is no necessary user in the database.
ERROR: role "role1" does not exist
As a workaround, support queryPreprocessing to allow creating the Postgres users before data restore.
TODO / How to implement
Support and extend queryPreprocessing
-
run
queryPreprocessing
after container readiness and Postgres starts in thelogicalRestore
job- if
queryPath
orinline
is not empty. If both fields are empty - skip - use
queryProcessor
from the snapshotting stage:engine/internal/retrieval/engine/postgres/snapshot/queryPreprocessor.go
- can be separated as a standalone package
-
inline
queries run after scripts placed inqueryPath
- store inline queries to a temporary file and run it with the query processor
- support multi-line queries
- if
-
allow inline SQL (single-threaded) for all preprocessing script fields:
- add
inline
to snapshot jobs (logicalSnapshot
,physicalSnapshot
) as well
- add
retrieval:
spec:
logicalRestore:
options:
...
queryPreprocessing:
# Path to SQL pre-processing queries.
queryPath: ""
# Worker limit for parallel queries.
maxParallelWorkers: 2
# Inline SQL.
inline: ""
Notes:
- it does not work with the
immediateRestore
option as it's defined inlogicalDump
Acceptance criteria
The user can run SQL before data restore
Edited by Nikolay Samokhvalov