Database disallowlist: add a configuration option to exclude the specified database for data retrieval
Goal
At the moment there are no options to exclude certain databases during logical dump, user can only specify off databases to copy.
This is useful when there are dozens (or hundreds) of databases in a single PostgreSQL instance and only one or more databases need to be excluded.
TODO / How to implement
Add a new configuration option (excludeDatabases
) to the section retrieval.spec.logicalDump.options
:
retrieval:
spec:
logicalDump:
options:
# Option for specifying the database list that must be copied.
# By default, DLE dumps and restores all available databases.
# Do not specify the databases section to take all databases.
# databases:
# database1:
# # Option for a partial dump. Do not specify the tables section to dump all available tables.
# tables:
# - table1
# database2:
# databaseN:
# Exclude these databases from copying.
excludeDatabases:
- database3
- database4
- Edit the function
GetDatabaseListQuery
. Add a condition for each database in the "excludeDatabases" list
Example:
select datname
from pg_catalog.pg_database
where
not datistemplate
and has_database_privilege('%s', datname, 'CONNECT')
and datname <> 'database3'
and datname <> 'database4'
-
Add tests (unit and integration) to verify the correctness of custom data retrieving
-
Describe new options in examples of configuration files (https://gitlab.com/postgres-ai/database-lab/-/tree/master/engine/configs) and in the official docs (https://gitlab.com/postgres-ai/docs)
By default, the new options should be opt-out (commented out).
Acceptance criteria
- User can exclude specified databases.
- The rest of the databases are dumped as usual.
- Consider the case when both options(
databases
andexcludeDatabases
) are specified at once - New tests covers all possible cases of retrieving data
- Options are described in the docs
Edited by Vitaliy Kukharik