Skip to content

feat(engine): add customOptions for pg_dump and pg_restore commands (#361)

Artyom Kartasov requested to merge 361-custom-options into master

Description

Support customOptions for logicalDump (pg_dump) and logicalRestore (pg_restore) jobs:

  • add customOptions to logicalDump (including immediateRestore) and logicalRestore jobs
  • support slice in UI Configurator projections
  • add projection fields to manage customOptions via API
  • update default config files for logical mode

Implementation features:

  • Options are supposed to have free values. The user is responsible for the content. For example,
    logicalRestore:
      options:
        customOptions:
          - "--verbose"
          - "--jobs=2"
  • Keep custom config fields (parallelJobs, ...), but consider them deprecated (and then probably remove later) to have a single way of dump/restore customization
  • Custom options are not checked (in terms of validity, duplicates, etc.) before running the command
  • Custom options have a high priority and can override the default options.
  • The UI Configurator doesn't custom options for the restore command in the case of immediateRestore because it doesn't use them and it will take extra effort to detect if the section is enabled

Related issue

#361 (closed)

Examples

GET /admin/config

{
    "databaseConfigs": {
        "configs": {
            "shared_buffers": "256MB",
            "shared_preload_libraries": "pg_stat_statements, auto_explain, pgaudit, logerrors, pg_stat_kcache"
        }
    },
    "databaseContainer": {
        "dockerImage": "postgresai/extended-postgres:14"
    },
    "global": {
        "debug": true
    },
    "retrieval": {
        "refresh": {
            "timetable": "0 * * * *"
        },
        "spec": {
            "logicalDump": {
                "options": {
                    "customOptions": [
                        "--exclude-schema=rdsdms"
                    ],
                    "databases": {},
                    "parallelJobs": 1,
                    "source": {
                        "connection": {
                            "dbname": "test",
                            "host": "172.17.0.2",
                            "port": 5432,
                            "username": "postgres"
                        }
                    }
                }
            },
            "logicalRestore": {
                "options": {
                    "customOptions": [
                        "--no-privileges",
                        "--no-owner",
                        "--exit-on-error"
                    ],
                    "parallelJobs": 1
                }
            }
        }
    }
}

POST /admin/config has the same body, respectively

Checklist

  • MR description has been reviewed
  • MR changes are functionally tested
  • MR does NOT have API/CLI changes OR there are API/CLI changes and they have been reviewed & DOCS ARE ADJUSTED (reference doc, etc)
  • MR does NOT have UI changes OR there are UI changes and they have been reviewed & UX IS REVIEWED

Closes #361 (closed)

Merge request reports