Redesign the Retrieval service

Goal

Redesign the Retrieval service to allow provisioning operations in DLE API

TODO / How to implement

Retrieval interface

Retrieval should have the following public methods:

Setup()
Run()
Reload()
Stop()
ValidateConfig()
RefreshData()
SnapshotData()
CollectRestoreTelemetry() / ReportState()

Parsing jobs

Retrieval jobs can be one of two types:

  • refreshing job (logicalDump, logicalRestore, physicalRestore)
  • snapshotting job (logicalSnapshot, physicalSnapshot)

RefreshData() must contain and run all jobs to get the freshest version of data. SnapshotData() must contain and run all jobs to take a snapshot from the current version of data.

  • modify parseJobs to define a type of retrieval jobs
  • distribute and remember jobs according to their type: refreshing job or snapshotting job
  • a set of jobs must be rearranged on config reloading
  • an order of job items matters

Jobs validation

Extend the ValidateConfig() method to validate for more checks. Make check more detailed and specific - #376

  • ValidateConfig() should validate all available fields.
  • ValidateConfig() should verify specified access parameters and check connections

Running

Run() combine calls of all available jobs using their methods: RefreshData() and SnapshotData()

  • make sure the scheduler is up and running after changes
  • consider that RefreshData() and SnapshotData() can be running outside of the Retrieval service

State

Provide an easy way to get the current state

  • rename method CollectRestoreTelemetry() to ReportState()

Acceptance criteria

Edited by Artyom Kartasov