Skip to content

backup: Refactor Pipeline struct

James Liu requested to merge jliu-track-restored-repos into master

While working through #5357 (closed), I identified a few refactoring opportunities to improve the backup and restore code.

Central to this mechanism is a Pipeline construct that receives jobs streamed via stdin to the gitaly-backup backup|restore commands. Each job defines a repository to be backed up or restored. The pipeline is responsible for queuing these requests, possibly executing requests concurrently, and reporting on errors.

Previously there were two pipeline implementations along with an interface to abstract them; a LoggingPipeline and a ParallelPipeline that adapts it to enable concurrency. Since a parallel pipeline with a single worker is effectively a sequential pipeline, this MR refactors the pipeline code by consolidating both implementations into a single struct. This makes the overall code a little easier to parse by reducing the amount of cross-object interactions. The MR also makes some changes to how the amount of concurrency is enforced within the pipeline.

Edited by James Liu

Merge request reports