Skip to content

Remove processed replication jobs to prevent table growing rapidly

Pavlo Strokov requested to merge ps-cleanup-queue into master

The replication process continuously creates replication events for majority of the mutator operations. Once event is persisted it remain in the storage despite if it is was processed and what is the latest status of the processing. As there are no removals the storage growths continuously consuming more memory. But more important is that it affects execution of other operations in a bad way.

To deal with it the new background processed is implemented to remove events from the storage. It removes events that were not updated for the past week and the current state for them is dead or completed. It won't remove the latest event of the repository even if it is old enough and has proper state as it is used for other purposes. The removal process is limited to delete at most 100 events in a run to reduce load on the database. And each run happens once/50 sec (to have shift with other jobs).

Closes: #2941 (closed)

Merge request reports