Prune web hook logs older than one month
What does this MR do?
This adds a Sidekiq worker that removes up to 50 000 web hook logs that are older than one month, per hour.
This worker works best with an index on web_hook_logs.created_at
. Since this index is already being added as part of https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20247 I have skipped adding one in this MR.
Why was this MR needed?
The web_hook_logs
table currently grows indefinitely, and on GitLab.com already takes up 34 GB of space. By removing web hook logs older than one month we can remove an estimated 20 GB of data.
Screenshots (if relevant)
Does this MR meet the acceptance criteria?
-
Changelog entry added, if necessary -
Documentation created/updated -
Tests added for this feature/bug - Conform by the code review guidelines
-
Has been reviewed by a Backend maintainer -
Has been reviewed by a Database specialist
-
-
Conform by the merge request performance guides -
Conform by the style guides -
If you have multiple commits, please combine them into a few logically organized commits by squashing them -
End-to-end tests pass ( package-and-qa
manual pipeline job)