Put AutoVacuum DB health check behind a feature flag
After enabling the :responsive_throttling
feature for placeholder user reassignment, customers reported that after accepting a reassignment, user contribution records took several hours to be successfully reassigned.
The kibana logs showed repeated instances of reassignment rescheduling:
Import::ReassignPlaceholderUserRecordsService::DatabaseHealthError. Rescheduling reassignment
E.g. The reassignment process for one user took 281 minutes (4.61 hours) to finish due to the throttling (exported log).
The rescheduling was occurring frequently as autovaccum was running in the notes, merge requests and events tables (exported log):
The :responsive_throttling
feature flag was set to false
and the reassignments completed soon after.
Following discussion below, we've decided to put the AutoVacuum
health check, which is occurring on a table-by-table basis, behind a feature flag that is false
by default. This will allow us to have some code in place should we ever encounter an issue with reassignments due to a table being AutoVacuumed. We would be able to turn the AutoVacuum health check on, preventing writes to whichever table was effected, while allowing all other record reassignments to proceed.