Skip to content

Partition manager identifies partitions to detach

Simon Tomlinson requested to merge stomlinson/identify-partitions-to-drop into master

What does this MR do?

Adds an option to the monthly partitioning strategy so that we can specify a retention period:

partiton_by :monthly, retain_for: 3.months

We plan to eventually detach and then drop partitions that are older than the retention period, but for now this MR simply identifies the partitions that should be detached and logs them so that we can observe the behavior in prod and verify that it's selecting the correct partitions.

While the code to actually drop partitions is not included in this MR, there's a risk that a logic bug could lead to not creating the correct partitions for future months when the PartitionCreationWorker runs. To mitigate this, the feature flag partition_pruning_dry_run will disable the new code if we discover a logic issue.

So that this change can be observed in production, it is added to the web_hook_logs table (which will be pruned in #332200 (closed)).

Screenshots (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

If there is a logic error and we don't create the appropriate partitions in the future, there could be risk to inserting new rows in partitioned tables. The partition creator currently creates six months of future partitions, so we will be able to safely toggle the partition_pruning_dry_run feature flag off if there is a problem.

Edited by Simon Tomlinson

Merge request reports