Create transition log models and tables for BBO monitoring

Leonardo da Rosarequested to merge
562267/bbo-monitoring/tables-and-models into master

What does this MR do and why?

Create transition log models and tables for BBO monitoring.

Add database tables and models to track BBO state transitions at both operation and job levels.

Tables created:

  • background_operation_transition_logs
  • background_operation_transition_cell_local

Each table:

  • Partitioned by RANGE(created_at) with monthly partitions
  • 1-month retention period for automatic cleanup - Depending on the amount of data, we can lower it to 15 days
  • Tracks state transitions with exception details
  • Uses loose foreign keys for cell boundary isolation (this causes keeping track of deleted records for background_operation_jobs and background_operation_jobs_cell_local tables)

There's not much discussion around BBO logging, only that we'll test the same approach from BBM logging. https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/batched_background_operations/#batched_background_operation_transition_logs

References

#562267 (closed)

Edited by Leonardo da Rosa

Merge request reports