Skip to content

Live trace: Use efficient destroy all (for `dependent: :destory` problem)

Shinya Maeda requested to merge live-trace-v2-efficient-destroy-all into master

What does this MR do?

  • CE: Here

  • EE: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/5593

  • This MR is for resolving dependent: destory problem

  • This is based on @ayufan 's idea. We discussed synchrnously on 25th Apr.

  • The idea is 1) Create a closure(Proc.new) to remove all redis keys under the project 2) Schedule the closure with run_after_commit in before_destroy hook 3) Rows of chunks will be deleted by cascading deletion with FK 4) The scheduled closure is invoked and data in Redis keys are deleted

  • This method has been confirmed by @ayufan as it works

  • This way is effective for build.job_artifacts too. https://gitlab.com/gitlab-org/gitlab-ce/issues/35240. We need sidekiq because OS doesn't provide the multiple deletion API.

  • We don't want to leverage the efficiency of destory_all by configuring TTL. TTL should have a single meaning of the live-trace life span.

Are there points in the code the reviewer needs to double check?

Why was this MR needed?

Screenshots (if relevant)

Does this MR meet the acceptance criteria?

  • Tests added for this feature/bug
  • Review
    • Has been reviewed by Backend
    • Has been reviewed by Database
  • End-to-end tests pass (package-and-qa manual pipeline job)

What are the relevant issue numbers?

/cc @ayufan @nick.thomas @yorickpeterse

Edited by Grzegorz Bizon

Merge request reports