Skip to content
Snippets Groups Projects

Cleanup leftovers in packages_dependencies table

What does this MR do and why?

MR !20549 (merged) added two tables: packages_dependencies and packages_dependency_links. When a package is deleted, packages_dependency_links rows are properly deleted but rows in packages_dependencies are kept and they can become orphans. This MR aims to clean up those orphaned packages_dependencies records.

How to set up and validate locally

  1. In Rails console, create a bunch of orphaned packages_dependencies records: FactoryBot.create_list(:packages_dependency, 100)
  2. Enable the feature flag : Feature.enable(:packages_delete_orphaned_dependencies_worker)
  3. Run the cleanup sidekiq worker: Packages::Cleanup::DeleteOrphanedDependenciesWorker.perform_async, the orphaned records should be deleted.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Database analysis

  1. !113076 (comment 1297596269)
  2. !113076 (comment 1322050386)

Related to #38266 (closed)

Edited by David Fernandez

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
    • Resolved by David Fernandez

      Thanks @mkhalifa3!

      Can you check how many records from packages_dependencies table we're going to delete when we run the worker for the first time.

      Several database queries will need a database analysis. You can check the guide on how to do that.

      As soon as you have them you can assign database label to MR and danger bot will suggest database reviewers.

      Please pay attention to the danger bot :warning: and correct what is possible.

  • removed review request for @dmeshcharakou

  • Moaz Khalifa
  • Moaz Khalifa added 467 commits

    added 467 commits

    • 2b8aace0...112ed929 - 465 commits from branch master
    • 24cbd43a - Reduce BATCH_SIZE to 1000
    • 11901853 - Merge branch 'master' into 38266-Cleanup-leftovers-in-packages_dependencies-table

    Compare with previous version

  • Moaz Khalifa changed the description

    changed the description

  • added database label

  • A deleted user added databasereview pending label
  • Author Developer

    Hi @OmarQunsulGitlab can you plz do the database review for this MR? thanks in advance!

  • requested review from @OmarQunsulGitlab

  • Moaz Khalifa mentioned in merge request !113556 (merged)

    mentioned in merge request !113556 (merged)

  • Omar Qunsul removed review request for @OmarQunsulGitlab

    removed review request for @OmarQunsulGitlab

  • David Fernandez
  • Moaz Khalifa added 1292 commits

    added 1292 commits

    • 11901853...e1689e80 - 1289 commits from branch master
    • 5cf01172 - Merge branch 'master' into 38266-Cleanup-leftovers-in-packages_dependencies-table
    • 6bd8a801 - Merge branch 'master' into 38266-Cleanup-leftovers-in-packages_dependencies-table
    • ca899ce7 - Convert DeleteOrphanedDependenciesWorker to a LimitedCapacityWorker

    Compare with previous version

  • Moaz Khalifa added 3 commits

    added 3 commits

    • d90f4ff5 - Convert DeleteOrphanedDependenciesWorker to a LimitedCapacityWorker
    • 8552baf3 - Fix merge conflic
    • a367cf05 - Specs clean up

    Compare with previous version

  • Moaz Khalifa added 1 commit

    added 1 commit

    • 38e0e1c9 - Add DeleteOrphanedDependenciesWorker to retry_exceptions

    Compare with previous version

  • Moaz Khalifa added 1 commit

    added 1 commit

    Compare with previous version

  • Moaz Khalifa
  • Moaz Khalifa added 1 commit

    added 1 commit

    Compare with previous version

  • Moaz Khalifa
  • Moaz Khalifa
  • Moaz Khalifa
  • Moaz Khalifa changed the description

    changed the description

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading