Handle active context code repository deletion

What does this MR do and why?

Processes Ai::ActiveContext::Code::Repository records in pending_deletion state.

The SchedulingService checks if there are repos in this state every 10 minutes and enqueues RepositoryDeleteWorker for 100 repos at a time.

The RepositoryDeleteWorker runs in a lock so that we process a project once only - it's not strictly necessary since a delete is idempotent but this is to prevent multiple delete operations to Elastic at once.

It calls Indexer.run_delete!(repository) which calls the indexer with the right options (including operation = delete_project). If there's an error, it gets logged.

References

• [Index state tracking: Deletion] ProcessPending... (#545945 - closed)
• Requires gitlab-elasticsearch-indexer!795 (merged) to be merged and released first -> DONE

How to set up and validate locally

• Choose a repo that was indexed in Elastic already
• Change the state to :pending_deletion
• Run the SchedulingWorker
• Note that the documents were deleted in Elastic and that the repo is now in deleted state

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.