Resolve "Merge request stuck in locked state when getting merged in a merge train"

What does this MR do and why?

It rare cases, it can happen that a merge train car gets stuck.

This MR introduces a new addition to the MergeTrains::RefreshService that unsticks (==unlocks) a stuck merge request.

References

Query-Plan - Stuck cars:

https://postgres.ai/console/gitlab/gitlab-production-main/sessions/50997/commands/151003

Query-Plan - Refreshable:

https://postgres.ai/console/gitlab/gitlab-production-main/sessions/50997/commands/151030

Screenshots or screen recordings

Before	After

How to set up and validate locally

# Rails Console

# Set your project to have merge trains activated.

# 0. Find your project and merge request
project = Project.find_by_full_path('your/project') 
mr = project.merge_requests.find_by(iid: 123) 
user = User.find_by(username: 'your-username')

# 1. Put MR on train/create a car

car = mr.create_merge_train_car!(user: user, target_project: project, target_branch: mr.target_branch)

# 2. Set car to fresh (to have an appropriate initial state)

car.update_columns(status: MergeTrains::Car.state_machines[:status].states[:fresh].value, pipeline_id: mr.head_pipeline&.id)

# 3. Simulate stuck state

car.update_columns(status: MergeTrains::Car.state_machines[:status].states[:merging].value, updated_at: 3.hours.ago)

mr.update_columns(state_id: MergeRequest.available_states[:locked], in_progress_merge_commit_sha: 'abc123', updated_at: 3.hours.ago, auto_merge_enabled: true, merge_user_id: user.id)

# 4. Verify

puts "Car: #{car.reload.status_name}" # => merging 
puts "MR: #{mr.reload.state}" # => locked

# Look now into your user interface, the merge request will never get merged or unlocked.

# 5. Enable the feature and test the fix manually or wait for UnstickStuckMergesCronWorker to do its magic (will happen twice an hour)

Feature.enable(:unstick_stuck_merge_requests) MergeTrains::RefreshService.new(project.id, mr.target_branch).execute

# 6. Check - first must not merging, second must be locked, user interface must show ready to merge

puts "Car: #{car.reload.status_name}" puts "MR: #{mr.reload.state}"

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #389044