<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Work on this issue](https://contributors.gitlab.com/manage-issue?action=work&projectId=278964&issueIid=425248) - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=425248) </details> <!--IssueSummary end--> MR: Pending <!-- The first line of the MR must be one of the following: 1. `MR: Pending` 2. `MR: <MR link with trailing +>`, and the first description line of the MR should be `Issue: <Issue link with trailing +>` 3. `MR: No MR` For more context, see: https://about.gitlab.com/handbook/engineering/development/dev/create/ide/index.html#1-to-1-relationship-of-issues-to-mrs --> <!-- The following sections should be filled out as part of the refinement process before the issue is prioritized. For more context, see: https://about.gitlab.com/handbook/engineering/development/dev/create/ide/#2-pre-iteration-planning-meeting --> ## Description Implement plan to terminate orphaned workspaces as described in https://gitlab.com/groups/gitlab-org/-/epics/11452#note_1559480059: 1. We will repurpose the existing logic which detects and logs orphaned workspaces in `OrphanedWorkspacesObserver`. Currently this detects workspace names that are sent by the agent in the reconciliation request, but do not exist in the database. 1. Instead of just logging the orphaned workspaces, if we detect an orphaned workspace which is NOT yet in `actual_state` of `terminated`, we will add logic to also send a `rails_info` record in the reconciliation response to the agent, with the `desired_state` set to `terminated`. This is essentially the same thing that happens when a user normally requests to terminate a workspace, but we will be doing it automatically based on detecting the workspace as orphaned. 1. This will cause the agent to eventually terminate the workspace, and then it will be reported with `actual_state` of `terminated`, so we will then stop attempting to terminate it even though it remains orphaned. Because of the way we have designed the Railway Oriented Programming pipeline for the reconciliation logic, this should be straightforward to implement and test. ## Acceptance Criteria - [ ] Orphaned workspaces (workspaces the agent knows about but missing a database record based on unique workspace name) should be guaranteed to be eventually terminated, by process described above. ## Impact Assessment Ensures that orphaned workspaces due to cascade delete rules in database cannot continue to run and incur infrastructure costs, or put the UI in an permanently inconsistent state with the running kubernetes cluster state. <!-- Replace with other type, e.g. bug or maintenance, if appropriate --> <!-- Replace with other subtype if appropriate --> <!-- By default, all issues start in the unprioritized status. See https://about.gitlab.com/handbook/engineering/development/dev/create/ide/#-remote-development-planning-process --> <!-- For simplicity and to avoid triage bot warnings about missing workflow labels, we will default to issues starting at the refinement phase -->