Multiple workflow deployments with the same definition make undeploy impossible

It happens on ai. Multiple deployments exist for the same workflow definition. The consequence is that the runner is not able to stop it anymore, and the rest api crashes silently (at least from the ui point of view).

Steps to reproduce

I don't know how to get there specially, but we should be able to recover from it. Currently, the exception is handled, but we don't take action on it. (see https://gitlab.com/ryax-tech/ryax/ryax-runner/-/blob/master/ryax/runner/infrastructure/database/repositories/workflow_repository.py?ref_type=heads#L92)

Relevant logs and/or screenshots

runner=# select id, name, state, workflow_definition_id from workflow_deployment where workflow_definition_id = 'fdb7c179-7832-4240-919e-183d77d4d213' and state = 'RUNNING';                                                          
                   id                   |      name       |  state  |        workflow_definition_id
----------------------------------------+-----------------+---------+--------------------------------------
 WorkflowDeployment-1763135983-2qghnm2z | video detection | RUNNING | fdb7c179-7832-4240-919e-183d77d4d213
 WorkflowDeployment-1763132215-ijfqw6uq | video detection | RUNNING | fdb7c179-7832-4240-919e-183d77d4d213
 WorkflowDeployment-1763135778-t6rl2mkn | video detection | RUNNING | fdb7c179-7832-4240-919e-183d77d4d213
(3 rows)

This is the trace visible in the runner logs:

2025-11-17 10:54:05,102 ERROR [asyncio] [base_events.py:1785] [trace_id=0 span_id=0 resource.service.name= trace_sampled=False] - Task exception was never retrieved
future: <Task finished name='Task-1127409' coro=<consumer() done, defined at /data/.env/aio_pika/queue.py:25> exception=MultipleRunningWorkflowDeploymentsError('Multiple workflows are running for the same workflow definition: fdb7c179-7832-4240-919e-183d77d4d213')>
Traceback (most recent call last):
  File "/data/ryax/runner/infrastructure/database/repositories/workflow_repository.py", line 89, in get_eventually_running_workflow_by_workflow_definition_id
    workflow_deployment = self.session.execute(query).scalar_one()
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/.env/sqlalchemy/engine/result.py", line 1208, in scalar_one
    return self._only_one_row(
           ^^^^^^^^^^^^^^^^^^^
  File "/data/.env/sqlalchemy/engine/result.py", line 614, in _only_one_row
    raise exc.MultipleResultsFound(
sqlalchemy.exc.MultipleResultsFound: Multiple rows were found when exactly one was required

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/.env/aio_pika/queue.py", line 31, in consumer
    return await create_task(callback, message)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/.env/opentelemetry/instrumentation/aio_pika/callback_decorator.py", line 54, in decorated
    return_value = await callback(message)
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/ryax/common/infrastructure/messaging/consumer.py", line 86, in handle_message
    raise err
  File "/data/ryax/common/infrastructure/messaging/consumer.py", line 73, in handle_message
    await message_handler.handler_function(message)
  File "/data/.env/dependency_injector/wiring.py", line 1040, in _patched
    return await _async_inject(
           ^^^^^^^^^^^^^^^^^^^^
  File "src/dependency_injector/_cwiring.pyx", line 61, in _async_inject
  File "/data/ryax/runner/infrastructure/messaging/handlers/workflow_rmq_handler.py", line 263, in on_undeploy_workflow
    await message_bus.handle_command(
  File "/data/ryax/common/application/message_bus.py", line 238, in handle_command
    result = await handler(command)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/data/.env/dependency_injector/wiring.py", line 1040, in _patched
    return await _async_inject(
           ^^^^^^^^^^^^^^^^^^^^
  File "src/dependency_injector/_cwiring.pyx", line 61, in _async_inject
  File "/data/ryax/runner/application/handlers/workflow_deployment_handlers.py", line 317, in stop
    workflow = uow.workflow_repository.get_eventually_running_workflow_by_workflow_definition_id(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/ryax/runner/infrastructure/database/repositories/workflow_repository.py", line 93, in get_eventually_running_workflow_by_workflow_definition_id
    raise MultipleRunningWorkflowDeploymentsError(
ryax.runner.domain.workflow_deployment.workflow_deployment_entities.MultipleRunningWorkflowDeploymentsError: Multiple workflows are running for the same workflow definition: fdb7c179-7832-4240-919e-183d77d4d213

Link to the Ryax instance

The workflow is here: https://ai.ryax.io/app/workflow/fdb7c179-7832-4240-919e-183d77d4d213/overview I didn't take action on it so we could investigate further.

Possible fixes

Remove the deployed workflow when we catch the exception and set it as error ?