Managed resources are not deleted when there is a stop job defined

Given the following CI configuration:

stages:
  - ephemeral-environment

deploy-environment:
  stage: ephemeral-environment
  environment:
    name: ephemeral/$CI_COMMIT_REF_SLUG
    on_stop: stop-environment
    kubernetes:
      agent: timofurrer/playground:dev
  script:
    - echo "Deploy ephemeral environment"

stop-environment:
  stage: ephemeral-environment
  environment:
    name: ephemeral/$CI_COMMIT_REF_SLUG
    action: stop
  when: manual
  script:
    - echo "Remove ephemeral environment"

The managed resources are created then the deploy-environment runs, but they are not deleted when stop-environment runs. If the stop-environment job is removed together with the on_stop field in the deploy-environment job, the resources are correctly deleted.

It's also not clear if the stop-environment job should have an environment:kubernetes:agent association or not - both don't work in this case.