Ensure after_script is called for cancelled and timed out pipelines
Problem to Solve
It's possible to want a
after_script step for a gitlab-ci.yml, where it's important that it always run. However, the current behavior of
after_script is that if a build is canceled or times out the cleanup stage is skipped (even though this does not appear to be the original intent: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/102#note_11619214).
Backend, Step 1
We should add a new option for
interruptible when configuring CI jobs. When
interruptible: graceful is set
after_script should be run on a canceled job.
Runner, Step 1
When the runner sees a step with
when: graceful it will run it even if the job is canceled. It will also upload artifacts.
If a job is in either
pending, it should transition directly to
canceled as no execution has happened yet.
Regarding execution context:
Concern: the configuration documentation specifies that
after_scriptis run in a separate shell context. There's emphasis in the original issue that
after_scriptexecution needs to be tightly coupled with the job script specifically to have access to data (e.g. docker container IDs) from the script being cancelled. Is this going to be a problem? If so, is there any way to change that?
Response (@steveazz): I don't think is an issue, they will have all the environment variable that
scripthas, the only thing that
after_scriptwill not have is if
scriptcreate any environment variable of their own. They will be run on the same container just in a different context so they can't share information but this has always been like that from the start, users still has all the information needed to clean up their environment.
Changes will be required in the runner, and gitlab-runner#4843 is open to track that.
Backend, Part 2
Part two will be mostly for handling traces on
after_script for canceled jobs. The runner will continue sending them. Gitlab will introduce a new state:
teardown which will allow traces to be sent and display them.
This should be extended to Ci::Pipeline, where graceful cancellation sends the newly-implemented graceful cancellation to all of its builds.
pending builds would be cancelled immediately, while
running builds move immediately move on into their
Runner, Part 2
The runner should handle the new
teardown state and send traces for it. gitlab-runner#4843 is also for tracking this.
The main Cancel