Ensure after_script is called for cancelled and timed out pipelines
Release notes
Problem to Solve
It's possible to want a after_script
step for a gitlab-ci.yml, where it's important that it always run. However, the current behavior of after_script
is that if a build is canceled or times out the cleanup stage is skipped (even though this does not appear to be the original intent: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/102#note_11619214).
Proposal
Backend, Step 1
We should add a new option for interruptible
when configuring CI jobs. When interruptible: graceful
is set after_script
should be run on a canceled job.
Gitlab will communicate this with the runner through the requestJob
endpoint. A new value will be added to step::when
- graceful
. The step for after_script
will have when:graceful
.
Runner, Step 1
When the runner sees a step with when: graceful
it will run it even if the job is canceled. It will also upload artifacts.
If a job is in either created
or pending
, it should transition directly to canceled
as no execution has happened yet.
Regarding execution context:
Concern: the configuration documentation specifies that
after_script
is run in a separate shell context. There's emphasis in the original issue thatafter_script
execution needs to be tightly coupled with the job script specifically to have access to data (e.g. docker container IDs) from the script being cancelled. Is this going to be a problem? If so, is there any way to change that?
Response (@steveazz): I don't think is an issue, they will have all the environment variable that
before_script
&script
has, the only thing thatafter_script
will not have is ifbefore_script
/script
create any environment variable of their own. They will be run on the same container just in a different context so they can't share information but this has always been like that from the start, users still has all the information needed to clean up their environment.
Gitlab Runner
Changes will be required in the runner, and gitlab-runner#4843 is open to track that.
Backend, Part 2
Part two will be mostly for handling traces on after_script
for canceled jobs. The runner will continue sending them. Gitlab will introduce a new state: teardown
which will allow traces to be sent and display them.
This should be extended to Ci::Pipeline, where graceful cancellation sends the newly-implemented graceful cancellation to all of its builds. created
and pending
builds would be cancelled immediately, while running
builds move immediately move on into their after_script
.
Runner, Part 2
The runner should handle the new teardown
state and send traces for it. gitlab-runner#4843 is also for tracking this.
Frontend
The main Cancel interruptible: graceful
.
Intended users
User experience goal
Proposal
Further details
Permissions and Security
Documentation
Availability & Testing
What does success look like, and how can we measure that?
What is the type of buyer?
Is this a cross-stage feature?
Links / references
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.