Metrics specify that a job has failed/succeeded even when running after_script which breaks metrics
Overview
The following discussion from !2342 (merged) should be addressed:
-
@ayufan started a discussion: (+1 comment) @steveazz @ajwalker
Hmm. This kind of breaks a contract behind
a runtime status. That our metrics would haverunningwhen still executing a CI job, this still holds true forafter_script. It seems that we joint two responsibilities into a one operation, where rather this should be handled differently.Currently, as soon as we execute
after_scriptit will mark runtime status assuccess/failed/canceled, even though Runner still executes a CI job. I don't think that this should indicate something different thanrunning. TheCI_JOB_STATUSshould rather be passed between stages as soon as it is known.For example, this implementation assumes that we want to pass
CI_JOB_STATUSonly in context ofafter_script, but it does not take into account support forsteps:. Each step does haveStepWhen, it seems that we completely ignore that.
Proposal
Instead of using handleError which will change the state of the job for the metrics. When we set the CI_JOB_STATUS we use some other other to figure out if it's a fail/success such as !2350 (comment 421840876)