Metrics specify that a job has failed/succeeded even when running after_script which breaks metrics

Overview

The following discussion from !2342 (merged) should be addressed:

  • @ayufan started a discussion: (+1 comment)

    @steveazz @ajwalker

    Hmm. This kind of breaks a contract behind a runtime status. That our metrics would have running when still executing a CI job, this still holds true for after_script. It seems that we joint two responsibilities into a one operation, where rather this should be handled differently.

    Currently, as soon as we execute after_script it will mark runtime status as success/failed/canceled, even though Runner still executes a CI job. I don't think that this should indicate something different than running. The CI_JOB_STATUS should rather be passed between stages as soon as it is known.

    For example, this implementation assumes that we want to pass CI_JOB_STATUS only in context of after_script, but it does not take into account support for steps:. Each step does have StepWhen, it seems that we completely ignore that.

Proposal

Instead of using handleError which will change the state of the job for the metrics. When we set the CI_JOB_STATUS we use some other other to figure out if it's a fail/success such as !2350 (comment 421840876)

Edited by Steve Xuereb