Add the ability to set the build state from within the job (return code)

Description

First, I have searched gitlab-{ce,ee} issue but cannot find the relevant feature (nope, allow_failure is not what I need)

In my build pipeline I have a serverspec. Each server is tested on a seperate (parallel) job. And I need the job itself to be able to set the build state :

  • success : remote is UP and serverspec didn't found any error
  • failed : remote is UP and serverspec has found at least one error
  • skipped or warning : remote is DOWN

Today I "KO" or "not tested" jobs are all in failed state and I have no way to add granularity at this level.

As a workaround I have to log the jobs ouputs, reference the logs as artifacts, and check them in a subsequent serverspec check stage which contains 1 job parsing the log files and failing at least 1 of the previously failed jobs actually failed at serverspec test.

Proposal

I'd like to be able to set the build state from within the job.

.test: &test
  <<: *serverspec
  return_code:
    success: 0
    skipped: 1
    warning: 2-254
    failed:  255
    
test server1: *test
test server2: *test
test server3: *test

I know the above is not the ideal implementation one would want. But you get the idea 😄

Links / references

Jenkins can do this : https://support.cloudbees.com/hc/en-us/articles/218554077-How-to-set-current-build-result-in-Pipeline-

For the yaml implementation ansible handles it (poor design though...) : http://docs.ansible.com/ansible/latest/playbooks_error_handling.html

Documentation blurb

Overview

What is it? Why should someone use this feature? What is the underlying (business) problem? How do you use this feature?

Use cases

Who is this for? Provide one or more use cases.

Feature checklist

Make sure these are completed before closing the issue, with a link to the relevant commit.