gitlab-ci runners report a job as correctly done even though exit code was non-zero
Description of the problem
When compiling a large project, I occasionally get successful runs of the test suite even if the test suite failed (run using .gitlab-ci.yml). An example can be seen in the build: https://gitlab.com/gnutls/gnutls/pipelines/5149934
While on the overview page all looks ok, clicking on the libvirt build we see: https://gitlab.com/gnutls/gnutls/builds/6693360
FAILED
FAIL virnettlssessiontest (exit status: 1)
+ exit 1
error: Bad exit status from /var/tmp/rpm-tmp.kj5l20 (%check)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.kj5l20 (%check)
However the runner detects:
Build succeeded
The actual .gitlab-ci.yml script contains (the full is on this link):
Fedora/apps/libvirt:
stage: compile
image: $CI_REGISTRY/$PROJECT_GROUP:$FEDORA_BUILD
script:
- dnf install fedpkg numactl-devel sheepdog xen-devel numad -y
- dnf builddep libvirt -y
- make autoreconf && mkdir -p build && cd build &&
../configure --disable-non-suiteb-curves --prefix=/usr --libdir=/usr/lib64 --disable-cxx --disable-do
make -j$(nproc) && make install
- fedpkg clone -a libvirt
- cd libvirt && fedpkg local
tags:
- shared
dependencies:
- build/Fedora24
except:
- tags
and the command that failed is the last one, (cd libvirt && fedpkg local), which should have failed since fedpkg's exit code is 1.
Is that a limitation of the .gitlab-ci.yml format for the script directive (e.g., in the documentation only one script command is mentioned rather than the multiple used in that build.
Which Group/Project (with full path) is experiencing the issue?
gnutls/gnutls
Approximate date/time when the error occurred.
2016-11-27