Add groups concept to Gitlab CI jobs
Problem to solve
The Job logs in Gitlab CI can be huge for some projects and often are made up of different steps. Currently it is nearly impossible to properly view it in the log viewer and it is really hard to figure out at what step something failed.
Usually the answer here would be to use stages like build, test, etc. to split this up but this is not feasible for some projects as the tests in a C project for example usually depend on build artifacts one does not want to upload (hundreds or thousands of object files and intermediate buildsystem generated metadata stuff like libtool objects).
In such cases where the "stages" are tightly coupled and not easy to split up due to the requirement to upload tons of artifacts to restore the needed state, it would make sense to add a concept of groups similar to what GitHub Actions offers (they call it steps).
It would help to be able to manually create named "groups" that show up in the output like GitLab already does for things like the before/after scripts:
Intended users
- Devon (DevOps Engineer) - Who is authoring .gilab-ci.yml files that generate lengthly job logs.
Further details
At VLC we would use this to group different steps of our build together, mostly to get a cleaner
output like (each >
marks a expandable section of the log):
> Building contribs
> Building VLC
> Running unit tests
> Packaging contribs
In the log viewer having such collapsible sections and being able to see which failed would help us tremendously to analyze build failures without having to download the raw log and grep through it.
It could enrich emails sent too as it could better inform what failed, like:
Job XYZ failed at build stage when 'Building VLC'
Proposal
As an example let's look at this hypothetical job definition to illustrate how it could work:
debian:
extends: .docker-template
- group:
name: 'Build tools'
script: |
cd extras/tools
./bootstrap && make -j4 --output-sync=recurse
cd ../..
- group:
name: 'Build contrib'
script: |
cd contrib
../bootstrap
make -j4 --output-sync=recurse fetch
make package
cd ..
- group:
name: 'Build VLC'
script: |
./bootstrap
./configure
make -j4
- group:
name: 'Run unit tests'
script: |
VLC_TEST_TIMEOUT=60 sh -x ./test/make_check_wrapper.sh -j4
It would not impact existing usage at all as when people do not need groups, they could just not use them and continue to use a single script entry.
Permissions and Security
No changes
Documentation
TBA
Testing
TBA
What does success look like, and how can we measure that?
I don't understand what I am supposed to write here.
What is the type of buyer?
Given that VLC uses the open source edition, it would need to be included in GitLab Core to be useful for us. Additionally I think it probably is not a good idea to split up CI features that affect the gitlab-ci.yml definition to specific tiers as it would lead to different tiers having different "valid" definitions which would be incredibly confusing.