3 - Parallel Matrix
Introduction
The goal of this exercise is to implement parallel matrix jobs to reduce job definition redundancy.
Step 01 - Parallelize the build job
- Instead of writing out separate instances of our build job, we will use the parallel matrix feature to run multiple instances of the same job with different variables.
We will repurpose the .build job as our single build job.
- Go to Build > Pipeline Editor.
- First, rename .build to build, removing the "." in the front of the name so that it is no longer a hidden job.
- Then modify the build job to remove the script line that echos the
IMAGE
variable to theimagevars.env
file, and remove theartifacts:reports:dotenv
report section.
The build job should now look like this:
build:
stage: build
variables:
IMAGE: $CI_REGISTRY_IMAGE/$PROJECT_DIR:$CI_COMMIT_SHA
script:
- cd $PROJECT_DIR
- pwd
- echo "building $PROJECT_DIR"
- docker info
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build -t $IMAGE .
- docker push $IMAGE
- Add the parallel:matrix section underneath the script block of the build job:
parallel:
matrix:
- PROJECT_DIR: [python1, python2]
Your build job should now look like this:
build:
stage: build
variables:
IMAGE: $CI_REGISTRY_IMAGE/$PROJECT_DIR:$CI_COMMIT_SHA
script:
- cd $PROJECT_DIR
- pwd
- echo "building $PROJECT_DIR"
- docker info
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build -t $IMAGE .
- docker push $IMAGE
parallel:
matrix:
- PROJECT_DIR: [python1, python2]
- Delete the build1 and build2 jobs.
Note: You will want to complete Step 02 below before committing your changes
Step 02 - Parallelize the container test job
- Delete the container1_job and container2_job jobs.
- Modify the .get_container job in a similar way to the build job above:
- remove the "." at the front of the name so that it is no longer a hidden job
- add the parallel:matrix block underneath the script block
parallel:
matrix:
- PROJECT_DIR: [python1,python2]
- Add a variables: section to the get_container job to explicitly define the
IMAGE
value
variables:
IMAGE: $CI_REGISTRY_IMAGE/$PROJECT_DIR:$CI_COMMIT_SHA
The resulting get_container job should look like this:
get_container:
stage: test
variables:
IMAGE: $CI_REGISTRY_IMAGE/$PROJECT_DIR:$CI_COMMIT_SHA
script:
- echo "The image being used is $IMAGE"
# - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
# - docker pull $IMAGE
parallel:
matrix:
- PROJECT_DIR: [python1,python2]
Commit your changes and navigate to the running pipeline to view its details.
3-parallel-matrix
branch.
_Need additional help with this exercise? You can see the full .gitlab_ci.yml file solution for this exercise in the BONUS
See how GitLab uses the parallel:matrix
capability for our own Workhorse testing -it's used to run tests for the workhorse code against multiple Go versions and Redis versions concurrently.