Skip to content

Create network per build in docker executor

Steve Exley requested to merge steve.exley/gitlab-runner:gitlab-runner-add-network-per-build into master

What does this MR do?

Re-Opens !1041 (closed). Rebased onto v12

Add support for per-job Docker network creation, based on !609 (closed), and built on top of !895 (closed) (rebased on master). By nature of its implementation it also provides a workaround fix for corporate DNS resolution as outlined in https://github.com/docker/libnetwork/issues/2187, and may mitigate issues such as #1541 (closed).

Why was this MR needed?

There is a huge need of proper service discovery b/w the job container and the service container(s). Use cases include:

  • Running a selenium grid peer to the build container which can see the build container (i.e. karma tests).
  • Replicating service based environments which depend on multiple services being able to communicate using Docker networks
  • Set up complex interdependent application stacks.

Docker has indicated that the current links functionality used by Gitlab Runners is a legacy feature that may eventually be removed and advises using Docker Networks instead

Previous MR !1041 (closed) was the most popular MR for the Gitlab Runner project by a significant margin and received community contributions indicating the need for this feature.

Are there points in the code the reviewer needs to double check?

The feature works and has been used extensively by some of the followers on !1041 (closed). However it would benefit from a review by somebody more familiar with the code.

At the time of opening the MR, there are some issues around static QA which I've not yet addressed.

Does this MR meet the acceptance criteria?

  • Documentation created/updated
  • Added tests for this feature/bug
  • In case of conflicts with master - branch was rebased

Network per build vs Container mode networking

We had extensive discussions on whether which option is better, we had 2 to choose from Network per build (what this merge request is achieving) and Container Mode what !1674 (closed) is achieving. Below are some links that are noteworthy on how we come to a decision:

What are the relevant issue numbers?

Closes #1042 (closed) #4430 (closed) #2699 (closed)

Based off of !609 (closed) and !895 (closed).

Partial fix for #1541 (closed).

Edited by Steve Xuereb

Merge request reports