Add docker sandbox container support (network isolation)
What does this MR do?
Adds sandbox container support, similar to how Kubernetes creates Pods, initially focusing on network isolation.
For each build, before services, a parent container is created. This container is referred to as the sandbox container. Docker allows you to then join additional containers (services, build) into the sandbox's network, IPC and Pid namespaces.
This MR focuses solely on joining containers to the sandbox's network namespace. In the future, I think we should also allow IPC and Pid namespace sharing to be user controlled.
The global configured Docker.NetworkMode, Docker.ExtraHosts and Docker.Links can still be used and are compatible, even with the sandbox container enabled. However, these settings are only applied to the sandbox container, rather than all containers. The behaviour is as you'd expect: containers sharing the same network namespace inherit the ExtraHosts, can access any linked containers and share the configured network mode.
Why was this MR needed?
-
There is a clear need for network isolation and to escape Docker's legacy linking. One such MR already plans to add this using a network manager and bringing up a network for each build (!1569 (merged)). The end-user experience of the two is that ultimately the sandbox container support mimics Kubernetes' behaviour. All containers are available to
localhost, as opposed to each being given a different IP address. This does have the benefit that moving between the two executors should be less painful.This MR doesn't have to be a replacement for !1569 (merged). If there is a requirement for network isolation through bringing up a new network for whatever reason, both solutions should work even when both enabled. If both enabled, the sandbox container would be assigned the new network, and the child containers would still share that network's namespace.
-
In the future, we can use the sandbox container to support IPC namespace sharing.
-
In the future, we can use the sandbox container to support Pid namespace sharing, and some of the bonuses that go along with that, such as zombie process cleanup.
Are there points in the code the reviewer needs to double check?
- There's a test for creating the sandbox container, and checking that child containers are then using it, but nothing that explicitly does a working e2e test with Docker and the sandbox container enabled. Unless there's already e2e tests that are executing and modifying the feature flags, this might be good to add? But I'm not sure if anything else does that.
Does this MR meet the acceptance criteria?
- Documentation created/updated
- Added tests for this feature/bug
-
In case of conflicts with
master- branch was rebased