Profiling and linux tracing on kubernetes

One of the capabilities we are currently sacrificing when moving workloads to kubernetes is profiling via perf and low-level tracing via bcc and bpftrace.

This is something we should figure out how to do in kubernetes (and GKE in particular).

For bpftrace, we should evaluate kubectl-trace.
For BCC, we can take a look at Linux BPF CPU Profiling with kubectl on Kubernetes
For perf, we can take a look at Enabling perf in Kubernetes with Docker’s default seccomp profile

One overall question here is whether we need to allow privileged containers in GKE, if these approaches even work in GKE, and what the security implications of that are.