Skip to content

Adds cilium alerts module for agentk

Zamir Martins requested to merge generate_cilium_alerts into master

Goal of this MR?

To propose and initiate a discussion on extending agentk in order to support the creation of alerts related to cilium logs and cilium network policies.

***I am happy to decompose this MR in smaller pieces, I am keeping as is so the whole context can be seen.

What does it do?

Adds a new module which is loading data from hubble relay (centralized logs and provided as part of cilium installation) mapping this data with a possible related cilium network policy (loaded through k8s rest api). In case there is a match, an alert will be created by making a request to GitLab RoR internal api.

What's pending?

Unit tests and the request to internal api.

Potential future enhancements

  • Adding L4 and L7 rules into the logic
  • Add logic in regards to the annotation for each network policies
  • Caching the latest version of cilium network policies
  • Improve the failure handling while loading the flows

Local testing

It requires cilium to be installed with hubble-relay (in this example it is assumed GKE as the k8s platform):

kubectl create namespace cilium
helm repo add cilium https://helm.cilium.io
helm repo update
helm install cilium cilium/cilium --version 1.8.1 \
  --namespace cilium \
  --set nodeinit.enabled=true \
  --set nodeinit.reconfigureKubelet=true \
  --set nodeinit.removeCbrBridge=true \
  --set cni.binPath=/home/kubernetes/bin \
  --set gke.enabled=true \
  --set ipam.mode=kubernetes \
  --set nativeRoutingCIDR=$NATIVE_CIDR \
  --set nodeinit.restartPods=true \
  --set hubble.listenAddress=":4244" \
  --set hubble.relay.enabled=true

Port forwarding is an option when running agent from outside of the cluster:

kubectl port-forward -n cilium service/hubble-relay 9000:80

New argument has been added for agentk:

bazel run //cmd/agentk -- --kas-address=grpc://:8150 --token-file="$(pwd)/token.txt" --hubble-relay-addr=127.0.0.1:9000

A simple example of Pods and NetworkPolicies can be found in cilium documentation

Other related information

Related issues: gitlab-org/gitlab#277153 (closed) and gitlab-org/gitlab#277156 (closed)

Edited by Zamir Martins

Merge request reports