Engineering Proposal for Host-based IDS/IPS Protection
For more details and context, please reference the Proof of Concept research issue that preceded this.
Engineering will propose an architecture that meets the following customer requirements:
- File integrity monitoring - The ability to monitor and prevent changes to the container after the container has been started
- Application allow listing - The ability to monitor and prevent processes from starting in the container if they do not match a predefined list of process names or hashes. This could also include the ability to monitor and prevent new processes from starting after the container has been fully started.
- Active response / blocking - This is the "prevent" capability described in the above requirements. Fundamentally it is the ability to stop unwanted activity from occurring (processes, network communications, file changes, etc.).
- Acceptable performance overhead - This needs to be considered both two perspectives. 1) Performance overhead should not interfere with the smooth operation of the customer's application. 2) Performance overhead should not cause a customer's cloud bill to increase dramatically.
Nice to Have
- Malware scanning - The ability to scan all container file systems for known malware while they are running / after the containers have started.
- Vulnerability scanning - The ability to identify known vulnerabilities in the packages of a container after it has been started and is running in production. Note: An acceptable solution would be to re-scan images that are running in production using the Secure stage capabilities (rather than scanning the running container itself).
- Configuration vulnerability detection - The ability to identify known vulnerabilities in the way that running applications are configured after a container has been started and while the application is running in production. Note: An acceptable solution would be to re-scan images that are running in production using the Secure stage capabilities (rather than scanning the running container itself).
- No need to give GitLab credentials to production Kubernetes or container environments - Customers have an option to do a manual install to avoid concerns about GitLab storing credentials to a production environment.
Engineering Architecture Proposal
|Priority||T-Shirt Size||Technology||Requirements Addressed|
|1||M||Falco||Prerequisite to all other technologies below|
|2||L||AppArmor + Pod Security Policy||Inline Blocking/Prevention, Application Allow Listing, File Integrity Monitoring|
|3||M||Falco Sidekick||Active Response Options (create GitLab issue, send Slack message, run shell script, etc.)|
|4||S||GitLab Scheduled Pipeline running Secure Scans||Vulnerability Scanning, Configuration Vulnerability Scanning|
|TECHNOLOGY||REQUIREMENTS ADDRESSED||PROS||CONS||ITERATION PLAN||POSSIBLE ADDITIONAL FEATURES BUILT WITH THAT TECHNOLOGY||T-SHIRT SIZE||DEPENDENCIES|
|Falco||File Integrity Monitoring||
||potentially not every Cloud provider accepts Falco kernel modules||
|Pod Security Policy + AppArmor||Application allow listing||
||A Helm chart with the DaemonSet which loads policies will be maintained by us.||
||Pod Security Policy also allows the definition of permissions related to capabilities, privileges (user, volumes, network) in addition to secComp and sysctl profiles.||L||None|
|Simple Go app||Active response / blocking||
||application written in-house (simple, but we still have to maintain it (+ helm charts) and fix all potential issues)||
||Capabilities are limitless as we allow customer to write their own script to be executed when certain event is triggered (ie. send a slack update, do API requests, etc.)||L||- Falco|
|NATS + Kubeless||Active response / blocking||
|GitLab (scheduled pipeline)||Malware scanning||
||we will not detect malwares introduced during container runtime (but we should be able to limit malicious behavior with AppArmor/Pod Policy)||Make sure that we can detect malwares and that when we detect them during scheduled pipeline they are visible on project page||S||None|
|ClamAV||Malware scanning||Has a stable helm chart which can be used to scan on a node level or within a container if proper mapping is set.||
|Dagda||Malware scanning||Uses ClamAV for malware detection in addition to other features||It does not have a helm chart so it would have to be created and maintained by our team.||
||it has other features like static analyses, anomaly detection as it has integration with Falco.||L||Falco|
|GitLab (scheduled pipeline)||Vulnerability scanning||
||we will not detect vulnerabilities introduced during container runtime (but we should be able to limit malicious behavior with AppArmor/Pod Policy)||Make sure that all vulnerabilities discovered during scheduled pipeline are visible on project page||S||None|
|Dagda||Vulnerability scanning||it has the ability to scan running containers for vulnerabilities.||It does not have a helm chart so it would have to be created and maintained by our team.||
||it has other features like static analyses, anomaly detection, malware detection as it has integration with ClamAV and Falco.||L||Falco / ClamAV|
|OpenSCAP - GitLab Secure Stage||Configuration vulnerability detection||we can detect if packages installed in the container are configured and compliant with security standards||can only be executed for the image (not running container), so we will not detect configuration vulnerabilities that were introduced during the runtime (AppArmor should help here)||Add OpenSCAP as a part of CI/CD pipeline (when we are checking container image)||L/XL||None|
|No available solution that is working for running containers (possibly we could extend OpenSCAP to support that)||Configuration vulnerability detection||N/A||N/A||N/A||N/A||?||None|
|kube-bench / kubeaudit (optional)||Configuration vulnerability detection (for Kubernetes cluster)||
||Add kube-bench / kubeaudit as a part of CI/CD process, parse and collect vulnerabilities (possibly new vulnerability report type will have to be added)||L||None|
|Kubesec||Configuration vulnerability management (Passwords/Secrets)||Allow users to encrypt their secrets within K8s||L||None|