Skip to content

Add docs for SAST in offline environments (no DinD)

Greg Myers requested to merge docs-secure-sast-offline-howto into master

What does this MR do?

Adds instructions on how to get SAST working offline to our documentation for users in offline environments, restricted networks, and air gaps, and without Docker in Docker.

For #11520 (closed)

SAST offline (without DinD) is technically possible but we don't have instructions on how to configure it to work without internet access.

Docs for SAST offline with DinD are not available this time. I can't get it working in a way that doesn't require hacking the SAST CI job template for every project and/or disabling HTTPS for container registry traffic. For SAST using DinD, see: !22713 (closed)

Adding these instructions should allow users to use GitLab's SAST analyzers in their offline environment.

!22713 (comment 285846966)

Background and steps for testing offline SAST functionality:

From: !22713 (comment 285846966)

"Offline" SAST Test Environment Instructions
Requirements:
  • Linux system or VM with 4+ CPU threads and 12GB+ RAM

To emulate an "on-prem" setup, I suggest using a local machine running Ubuntu as the host OS.

If on-prem setup is not an option, you can use a VM . To emulate an "offline" environment with a VM:

  • block external traffic and requests (for realism, block everything; for testing, block registry.gitlab.com and hub.docker.com at the very least)
  • Use tcpdump (or ngrep) to verify that there are no external resources being fetched
Setup Process:
  1. Install GitLab

  2. Install Docker

    curl -sSL https://get.docker.com/ | sudo bash
  3. Install & Register GitLab Runner using Docker Executor

    sudo docker run -d --name gitlab-runner --restart always \
      -v /srv/gitlab-runner/config:/etc/gitlab-runner \
      -v /var/run/docker.sock:/var/run/docker.sock \
      gitlab/gitlab-runner:latest
  4. Install and start Docker Registry

    sudo docker run -d -p 5000:5000 --name registry registry:2
  5. Push/import default SAST analyzers to Local Docker Registry. (see analyzerator.sh below)

  6. Modify /etc/gitlab-runner/config.toml

    [[runners]]
      executor = "docker"
      [runners.docker]
        pull_policy = "if-not-present"
  7. Upload license for GitLab Ultimate

  8. Import or create a project for testing SAST functionality

  9. Disable Internet access at system and/or network level to simulate an "air gap".

  10. Customize SAST template .gitlab-ci.yml : Disable Docker in Docker and set SAST_ANALYZER_IMAGE_PREFIX to point to your local registry

    include:
      - template: SAST.gitlab-ci.yml
    
    variables:
      SAST_ANALYZER_IMAGE_PREFIX: "localhost:5000/analyzers"
      SAST_DISABLE_DIND: "true"
  11. If all was successful, the SAST analyzers will scan and create reports without requiring Internet access. 🎉

Scripts created to help set up and test this:

setup.sh - script to automate Steps 1 - 4 (on Ubuntu)

#!/bin/bash
sudo apt-get update
## sudo apt-get upgrade -y
# install system dependencies
sudo apt-get install -y curl ca-certificates openssh-server
# add gitlab sources
curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh | sudo bash
# install gitlab: https://about.gitlab.com/install/
## offline local:
sudo EXTERNAL_URL="http://127.0.0.1" apt-get install gitlab-ee
# install docker: https://docs.gitlab.com/runner/install/docker.html#docker-image-installation
curl -sSL https://get.docker.com/ | sudo bash
sudo usermod -aG docker $whoami
# registry: https://docs.docker.com/registry/#basic-commands
docker run -d -p 5000:5000 --name registry registry:2
# gitlab-runner: https://docs.gitlab.com/runner/install/docker.html
docker run -d --name gitlab-runner --restart always \
  -v /srv/gitlab-runner/config:/etc/gitlab-runner \
  -v /var/run/docker.sock:/var/run/docker.sock \
  gitlab/gitlab-runner:latest

analyzerator.sh - script to automate pull/push of all SAST analyzers (this would be docker export/import in air gap, but same idea here)

#!/bin/bash
analyzers=(bandit brakeman eslint flawfinder gosec kubesec nodejs-scan phpcs-security-audit pmd-apex secrets security-code-scan sobelow spotbugs tslint)
gitlab=registry.gitlab.com/gitlab-org/security-products/analyzers/
registry=localhost:5000
for i in "${analyzers[@]}"
do
  echo pulling $gitlab$i:2
  docker pull $gitlab$i:2
  docker tag $(sudo docker images | grep $i | awk '{print $3}') $registry/analyzers/$i:2;
  docker push $registry/analyzers/$i;
done

Does this MR meet the acceptance criteria?

I think so... but if not, I'll fix it!

Conformity

@NicoleSchwartz @twoodham @theoretick @dsearles @david @stkerr @tmccaslin @marcia @marcel.amirault @eread @axil

Edited by Achilleas Pipinellis

Merge request reports