Commit 6362d1cb authored by Vesa Vertainen's avatar Vesa Vertainen

updated setup-pipeline.sh to work with new pipelines-setup

parent ca330f2b
......@@ -14,37 +14,46 @@ The full list of components:
* [Worker node](https://concourse-ci.org/concourse-worker.html)
* [PostgreSQL node](https://concourse-ci.org/postgresql-node.html)
* Gitlab CE
* [dns-proxy-server](https://github.com/mageddo/dns-proxy-server) (optional)
* [add2git-lfs](https://gitlab.com/saguywalker/add2git-lfs) (not implemented yet)
* [dns-proxy-server](https://github.com/mageddo/dns-proxy-server) (optional)
* Docker Registry 2.0 (optional private registry)
* [add2git-lfs](https://gitlab.com/saguywalker/add2git-lfs) (experimental)
# Prerequisites
* Ubuntu 18.04 (others untested)
* Ensure that your system has more than 6 GB memory
* Ensure that your system has more than 4 GB free disk space
* [Docker](https://docs.docker.com/install/) >= 17.12.0
* [Docker Compose](https://docs.docker.com/compose/install/) > 1.17.1
* Ensure that your system has more than 6 GB memory
* Ensure that your system has more than 4 GB free disk space
[Read the wiki for help about installing/updating docker & docker-compose.](https://gitlab.com/CinCan/environment/wikis/home)
# Deploying the pilot environment
```bash
# Clone the repository
git clone -b dev https://gitlab.com/CinCan/environment
git clone https://gitlab.com/CinCan/environment
# Change directory
cd ./environment
# Run the script (requires root privileges)
sudo bash build.sh -e BUILD_PATH=/path/to (default: /opt/cincan)
# Change directory
cd /path/to/build
# Build the environment using Docker Compose
docker-compose up -d db \
ci \
worker \
gitlab \
dns (optional)
dns (optional) \
reg (optional)
```
The Gitlab will take some time to get up (~5 minutes). When it's up and running, you can move on to setting up a pipeline.
The Gitlab will take some time to get up (~5 minutes).
You can follow the starting process using following command:
```bash
......@@ -53,38 +62,32 @@ docker logs gitlab.cincan.io --follow
# Setting up a pipeline
You can list all pipelines that have a setup script, by typing ```sudo bash setup-pipeline.sh```
You can list all pipelines that have a setup script with ```sudo bash setup-pipeline.sh```
```bash
sudo bash setup-pipeline.sh
[+] Cloning the pipelines.git
This script executes setup.sh from pipelines/<PIPELINE>
Usage: sudo bash setup-pipeline.sh <PIPELINE>
Available pipelines:
pdf-pipeline
document-pipeline
Available pipelines
1) pdf-pipeline
2) pdf-pipeline Private registry version
3) document-pipeline
4) Quit
```
Currently, there are two pilot pipelines with quick setup: [pdf-pipeline](https://gitlab.com/CinCan/pipelines/tree/master/pdf-pipeline), and the more advanced [document-pipeline](https://gitlab.com/CinCan/pipelines/tree/master/document-pipeline).
Currently, there are two pilot pipelines with quick setup: [pdf-pipeline](https://gitlab.com/CinCan/pipelines/tree/master/pdf-pipeline), and the more advanced [document-pipeline](https://gitlab.com/CinCan/pipelines/tree/master/document-pipeline)
For example, to setup the document-pipeline, follow these setup steps:
To setup a pipeline:
1. Set up [the pilot environment](https://gitlab.com/CinCan/environment)
2. sudo bash setup-pipeline.sh pdf-pipeline
2. sudo bash setup-pipeline.sh
3. The script will ask to login and to set up the pipeline to concourse. You can alternatively manually run the commands, see PIPELINE-README.
3. Select the pipeline to set up
4. Login to https://172.20.0.3 to see the pipeline work. (Or https://concourse.cincan.io if proxy is enabled)
4. Login to https://172.20.0.3 to see the pipeline work. (Or https://concourse.cincan.io if proxy is enabled)
2. Type in username and password when prompted.
3. The script will ask to login and to set up the pipeline to concourse. You can alternatively manually run the commands, see PIPELINE-README.
4. Login to https://172.20.0.3 to see the pipeline work (or https://concourse.cincan.io if using dns)
# How to stop and remove the whole pilot environment
......@@ -94,8 +97,30 @@ cd /path/to/build
docker-compose down --volumes --rmi all && rm -rf /path/to
```
# Steps to set up a pipeline in the private registry environment
1. Deploy the pilot environment as described above, including the "reg" option
2. Run ```sudo bash setup-private-registry.sh```
```bash
sudo bash setup-private-registry.sh
```
* Type in environment username and password when prompted.
* Select the tools to use from the pop up dialog box. For example, if you are setting up the pdf-pipeline, check "jsunpack-n", "pdfid" and "peepdf".
* Choose whether you want to download the tool images from the Docker hub, or to build them locally.
3. Run ```sudo bash setup-pipeline.sh``` and choose a Private registry version pipeline
4. Login to https://172.20.0.3 to see the pipeline work. (Or https://concourse.cincan.io if proxy is enabled)
# Read more
Read more about [the pilot environment installation options from the wiki](https://gitlab.com/CinCan/environment/wikis/home)
Read more about [the pdf-pipeline here.](https://gitlab.com/CinCan/pipelines/tree/master/pdf-pipeline)
Read more about [the document-pipeline here.](https://gitlab.com/CinCan/pipelines/tree/master/document-pipeline)
......@@ -14,6 +14,7 @@ if [ "$HOSTNAME" = "concourse.cincan.io" ]; then
concourse generate-key -t rsa -f "${KEYS_PATH}/session_signing_key"
concourse generate-key -t ssh -f "${KEYS_PATH}/tsa_host_key"
concourse generate-key -t ssh -f "${KEYS_PATH}/worker_key"
ssh-keygen -o -t rsa -m pem -b 4096 -q -N '' -f "${KEYS_PATH}/cincan_rsa"
cp "${KEYS_PATH}/worker_key.pub" "${KEYS_PATH}/authorized_worker_keys"
concourse web
fi
......
......@@ -28,6 +28,10 @@ ERROR="\e[0;91m[x]\e[0m"
: "${ALT_EXTERNAL_URL_GITLAB:=gitlab.cincan.io}"
: "${BUILD_PATH:=/opt/cincan}"
: "${CONCOURSE_VERSION:=5.4.1}"
: "${GITLAB_VERSION:=12.2.4-ce.0}"
: "${COMMON_NAME_REGISTRY:=registry.cincan.io}"
: "${EXTERNAL_URL_REGISTRY:=172.20.0.6}"
: "${ALT_EXTERNAL_URL_REGISTRY:=registry.cincan.io}"
# Function definitions
......@@ -48,6 +52,8 @@ help (){
echo "ALT_EXTERNAL_URL_CONCOURSE = Alternative URL used for Concourse [default: concourse.cincan.io]"
echo "EXTERNAL_URL_GITLAB = External URL used for Gitlab [default: 172.20.0.5]"
echo "ALT_EXTERNAL_URL_GITLAB = Alternative URL used for Gitlab [default: gitlab.cincan.io]"
echo "EXTERNAL_URL_REGISTRY = External URL used for Docker private registry [default: 172.20.0.6]"
echo "ALT_EXTERNAL_URL_REGISTRY = Alternative URL used for Docker private registry [default: registry.cincan.io]"
echo "BUILD_PATH = Path whereto CinCan environment configuration files will be generated [default: /opt/cincan]"
echo ""
exit 0
......@@ -112,6 +118,12 @@ for val in "${ENVS[@]}"; do
if [[ ${val%=*} =~ ^(ALT_EXTERNAL_URL_GITLAB|alt_external_url_gitlab)$ ]]; then
ALT_EXTERNAL_URL_GITLAB=${val##*=}
fi
if [[ ${val%=*} =~ ^(EXTERNAL_URL_REGISTRY|external_url_registry)$ ]]; then
EXTERNAL_URL_REGISTRY=${val##*=}
fi
if [[ ${val%=*} =~ ^(ALT_EXTERNAL_URL_REGISTRY|alt_external_url_registry)$ ]]; then
ALT_EXTERNAL_URL_REGISTRY=${val##*=}
fi
if [[ ${val%=*} =~ ^(BUILD_PATH|build_path)$ ]]; then
BUILD_PATH=${val##*=}
fi
......@@ -148,7 +160,7 @@ if [[ -z "$USER_NAME" || -z "$PASSWORD" || -z "$DNS" || -z "$DATABASE" ]]; then
DNS=$CHOOSE_DNS
break
else
if [ "$REPLY" = "q" ] || [ "$REPLY" = "Q" ]; then exit; fi
if [ "$REPLY" = "q" ] || [ "$REPLY" = "Q" ]; then exit; fi
echo -e "$ERROR invalid option"
fi
done
......@@ -208,9 +220,9 @@ cat > "${BUILD_PATH}/build/config.json" << EOL
EOL
fi
# Generate concourse.env environment file with user details
# Generate concourse.env environment file with user details
cat > "${BUILD_PATH}/build/concourse.env" << EOL
# concourse-web
......@@ -219,6 +231,7 @@ CONCOURSE_POSTGRES_HOST=db
CONCOURSE_POSTGRES_USER=$USER_NAME
CONCOURSE_POSTGRES_PASSWORD=$PASSWORD
CONCOURSE_POSTGRES_DATABASE=$DATABASE
CONCOURSE_POSTGRES_PORT=5432
CONCOURSE_MAIN_TEAM_LOCAL_USER=$USER_NAME
CONCOURSE_ADD_LOCAL_USER=$USER_NAME:$PASSWORD
CONCOURSE_SESSION_SIGNING_KEY=${BUILD_PATH}/keys/session_signing_key
......@@ -246,15 +259,26 @@ GITLAB_ROOT_PASSWORD=$PASSWORD
# path to keys
KEYS_PATH=$BUILD_PATH/keys
# registry
REGISTRY_HTTP_SECRET=$(head -c 9 /dev/urandom | openssl base64 -e)
REGISTRY_HTTP_TLS_CERTIFICATE=${BUILD_PATH}/certs/$COMMON_NAME_REGISTRY.crt
REGISTRY_HTTP_TLS_KEY=${BUILD_PATH}/certs/$COMMON_NAME_REGISTRY.key
REGISTRY_AUTH=htpasswd
REGISTRY_AUTH_HTPASSWD_PATH=${BUILD_PATH}/auth/htpasswd
REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm
EOL
# Generate .env environment file for use of docker-compose.yml
cat > "${BUILD_PATH}/build/.env" << EOL
# Image tags
POSTGRES=11.4
CI=${CONCOURSE_VERSION}
GIT=11.9.0-rc7.ce.0
GIT=${GITLAB_VERSION}
# Volume paths
BUILD=${BUILD_PATH}/build
......@@ -263,6 +287,7 @@ CERTS=${BUILD_PATH}/certs
PIPELINES=${BUILD_PATH}/pipelines
CONCOURSE=${BUILD_PATH}/concourse
GITLAB=${BUILD_PATH}/gitlab
REGISTRY=${BUILD_PATH}
# 5.4.1
......@@ -271,9 +296,9 @@ CONCOURSE_TSA_BIND_IP=worker
CONCOURSE_TSA_BIND_PORT=2222
EOL
# Copy the docker-compose.yml and setup-pipeline.sh scripts to build directory
# Copy the docker-compose.yml, setup-private-registry.sh and setup-pipeline.sh scripts to build directory
cp ./docker-compose.yml ./setup-pipeline.sh "${BUILD_PATH}/build"
cp ./docker-compose.yml ./setup-private-registry.sh ./setup-pipeline.sh "${BUILD_PATH}/build"
# Generate certificates
......@@ -286,7 +311,7 @@ if [ -z "$CC_TLS_PUBKEY" ]; then
-nodes \
-x509 \
-days 365 \
-subj '/CN=concourse.cincan.io/O=CinCan/C=FI' \
-subj '/CN='${COMMON_NAME_CONCOURSE}'/O=CinCan/C=FI' \
-extensions san \
-config <(echo "[req]"; echo distinguished_name=req; echo "[san]"; echo subjectAltName=DNS:"${ALT_EXTERNAL_URL_CONCOURSE}",IP:"${EXTERNAL_URL_CONCOURSE}") \
-sha256 \
......@@ -299,19 +324,36 @@ if [ -z "$CC_TLS_PUBKEY" ]; then
-nodes \
-x509 \
-days 365 \
-subj '/CN=gitlab.cincan.io/O=CinCan/C=FI' \
-subj '/CN='${COMMON_NAME_GITLAB}'/O=CinCan/C=FI' \
-extensions san \
-config <(echo "[req]"; echo distinguished_name=req; echo "[san]"; echo subjectAltName=DNS:"${ALT_EXTERNAL_URL_GITLAB}",IP:"${EXTERNAL_URL_GITLAB}") \
-sha256 \
-keyout "${BUILD_PATH}/certs/${COMMON_NAME_GITLAB}.key" \
-out "${BUILD_PATH}/certs/${COMMON_NAME_GITLAB}.crt" > /dev/null 2>&1
openssl req \
-newkey rsa:4096 \
-new \
-nodes \
-x509 \
-days 365 \
-subj '/CN='${COMMON_NAME_REGISTRY}'/O=CinCan/C=FI' \
-extensions san \
-config <(echo "[req]"; echo distinguished_name=req; echo "[san]"; echo subjectAltName=DNS:"${ALT_EXTERNAL_URL_REGISTRY}",IP:"${EXTERNAL_URL_REGISTRY}") \
-sha256 \
-keyout "${BUILD_PATH}/certs/${COMMON_NAME_REGISTRY}.key" \
-out "${BUILD_PATH}/certs/${COMMON_NAME_REGISTRY}.crt" > /dev/null 2>&1
else # premade cert paths passed in env vars
if [[ -f "$CC_TLS_PUBKEY" && -f "$CC_TLS_PRIVKEY" ]]; then
cp "$CC_TLS_PUBKEY" "${BUILD_PATH}/certs/${COMMON_NAME_CONCOURSE}.crt"
cp "$CC_TLS_PRIVKEY" "${BUILD_PATH}/certs/${COMMON_NAME_CONCOURSE}.key"
cp "$CC_TLS_PUBKEY" "${BUILD_PATH}/certs/${COMMON_NAME_GITLAB}.crt"
cp "$CC_TLS_PRIVKEY" "${BUILD_PATH}/certs/${COMMON_NAME_GITLAB}.key"
cp "$CC_TLS_PUBKEY" "${BUILD_PATH}/certs/${COMMON_NAME_REGISTRY}.crt"
cp "$CC_TLS_PRIVKEY" "${BUILD_PATH}/certs/${COMMON_NAME_REGISTRY}.key"
else
echo "CC_TLS_PUBKEY env var provided but one of the pubkey/privkey files is missing"
exit 1
......@@ -331,7 +373,23 @@ nginx['ssl_certificate'] = '/etc/ssl/certs/gitlab/$COMMON_NAME_GITLAB.crt';
nginx['ssl_certificate_key'] = '/etc/ssl/certs/gitlab/$COMMON_NAME_GITLAB.key';
EOF
# Create README and inform user
## Create certs and credentials for private registry
echo -e "$CHECK Added certificate for Docker private registry" "${BUILD_PATH}"/certs/"${COMMON_NAME_REGISTRY}".crt /etc/docker/certs.d/"${EXTERNAL_URL_REGISTRY}":5000/ca.crt
if [ ! -d /etc/docker/certs.d/"${EXTERNAL_URL_REGISTRY}":5000 ]; then mkdir -p /etc/docker/certs.d/"${EXTERNAL_URL_REGISTRY}":5000; fi
cp "${BUILD_PATH}"/certs/"${COMMON_NAME_REGISTRY}".crt /etc/docker/certs.d/"${EXTERNAL_URL_REGISTRY}":5000/ca.crt
if [ ! -d "${BUILD_PATH}"/auth ]; then mkdir "${BUILD_PATH}"/auth; fi
if htpasswd -Bbc "${BUILD_PATH}"/auth/htpasswd "${USER_NAME}" "${PASSWORD}"; then
echo -e "$CHECK passwd file created for private registry"
else
echo -e "$ERROR Htpasswd failed, is it installed? (Install: apache2-utils for Debian based or httpd-tools for RedHat derived OS)"
exit 1
fi
## Create README and inform user
cat > "${BUILD_PATH}/build/README" << EOL
Now you should be able to build the pilot environment using following commands on host system:
......@@ -341,9 +399,14 @@ Now you should be able to build the pilot environment using following commands o
.. or if you want to use dns-proxy
1. cd ${BUILD_PATH}/build
2. docker-compose up -d db ci worker gitlab dns
.. or with Docker private registry
2. docker-compose up -d db ci worker gitlab reg
3. sudo bash ./setup-private-registry.sh
Use and login to the services on the pilot environment with following credentials:
* username: $USER_NAME
* password: $PASSWORD
......@@ -355,9 +418,12 @@ Local Gitlab credentials:
Services:
* Concourse CI: https://${EXTERNAL_URL_CONCOURSE}:443 or https://${ALT_EXTERNAL_URL_CONCOURSE} (if proxy is enabled)
* CinCan Gitlab: https://${EXTERNAL_URL_GITLAB}:5443 or https://${ALT_EXTERNAL_URL_GITLAB} (if proxy is enabled)
* Private registry https://${EXTERNAL_URL_REGISTRY}:5000 (if enabled)
EOL
echo -e "$CHECK cat ${BUILD_PATH}/build/README"
echo ""
cat "${BUILD_PATH}/build/README"
echo ""
......@@ -28,24 +28,22 @@ x-logging: &logging
services:
firefox:
reg:
<<: *env
<<: *logging
image: jlesage/firefox:${FIREFOX_VERSION}
container_name: firefox.cincan.io
hostname: firefox.cincan.io
build:
context: .
args:
- shm-size=2g
image: registry:2
container_name: registry.cincan.io
hostname: registry.cincan.io
ports:
- '5800:5800'
- 5000:5000
volumes:
- '${FIREFOX}:/config:rw'
- /dev/shm:/dev/shm
- '${REGISTRY}/data:/var/lib/registry'
- '${REGISTRY}/auth:/opt/cincan/auth'
- '${KEYS}:${KEYS}'
- '${CERTS}:${CERTS}'
networks:
cincan:
ipv4_address: 172.20.0.8
ipv4_address: 172.20.0.6
db:
<<: *env
......
......@@ -10,56 +10,81 @@ echo ""
echo "Continuous Integration for the Collaborative Analysis of Incidents"
echo ""
PIPELINE=$1
CHECK="\e[32m[+]\e[0m"
ERROR="\e[0;91m[x]\e[0m"
WARNING="\e[33m[!]\e[0m"
PIPELINE=$1 # Pipeline name can be set as first argument
TOKEN=$2 # You can set a previously created token as the second argument
# Use current path
BUILD_PATH=${PWD%/build*}
# Check if user is root
if [ ! "${UID}" -eq 0 ] ;then
echo "Please run the script as root"; exit 0
fi
if [ ! "${UID}" -eq 0 ] ;then echo "Please run the script as root"; exit 0; fi
# Clone pipelines.git to local machine
echo -e "$CHECK Cloning the pipelines.git"
[ ! -d "pipelines" ] && git clone https://gitlab.com/CinCan/pipelines.git || echo -e "$WARNING" "Folder already exists"
[ ! -d "pipelines" ] && git clone https://gitlab.com/CinCan/pipelines.git || echo "Folder already exists, updating..." && cd pipelines && git pull && cd ..
# Get available pipelines
PIPELINES=$(find pipelines/ \( -name "setup.sh" \) | grep -oP './\K.*?(?=/)')
PS3="Your choice: "
PIPELINES=$(find pipelines/ \( -name "setup.json" \) | grep -oP './\K.*?(?=/)')
# Help, list available pipelines
[ "$1" = "-h" ] || [ "$1" = "--help" ] &&
echo -e "\nThis script executes setup.sh from pipelines/<PIPELINE>" &&
echo -e "Usage: sudo ./setup-pipeline.sh <PIPELINE>" &&
echo -e "\nAvailable pipelines: \n$PIPELINES" &&
exit
exit 0
# Pipeline selection menu
[ "$1" = "" ] &&
echo -e "\nAvailable pipelines: \n$PIPELINES" &&
i=0 &&
while [ $i -lt "$(echo "$PIPELINES" | wc -l)" ]; do
i=$((i+1))
OPTIONS[i]=$(echo "$PIPELINES" |awk NR==$i)
done &&
OPTIONS[$((i+1))]="Quit" &&
select CHOSEN_PIPELINE in "${OPTIONS[@]}"; do
if [[ -n "$CHOSEN_PIPELINE" ]]; then
PIPELINE="$CHOSEN_PIPELINE"
break
else
if [ "$REPLY" = "q" ] || [ "$REPLY" = "Q" ]; then exit;fi
echo -e "$ERROR invalid option"
fi
done
if [ "$1" == "" ]; then
# Get pipelines with setup.sh, and possible private registry versions
echo -e "\nAvailable pipelines"
i=0 && j=0
while [ "$i" -lt "$(echo "$PIPELINES" | wc -l)" ]; do
i=$((i+1)) && j=$((j+1))
OPTIONS[j]=$(echo "$PIPELINES" |awk NR==$i)
if [ -f pipelines/${OPTIONS[$j]}/pipeline-private-registry.yml ]; then
j=$((j+1))
REGISTRY_VERSION="Private registry version"
OPTIONS[j]=$(echo "$PIPELINES" |awk NR=="$i")" $REGISTRY_VERSION"
fi
done
# Add quit option
OPTIONS[$((j+1))]="Quit"
# Menu
PS3="Your choice: "
select CHOSEN_PIPELINE in "${OPTIONS[@]}"; do
if [[ -n "$CHOSEN_PIPELINE" ]]; then
break
else
if [ "$REPLY" = "q" ] || [ "$REPLY" = "Q" ]; then exit; fi
echo -e "$ERROR invalid option"
fi
done
fi
# Check if private registry was chosen
if [[ "${CHOSEN_PIPELINE}" == *"Private"* ]]; then
PIPELINE="${CHOSEN_PIPELINE/" $REGISTRY_VERSION"/}"
else
if ! [ $1 ]; then PIPELINE="${CHOSEN_PIPELINE}"; fi
REGISTRY_VERSION=""
fi
# Change directory and launch pipeline setup.sh, if exists
[ "$CHOSEN_PIPELINE" != "Quit" ] &&
cd pipelines || exit &&
[ -d "$PIPELINE" ] && ( cd "$PIPELINE" && echo -e "$CHECK Run $PIPELINE/setup.sh" && [ -f "./setup.sh" ] && ./setup.sh "$BUILD_PATH" "$2" \
|| echo -e "$ERROR $PIPELINE/setup.sh not found or script interrupted" ) || echo -e "$ERROR Pipeline $PIPELINE not found"
[ "$PIPELINE" != "Quit" ] &&
cd pipelines || exit &&
[ -d "$PIPELINE" ] && ( echo -e "$CHECK Run setup.sh -e $BUILD_PATH -p $PIPELINE" \
&& [ -f "./setup.sh" ] && ./setup.sh -e "$BUILD_PATH" -t "$TOKEN" -p "$PIPELINE" "$REGISTRY_VERSION" \
|| echo -e "$ERROR ...script interrupted..." ) || echo -e "$ERROR Pipeline $PIPELINE not found"
#!/bin/bash
# Variable definitions
CHECK="\e[32m[+]\e[0m"
PRIVATE_REGISTRY=172.20.0.6
# Whiptail colors
export NEWT_COLORS='
backtitle=,red
root=,gray
border=black,lightgray
window=,lightgray
shadow=gray,black
title=red,lightgray
checkbox=white,gray
actcheckbox=green,red
button=lightgray,red
actbutton=red,lightgray'
# Check if user is root
if [ ! "${UID}" -eq 0 ] ;then
echo "Please run the script as root"; exit 0
fi
# Welcome banner
echo " _______ ______ "
echo " / ____(_)___ / ____/___ _____ "
echo " / / / / __ \/ / / __ \`/ __ \ "
echo "/ /___/ / / / / /___/ /_/ / / / / "
echo "\____/_/_/ /_/\____/\__,_/_/ /_/ "
echo ""
echo "Continuous Integration for the Collaborative Analysis of Incidents"
echo ""
echo "* Private registry setup *"
echo ""
# Functions
# Get/ask for Docker username (because root is not necessarily the Docker user), cincan credentials, and login to registry
function getCredentials() {
# Get docker user's name from /etc/group (tries the last user mentioned in docker group)
if [ ! "$DOCKER_USERNAME" ]; then
DOCKER_USERNAME=$(cat /etc/group |grep docker: |tail -1 | awk -F "," '{print $NF }' |awk -F ":" '{print $NF }')
fi
echo "Login to private registry:"
sudo su "$DOCKER_USERNAME" -c "docker login 172.20.0.6:5000" || { echo "Could not resolve docker user."; \
echo "Type in your username in docker group (q to quit): "; read -r DOCKER_USERNAME; \
if [ "$DOCKER_USERNAME -eq 'q'" ];then exit 0;else getCredentials; fi }
}
# Clone tools repository (or update if exists)
function cloneRepo() {
echo -e "$CHECK Cloning the tools repository"
if [ ! -d "tools" ];then
git clone https://gitlab.com/CinCan/tools.git
cd tools || exit
else
cd tools || exit
git pull
fi
}
# User menu
# Create image selection options
function createMenuOptions() {
TOOLS_LIST=$(ls -d -- */)
TOOLS_COUNT=$(echo "$TOOLS_LIST" |grep -o "/" |wc -l)
i=1; while
OPTION="$(echo $TOOLS_LIST |cut -d " " -f $i)"
OPTION="${OPTION::-1}"
if [ -f "$OPTION/Dockerfile" ]; then
if cat "$OPTION/Dockerfile" |head -n 10 |grep -q microsoft; then
OS="Windows"
else
OS="Linux"
fi
OPTIONS="${OPTIONS} ${OPTION} ${OS} OFF"
fi
i=$((i+1))
[ $i -lt $((TOOLS_COUNT+1)) ]
do :;done
}
# Menu to select images
function selectImages() {
TOOLS_TO_INSTALL=$(whiptail --title "Cincan private registry setup" --checklist --fb --backtitle \
" CinCan - Continuous Integration for the Collaborative Analysis of Incidents " \
"There are $TOOLS_COUNT images available. Choose the images to install:" \
30 80 20 $OPTIONS 3>&1 1>&2 2>&3)
if [ ! "$TOOLS_TO_INSTALL" ]; then echo "Cancelled"; exit; fi
TOOLS_TO_INSTALL="${TOOLS_TO_INSTALL//\"}"
}
# Menu for selecting installation method
function installationMethod() {
INSTALLATION_METHOD=$(whiptail --title "Cincan private registry setup" --fb --menu "Choose installation method" --backtitle \
" CinCan - Continuous Integration for the Collaborative Analysis of Incidents " \
15 60 4 \
"1" "Build images from dockerfiles" \
"2" "Download images from hub.docker.com" \
"3" " ⇦ RETURN TO IMAGE SELECTION MENU" 3>&1 1>&2 2>&3)
case "$INSTALLATION_METHOD" in
"")
echo Cancelled; exit 0
;;
1)
echo "Building images from the dockerfiles"
;;
2)
echo "Downloading selected images from hub.docker.com"
;;
3)
# Set previous selections to be selected also when returning to menu
i=1; while
TOOL_NUMBER="\$"$i
TOOL_ON=$(echo "$TOOLS_TO_INSTALL" |awk "{print $TOOL_NUMBER}")
TOOL_OS=$(echo "$OPTIONS" |grep -Po -- "$TOOL_ON \K\w*")
OPTIONS="${OPTIONS//$TOOL_ON $TOOL_OS OFF/$TOOL_ON $TOOL_OS ON}"
i=$((i+1))
[ $i -lt $((TOOLS_COUNT+2)) ]
do :;done
selectImages
installationMethod
;;
esac
}
# Build the images
function buildImages(){
echo -e "$CHECK Building Dockerfiles..."
i=1
while
CURRENT_TOOL="$(echo $TOOLS_TO_INSTALL |cut -d " " -f $i)"
echo -e "\n$CHECK Building tool $i/$TOOLS_COUNT_TO_INSTALL: $CURRENT_TOOL\n"
cd "$CURRENT_TOOL" || exit
echo "$CURRENT_TOOL" >> times
sudo su "$DOCKER_USERNAME" -c "docker build . -t cincan/$CURRENT_TOOL"
cd ..
i=$((i+1))
[ $i -lt $((TOOLS_COUNT_TO_INSTALL+1)) ]
do :;done
}
# Pull images
function pullImages(){
echo -e "$CHECK Downloading images from Docker hub"
i=1
while
CURRENT_TOOL=$(echo "$TOOLS_TO_INSTALL" |cut -d " " -f $i)
echo -e "\n$CHECK Downloading tool $i/$TOOLS_COUNT_TO_INSTALL: $CURRENT_TOOL\n"
cd "$CURRENT_TOOL" || exit
echo "$CURRENT_TOOL" >> times
docker pull cincan/"$CURRENT_TOOL"
cd ..
i=$((i+1))
[ $i -lt $((TOOLS_COUNT_TO_INSTALL+1)) ]
do :;done
}
# Push images to the private registry
function pushImages() {
echo -e "$CHECK Pushing images to Docker private registry"
i=1
while
CURRENT_TOOL=$(echo "$TOOLS_TO_INSTALL" |cut -d " " -f $i)
docker tag cincan/"$CURRENT_TOOL" "$PRIVATE_REGISTRY:5000/$CURRENT_TOOL"
echo -e "$CHECK tagged cincan/$CURRENT_TOOL as $PRIVATE_REGISTRY:5000/$CURRENT_TOOL"
sudo su "$DOCKER_USERNAME" -c "docker push $PRIVATE_REGISTRY:5000/$CURRENT_TOOL"
echo -e "$CHECK pushed $CURRENT_TOOL to $PRIVATE_REGISTRY:5000/"
docker rmi -f cincan/"$CURRENT_TOOL"
docker rmi -f "$PRIVATE_REGISTRY:5000/$CURRENT_TOOL"
echo -e "$CHECK removed $CURRENT_TOOL from host"
i=$((i+1))
[ $i -lt $((TOOLS_COUNT_TO_INSTALL+1)) ]
do :;done
}
# Execute functions
getCredentials
cloneRepo
createMenuOptions
selectImages
installationMethod
TOOLS_COUNT_TO_INSTALL=$(echo "$TOOLS_TO_INSTALL" |wc -w)
if [ "$INSTALLATION_METHOD" = 1 ]; then
buildImages
else
pullImages
fi
pushImages
echo -e "\n\n$CHECK \e[32mPrivate registry is ready.\e[0m"
echo -e "\nTo get the list of images in the private registry, type:"
echo "curl -X GET https://<USERNAME>:<PASSWORD>@172.20.0.6:5000/v2/_catalog --insecure"
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment