Commit 358fb6f7 authored by Alvaro Hernandez's avatar Alvaro Hernandez
Browse files

Merge branch '202101-eks_on_firecracker' into 'master'

[202101-eks_on_firecracker] New blog post

See merge request !7
parents 3f9fe567 09859d21
......@@ -8,3 +8,4 @@ target/
202012-firecracker_cloud_image_automation/keypairs/
202012-firecracker_cloud_image_automation/disks/
202012-firecracker_cloud_image_automation/images/
202012-firecracker_cloud_image_automation/ansible/inventories/eks/hosts.yaml
......@@ -45,7 +45,7 @@ function launch_vm() {
local instance_number=$1
local tmpfile=/tmp/.$RANDOM-$RANDOM
local socket=$FIRECRACKER_SOCKET.$instance_number
local instance_id="id-"$RANDOM-$RANDOM
local instance_id=`printf "id%05d%05d" $RANDOM $RANDOM`
local log_file=/tmp/.$instance_id.log
# Start firecracker daemon
......@@ -65,7 +65,11 @@ function launch_vm() {
done
# VM config
firecracker_http_file PUT 'machine-config' conf/firecracker/instance-config.json
cat conf/firecracker/instance-config.json | \
./tmpl.sh __INSTANCE_VCPUS__ $VM_VCPUS | \
./tmpl.sh __INSTANCE_RAM_MB__ $(( $VM_RAM_GB * 1024 )) \
> $tmpfile
firecracker_http_file PUT 'machine-config' $tmpfile
# Drives
mkdir -p disks
......@@ -133,7 +137,7 @@ function launch_vm() {
# Start VM
firecracker_http_file PUT 'actions' conf/firecracker/instance-start.json
[ $? -eq 0 ] && echo -e "Instace $instance_id started. SSH with ssh -i $KEYPAIR_DIR/$DEFAULT_KP fc@$instance_ip\n"
[ $? -eq 0 ] && echo "Instace $instance_id started. SSH with ssh -i $KEYPAIR_DIR/$DEFAULT_KP fc@$instance_ip"
}
......@@ -141,5 +145,8 @@ function launch_vm() {
# Main
for i in `seq 1 $NUMBER_VMS`
do
launch_vm $i
(
launch_vm $i
)&
done
wait
#!/bin/bash
source variables
ANSIBLE_INVENTORY_DIR=ansible/inventories/eks
ANSIBLE_INVENTORY=$ANSIBLE_INVENTORY_DIR/hosts.yaml
KUBECTL_CONFIG_ANSIBLE_OUTPUT=/tmp/eks_config.yaml
# Static part of the inventory
mkdir -p $ANSIBLE_INVENTORY_DIR
cat << EOF > $ANSIBLE_INVENTORY
all:
vars:
ansible_ssh_common_args: '-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'
ansible_user: fc
ansible_ssh_private_key_file: keypairs/kp
EOF
# Dynamically generate Ansible inventory file
## Nodes
cat <<EOF >> $ANSIBLE_INVENTORY
eks_nodes:
hosts:
EOF
tokens=""
for i in `seq 2 $NUMBER_VMS`
do
token=`echo $( date +%s )$RANDOM | md5sum | cut -b 1-32`
cat <<EOF >> $ANSIBLE_INVENTORY
${VMS_NETWORK_PREFIX}.$(( $i + 1 )):
token: $token
EOF
tokens="$tokens $token"
done
cat <<EOF >> $ANSIBLE_INVENTORY
eks_master:
hosts:
${VMS_NETWORK_PREFIX}.2:
eks_config_outfile: $KUBECTL_CONFIG_ANSIBLE_OUTPUT
tokens:
EOF
for token in $tokens
do
cat <<EOF >> $ANSIBLE_INVENTORY
- $token
EOF
done
# Call ansible-playbook for eks cluster
ansible-playbook -i ansible/inventories/eks ansible/eks.yaml
# Copy or merge generated kubectl config to ~/.kube/config
if [ -f ~/.kube/config ]
then
tmpfile=/tmp/.$RANDOM-$RANDOM
cp ~/.kube/config $tmpfile
KUBECONFIG=$tmpfile:$KUBECTL_CONFIG_ANSIBLE_OUTPUT kubectl config view --flatten > ~/.kube/config
else
mkdir -p ~/.kube
cp $KUBECTL_CONFIG_ANSIBLE_OUTPUT ~/.kube/config
fi
rm $KUBECTL_CONFIG_ANSIBLE_OUTPUT
......@@ -19,3 +19,4 @@ sudo ip link delete $FIRECRACKER_BRIDGE
rm -rf disks
rm -rf images
rm -rf keypairs
[ -f ansible/inventories/eks/hosts.yaml ] && rm ansible/inventories/eks/hosts.yaml
......@@ -12,8 +12,8 @@ cloud-init.
Edit the `variables` file to suit your needs.
Follow the scripts in the specified order. Some basic Linux CLI tools are
required like `curl`, `jq`, `tar` with `xz` support, `ssh-keygen`, `iptables`
and possibly others.
required like `curl`, `jq`, `tar` with `xz` support, `ssh-keygen`, `iptables`,
the `binutils` package and possibly others.
A Linux bridge will be created in the host and will be NATTed to the main
Internet egress device on the host to give connectivity to the created VMs,
......@@ -25,3 +25,28 @@ Modify scripts at your convenience. It has only been tested with `bionic`
[Ubuntu cloud images](https://cloud-images.ubuntu.com/), but in principle should
work with minor modifications with any Linux cloud image that supports
initialization via `cloud-init`.
# Optional: create EKS cluster on Firecracker VMs
Once the VMs have been created, you may want to run script
`05-install_eks_via_ansible.sh`. This requires you to have Ansible (2.9+)
installed (see Ansible source code in `ansible/` folder). It will create an
[EKS-D compatible](https://snapcraft.io/eks) cluster and will generate/add a
valid `~/.kube/config` configuration in your (local) environment (provided you
have `kubectl` installed). After running this script you should be able to
`kubectl cluster-info` and `kubectl get nodes` successfully.
Please review the `variables` file to ensure you will be creating the desired
number of nodes. At least `4GB` of RAM are recommended per VM.
You can obviously chain commands. While the VM creation script does not know
exactly when the `user-data` part is finished (as it happens within the VM), it
is quite fast, and usually (YMMV) waiting a bit works well. So once scripts `01`
to `03` are run, you may do something like:
```sh
time (./04-launch_vms.sh ; sleep 1m; ./05-install_eks_via_ansible.sh )
```
to create the VMs and EKS cluster in a single line.
- hosts: eks_master
roles:
- eks_common
- eks_master
- hosts: eks_nodes
roles:
- eks_common
- eks_node
- hosts: eks_master
roles:
- eks_config
- name: apt-get update
apt:
update_cache: yes
cache_valid_time: 3600
become: true
- name: gather kernel version
command: uname -r
register: uname_r
- name: install kernel image, required to load module ip_tables
apt:
name: "linux-image-{{ uname_r.stdout }}"
state: present
become: true
- name: install eks snap
command: snap install --classic --edge --color=never eks
become: true
- name: add user to eks group
command: usermod -a -G eks fc
become: true
- name: wait for eks to be ready
shell: |
eks status --wait-ready
become: true
- name: install kubectl snap
command: snap install --classic --color=never kubectl
become: true
- name: ensure /home/fc/.kube
file:
path: /home/fc/.kube
state: directory
owner: fc
group: fc
mode: u+rwx,g-rwx,o-rwx
- name: generate kubectl config from eks
shell: |
eks config |tee -a /home/fc/.kube/config
register: eks_config
become: true
- name: copy eks config to host system
local_action: copy content={{ eks_config.stdout }} dest={{ eks_config_outfile }}
- name: call eks add-node with the tokens per node
command: eks add-node --token {{ item }}
with_items: "{{ tokens }}"
become: true
- name: join master node
command: eks join {{ groups['eks_master'][0] }}:25000/{{ token }}
become: true
throttle: 1 # Avoid potential problems with K8s rate limiting
{
"vcpu_count": 2,
"mem_size_mib": 1024,
"vcpu_count": __INSTANCE_VCPUS__,
"mem_size_mib": __INSTANCE_RAM_MB__,
"ht_enabled": false
}
NUMBER_VMS=4
UBUNTU_VERSION=bionic
NUMBER_VMS=3
VM_VCPUS=2
VM_RAM_GB=4
IMAGE_SIZE=8G
FIRECRACKER=`which firecracker`
FIRECRACKER_PID_DIR=/tmp/.firecracker
......@@ -8,9 +11,11 @@ FIRECRACKER_BRIDGE=fcbr0
EGRESS_IFACE=`ip route get 8.8.8.8 |grep uid |sed 's/.* dev \([^ ]*\) .*/\1/'` # or hardcode to specific device if wanted
VMS_NETWORK_PREFIX=172.26.0 # for scripts simplicity, it is always assumed here a /24 network
IMAGE_SIZE=8G
UBUNTU_VERSION=bionic
IMAGE_ROOTFS=images/$UBUNTU_VERSION/$UBUNTU_VERSION.rootfs
KERNEL_IMAGE=images/$UBUNTU_VERSION/$UBUNTU_VERSION.vmlinux
INITRD=images/$UBUNTU_VERSION/$UBUNTU_VERSION.initrd
KEYPAIR_DIR=keypairs
DEFAULT_KP=kp
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment