Commit 033ceaa1 authored by Nick Busey's avatar Nick Busey

Adding easy HomelabOS backup and restore, adding NAS support

parent 0d1e5aa5
.PHONY: deploy build restore
# Deploy HomelabOS
deploy:
ansible-playbook -i hosts -t homelabos homelabos.yml
......@@ -5,3 +7,7 @@ deploy:
# Build the HomelabOs Documentation - Requires mkdocs with the Material Theme
build:
mkdocs build
# Restore a server with the most recent backup. Assuming Backups were running.
restore:
ansible-playbook -i hosts restore.yml
......@@ -8,7 +8,7 @@ Your very own offline-first open-source data-center!
## Summary
A set of Ansible scripts to configure a Docker based Homelab server with all sorts of goodies.
A set of Ansible scripts to configure a Docker based Homelab server with all sorts of goodies. Following the unix philosophy we gather together many specific tools to build the exact end result desired.
## Goals
......@@ -17,12 +17,12 @@ To make it easy for anyone to own all their data in an easy and secure way, with
## Features
* One command depyloment
* Automated Backups
* Easy Restore
## Planned Features
* Automated HTTPS endpoints
* Automated Backups
* Easy Restore
* Dynamic DNS Support
* Automated Apple Health Import
* Automated LDAP Sync
......@@ -46,7 +46,6 @@ To make it easy for anyone to own all their data in an easy and secure way, with
* [Huginn](https://github.com/huginn/huginn) - Open Source IFTTT replacement. Create agents that monitor and act on your behalf
* [InfluxDB](https://www.influxdata.com/time-series-platform/influxdb/) - Time series data storage
* [NextCloud](https://nextcloud.com/) - Private Cloud Storage, Calendar, Contacts, etc.
* [Pi-hole](https://pi-hole.net/) - Ad blocking
* [Paperless](https://github.com/danielquinn/paperless) - Document management
* [Portainer](https://www.portainer.io/) - Easy Docker management
* [Sonerezh](https://www.sonerezh.bzh/) - Music streaming and library management
......@@ -58,6 +57,7 @@ To make it easy for anyone to own all their data in an easy and secure way, with
* BitWarden - Password manager
* BulletNotes - Note taking knowledgebase with kanban and calendar functionality.
* [Pi-hole](https://pi-hole.net/) - Ad blocking
* OwnTracksRecorder - https://github.com/owntracks/recorder
## Requirements
......
# Backups
# HomelabOS Backups
HomelabOS backs itself up to any S3 compatible storage out of the box using [Restic](https://restic.net/).
HomelabOS automatically backs itself in a smart, incremental, encrypted way to any S3 compatible storage provider, using [Restic](https://restic.net/).
## Set up your own S3
This is in addition to and separate from the Backup service included within HomelabOS, which backs up your personal computers to the HomelabOS servers.
## Get access to an S3 Bucket
Ideally you want your backups to be offsite, so an S3 bucket is not set up by default for you on your HomlabOS machine. Since we use encrypted backups, you can easily have a friend or enemy host an S3 server for you, and as long as they don't get your backup password, everything will be safe. We recommend generating a very secure password for your backup password.
Or you could pay Amazon and use their S3 service, if you're into that sort of thing.
## Set up your own S3
We recommend Minio.
......@@ -13,10 +21,16 @@ minio server /data
Create a bucket called `homelabos`.
Of course keep in mind when self hosting your own S3, you likely want to keep good backups of the S3 data files as well.
## Configure the Backup Service
Set your S3 path and keys in `host_vars/myserver`.
If you use minio for your S3 hosting, your s3_path should look something like `s3:http://192.168.1.212:9000/homelabos/`
Once these are set, HomelabOS will back up all it's core data every night.
Once these are set, HomelabOS will back up all it's core data every night at 4 AM.
## Restoring Backups
To back up your NAS and data on it is beyond the scope of this document.
\ No newline at end of file
Let's say your machine gets wiped, or you want to migrate to a new machine. To restore your most recent backup, simply run `make restore`. Then proceed with the normal install step of running `make` and boom, you have all your data back with a fully working HomelabOS setup.
\ No newline at end of file
......@@ -23,7 +23,6 @@ darksky_key:
latitude:
longitude:
<<<<<<< Updated upstream
timezone: America/Denver
# Change to https to enable LetsEncrypt
......@@ -33,9 +32,19 @@ protocol: http
openvpn_provider: PIA
openvpn_username: username
openvpn_password: password
=======
# S3 Server Information
# NAS Stores
nas_backups:
nas_music:
nas_movies:
nas_tv:
nas_downloads:
nas_documents:
# S3 Backup Server Information (More information can be found in the Backups section of the HomelabOS Documentation)
s3_path:
s3_access_key:
s3_secret_key:
>>>>>>> Stashed changes
s3_backup_password:
# Backup every day at 4:00 AM
s3_backup_cron: 0 4 * * *
\ No newline at end of file
......@@ -4,7 +4,7 @@
path: /var/homelabos
state: directory
- name: Configure docs
- name: Configure HomelabOS Documentation.
copy: src=site dest=/var/homelabos/docs
- name: Configure Telegraf.
......@@ -13,24 +13,13 @@
- name: Configure Traefik.
template: src=traefik.toml dest=/var/homelabos/traefik/traefik.toml
- name: Configure HomelabOS Documentation
template: src=mkdocs.yml dest=/var/homelabos/docs/mkdocs.yml
- name: Configure HomelabOS Documentation
template:
src: '{{ item.src }}'
dest: /var/homelabos/docs/docs/{{ item.path }}
force: yes
with_filetree: docs/
when: item.state == 'file'
- name: Configure HomelabOS systemd service.
template: src=homelabos.service dest=/etc/systemd/system/homelabos.service
- name: Copy HomelabOS docker-compose.yml file into place.
template:
src: docker-compose.yml
dest: /var/homelabos/docker/docker-compose.yml
dest: /var/homelabos/docker-compose.yml
- name: Pull latest HomelabOS service docker images. (This could take a while the first time, it has to download quite a bit.)
command: docker-compose -f /var/homelabos/docker/docker-compose.yml pull
......
......@@ -77,9 +77,11 @@ services:
image: lobaro/restic-backup-docker:v1.0
environment:
- RESTIC_REPOSITORY={{ s3_path }}
- AWS_ACCESS_KEY_ID={{ s3_access_key }}
- AWS_SECRET_ACCESS_KEY={{ s3_secret_key }}
- RESTIC_PASSWORD={{ default_password }}
- AWS_ACCESS_KEY_ID={{ s3_access_code }}
- AWS_SECRET_ACCESS_KEY={{ s3_secret_code }}
- RESTIC_PASSWORD={{ s3_backup_password }}
- BACKUP_CRON={{ s3_backup_cron }}
- HOSTNAME={{ domain }}
volumes:
- /var/homelabos:/data:ro
......@@ -100,12 +102,13 @@ services:
image: "couchpotato/couchpotato"
volumes:
- /var/homelabos/couchpotato/data:/datadir
- /mnt/Gangsternas/Movies:/media
- {{ nas_movies }}:/media
ports:
- "5050:5050"
restart: always
links:
- "transmission"
- transmission
- jackett
labels:
- "traefik.enable=true"
- "traefik.admin.frontend.rule=Host:couchpotato.{{ domain }}"
......@@ -141,7 +144,8 @@ services:
restart: always
volumes:
- /var/homelabos/emby:/config
- /mnt/Gangsternas:/mnt/Gangsternas
- {{ nas_tv }}:/mnt/tv
- {{ nas_movies }}:/mnt/movies
ports:
- 8096:8096
labels:
......@@ -229,7 +233,6 @@ services:
volumes:
- /var/homelabos/gitea_db:/var/lib/mysql
grafana:
image: grafana/grafana
restart: always
......@@ -302,6 +305,18 @@ services:
ports:
- 8086:8086
jackett:
image: linuxserver/jackett
restart: always
volumes:
- /var/homelabos/jackett/config:/config
- /var/homelabos/jackett/downloads:/downloads
- /etc/localtime:/etc/localtime:ro
environment:
- TZ={{ timezone }}
ports:
- 9117:9117
nextcloud:
image: nextcloud
restart: always
......@@ -330,6 +345,21 @@ services:
- MYSQL_DATABASE=nextcloud
- MYSQL_USER=nextcloud
nzbget:
image: linuxserver/nzbget
ports:
- 6789:6789
environment:
- TZ={{ timezone }}
volumes:
- /var/homelabos/nzbget:/config
- {{ nas_downloads }}:/downloads
labels:
- "traefik.enable=true"
- "traefik.admin.frontend.rule=Host:nzbget.{{ domain }}"
- "traefik.admin.protocol={{ protocol }}"
- "traefik.admin.port=6789"
organizr:
image: lsiocommunity/organizr
restart: always
......@@ -421,10 +451,13 @@ services:
- PGID=1000
- PUID=1000
- TZ={{ timezone }}
links:
- jackett
- transmission
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/homelabos/sonarr/config:/config
- /mnt/Gangsternas/TV:/tv
- {{ nas_tv }}:/tv
- /var/homelabos/sonarr/downloads:/downloads
labels:
- "traefik.enable=true"
......@@ -485,12 +518,12 @@ services:
- "traefik.admin.port=2233"
traefik:
image: traefik # The official Traefik docker image
image: traefik
restart: always
ports:
- "80:80" # The HTTP port
- "443:443" # HTTPS
- "8080:8080" # The Web UI (enabled by --api)
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /var/homelabos/traefik/traefik.toml:/etc/traefik/traefik.toml
......@@ -513,7 +546,7 @@ services:
- /etc/localtime:/etc/localtime:ro
- /var/homelabos/transmission/data:/data
- /var/homelabos/transmission/config:/config
- /mnt/Gangsternas/Downloads:/downloads
- {{ nas_downloads }}:/downloads
- /var/homelabos/transmission/watch:/watch
environment:
- OPENVPN_PROVIDER=PIA
......@@ -524,6 +557,7 @@ services:
- PUID=1000
- TZ={{ timezone }}
- TRANSMISSION_RPC_AUTHENTICATION_REQUIRED=true
# Password = `transmission`
- TRANSMISSION_RPC_PASSWORD="{62b16db87b89a91dd49a5110a7cafc06d20eb4f2wtK6kqPj"
- TRANSMISSION_RPC_USERNAME={{ default_username }}
labels:
......@@ -542,7 +576,7 @@ services:
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/homelabos/urbackup/db/:/var/urbackup
- /mnt/Gangsternas/Backups/urbackup/:/backup
- {{ nas_backups }}:/backup
labels:
- "traefik.enable=true"
- "traefik.admin.frontend.rule=Host:backup.{{ domain }}"
......
---
# Restore HomelabOS from the most recent backup.
- hosts: all
become: True
gather_facts: True
tags: restore
roles:
- restore
---
- name: Be sure HomelabOS is not running.
systemd:
name: homelabos
state: stopped
- name: Clear old HomelabOS local backup if it exists.
file:
path: /var/homelabos.bak
state: absent
- name: Create local backup of HomelabOS data (just in case).
command: mv /var/homelabos /var/homelabos.bak
- name: Create a new HomelabOS folder.
file:
path: /var/homelabos
state: directory
- name: Copy HomelabOS restore.yml file into place.
template:
src: restore.yml
dest: /var/homelabos_restore.yml
- name: Spin up restore utility.
command: "docker-compose -f /var/homelabos_restore.yml -p homelabos_restore up -d"
- name: Restore HomelabOS data. (This could take quite a while depending on how much data you have.)
command: "docker exec homelabosrestore_restore_1 restic restore latest --target /"
- name: Stop restore utility.
command: "docker-compose -f /var/homelabos_restore.yml -p homelabos_restore down"
\ No newline at end of file
version: '3'
services:
restore:
image: lobaro/restic-backup-docker:v1.0
environment:
- RESTIC_REPOSITORY={{ s3_path }}
- AWS_ACCESS_KEY_ID={{ s3_access_code }}
- AWS_SECRET_ACCESS_KEY={{ s3_secret_code }}
- RESTIC_PASSWORD={{ s3_backup_password }}
- BACKUP_CRON={{ s3_backup_cron }}
- HOSTNAME={{ domain }}
volumes:
- /var/homelabos:/data
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment