Skip to content
Snippets Groups Projects
README.adoc 7.74 KiB
Newer Older
= README 
:imagesdir: images/
:toc: left

== qnap-docker-compose

Creates the listed services with docker-compose on a QNAP NAS. Includes configuration examples to expose those services, attach your own domain and include ssl certs. 

- Proxy:
    * traefik incl. lets encrypt {{...}}:80

- Monitoring, logging: 
    * grafana {{...}}:3000
    * graphite {{...}}:8080
    * docker-fritzboxcollectd {{...}}
    * collectd {{...}}
    * statsd {{...}}
    * harvester1 {{...}}
    * log.io {{...}}:28778

- Collaboration: 
    * nextcloud {{...}}:80
    * gogs {{...}}:3000

- Logging:
    * log-file (docker-json)
    
    	docker logs xxx 
    
    * log.io

- Storage:
    * minio {{ ... }}:9000
    * etcd {{ ... }}
    * mariadb {{ ... }}

- Tools:
    * adminer {{ ... }}:8080
    * etcd-viewer: {{ ... }}:8080

    
- Applications
    * operator



== QNAP NAS configurations 

- http port 8079 (instead of 8080)
- https port 442 (instead of 443)
- activate webserver to deactivate port 80 forwarding: port 78, no https
- container station app has to be installed
- a new user (NOT admin) needs to be conigured for ssh access
- container will use a seperate physical network adapter (eth1) 


== Install instructions

NOTE: "{{ ... }}" are variables to be changed by the user 

- dependencies:
    
    * fritz box 
    * QNAP NAS with Container Station (ssh access)
    * optionally some webcams
    * your domains CNAME needs to  show to the static ip (or dyndns) of your trafik container
    * your fritzbox ip is 192.168.178.1 and connected per first lan port
    * your qnap has container station installed
    * "Network and Virtual Switch" application can be used
    * all your domains are free for dns rebind (fritzbox setting "dns-rebind-schutz")
    * qnap is connected per its second ethernet port to the fritzbox

- ssh to nas
- copy the complete repository to a folder on your qnap nas 

    cd /share/Services/{{ ... }}/
    cp ........

- create a network for our services

    docker network create -d qnet --ipam-driver=qnet \
    --ipam-opt=iface=eth1 --subnet=192.168.178.0/23 \
    --gateway=192.168.178.1 qnet-static-eth1

- create directories for data persistency 


    #export SERVICES_HOME=share/Services/{{ ... }}/persist
    mkdir -p ./persist/data/nextcloud
    mkdir -p ./persist/config/nextcloud
    mkdir -p ./persist/data/mysql
    mkdir -p ./persist/data/graphite
    mkdir -p ./persist/config/graphite
    mkdir -p ./persist/data/grafana
    mkdir -p ./persist/config/grafana
    mkdir -p ./persist/data/minio
    mkdir -p ./persist/config/minio
    mkdir -p ./persist/data/etcd
    mkdir -p ./persist/data/gogs


    #stinky change of data folder permissions
    
    chmod 777 ./persist -R

- configure

    cp acme.json.default acme.json
    chmod 600 acme.json
    
    cp traefik.toml.default traefik.toml
    
    cp .env.default .env

- start the services

    docker-compose up -d

== Configuration examples


=== traefik.toml:

	# to switch on the api and the dashboard. only for testing. ... 
	#[api]
	#entryPoint = "traefik"
	#dashboard = true


	# comment the caServer - config line, if done with testing and switching to production
	# now, this is using the test stage ... 
	[acme]
	caServer = "https://acme-staging-v02.api.letsencrypt.org/directory" # comment the caServer - config line, if done with testing and switching to production
	                                                                    # now, this is using the test stage ... 
	email = "a@b.net"  # change
	
	
	# domains are registered with traefik at lets encrypt to gather ssl certificates. 
	[[acme.domains]]
	  main = "collab.{{replace}}.net"
	[[acme.domains]]
	  main = "service.{{replace}}.net"
	[[acme.domains]]
	  main = "nextcloud.{{replace}}.net"
	[[acme.domains]]
	  main = "proxytraefik.{{replace}}.myfritz.net"

	  
	[docker]
	endpoint = "unix:///var/run/docker.sock"
	watch = true


=== docker-compose.yaml:

   # see the grafana container as example
   grafana: 
     image: grafana/grafana:latest # maybe want to set to a specific version?
     container_name: grafana
     hostname: grafana 
     #user: ${ID} # how to define which user it should use?
     restart: always
     mem_limit: ${RES_MEMLIMIT}
     mem_reservation: ${RES_MEMSOFTLIMIT}
     cpuset: ${RES_CPUSET} # qnap nas has 4 cpus. we dont want the containers block the nas
     ports:
       - "3000:3000"
     volumes:
       - data_grafana:/var/lib/grafana
     networks:
       proxy:
         ipv4_address: ${IP_GRAFANA} # needed for now for simple usage within qnaps networking
     labels:
       - "traefik.backend=grafana" # add, if service should be accessed by a domain and ssl-proxied by traefik
       - "traefik.frontend.rule=Host:${DOMAIN_SERVICE}"
       - "traefik.docker.network=proxy"
       - "traefik.port=3000"
       - "traefik.enabled=true" 
       - "collectd_docker_app=grafana" # add, to get this containers stats into monitoring
       - "collectd_docker_task=grafana" 


	#we have a qnap specific network (user defined network) defined manually
	networks:
	  proxy:
	   external:
		name: qnet-static-eth1

		
	#and to store the data permanently, we use volumes	
	volumes:
 	  install_nextcloud: # data will be installed into volume in /docker/bin ..or such.. 
	  data_grafana: # we bind it to a folder, so we know where the data is kept
		driver: local
		driver_opts:
		  type: 'none'
		  o: 'bind'
		  device: '$PWD/persist/data/grafana'
	

=== .env:

	ID=$(id -u) 

	# domains that have a CNAME record linking to our proxy servers ip (or dyndns entry)
	DOMAIN_COLLAB=collab.tigabeatz.net

	# internally used ips
	IP_ETCD={{replace}}
	

	# cheap passwords
	MYSQL_ROOT_PASSWORD=pa$$worD

	# get md5 from htpasswd or http://www.htaccesstools.com/htpasswd-generator/ 
	TRAEFIK_BASIC_PASS=$apr1$0As0jOd2$uRagOVpMPNzdjTLq0fGGO.

	# resource settings
	# after inspection, adapt each container to individual values
	RES_MEMLIMIT=1024MB
	RES_MEMSOFTLIMIT=256MB
	#on quadcore the cpus are 0,1,2,3
	RES_CPUSET=1,3

	# logging settings
	LOG_DRIVER=json-file
	LOG_MAXSIZE=1k
	LOG_MAXFILE=3

	# volumes
	# using workdir as a base. sadly these need,  for now,  to get directly
	# specified in the compose file, variables didnt work #headscratch
	#FOLDER_DATA=${PWD}/persist/data
	#FOLDER_CONFIG=${PWD}/persist/config
	```
	
=== Applications

==== Operator

The operator is a tool to handle incoming data .  

[.float-group]
--    
[.left]
.overview operator 
[plantuml, operator_overview, png]     
....
title operator
node device_send_metric
node udp_proxy
node decide_known_unknown
node drop_message
node forward_timeseries_db
node timeseries_db

device_send_metric --> udp_proxy
udp_proxy --> decide_known_unknown 
decide_known_unknown --> drop_message : unknown
decide_known_unknown --> forward_timeseries_db : known
forward_timeseries_db --> timeseries_db

....
    
    Zunächst muss ein Gerät registiert werden
    Hier kann auch eine ID mitgegeben werden, oder geprüft werden, ob eine Id angelegt ist  
    # register device 
        http://{{ ... }}:8080 
    # check for a device
        http://{{ ... }}:8080/register/device?deviceid=785127d89d07441c8a95595f201ee43c
    # how many devices are registered 
        http://{{ ... }}:8080/register/device/count
        
    Dann der Whitelist für den UDP Eingang hinzugefügt werden
    Im mit dem Link aufgerufenen Fenster die vorher generierte Geräte - Id Eintragen    
        
    # append an entry to whitelist
        http://{{ ... }}:8080/register/whitelist?deviceid=5dd216fe1cbf4e0f8a13ee896774e6c8
    
    Anschließend eine Testnachricht schicken 
    Im mit dem Link aufgerufenen Fenster die vorher generierte Geräte - Id Eintragen und die Parameter nach Wunsch anpassen
    
    # send test message
        http://{{ ... }}:8080/send/timeseries?deviceid=86dbfa0cbe38403fa288647c86eae042&sensor=lagesensorlinks&value=123
--