Newer
Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
= README
:imagesdir: images/
:toc: left
== qnap-docker-compose
Creates the listed services with docker-compose on a QNAP NAS. Includes configuration examples to expose those services, attach your own domain and include ssl certs.
- Proxy:
* traefik incl. lets encrypt {{...}}:80
- Monitoring, logging:
* grafana {{...}}:3000
* graphite {{...}}:8080
* docker-fritzboxcollectd {{...}}
* collectd {{...}}
* statsd {{...}}
* harvester1 {{...}}
* log.io {{...}}:28778
- Collaboration:
* nextcloud {{...}}:80
* gogs {{...}}:3000
- Logging:
* log-file (docker-json)
docker logs xxx
* log.io
- Storage:
* minio {{ ... }}:9000
* etcd {{ ... }}
* mariadb {{ ... }}
- Tools:
* adminer {{ ... }}:8080
* etcd-viewer: {{ ... }}:8080
- Applications
* operator
== QNAP NAS configurations
- http port 8079 (instead of 8080)
- https port 442 (instead of 443)
- activate webserver to deactivate port 80 forwarding: port 78, no https
- container station app has to be installed
- a new user (NOT admin) needs to be conigured for ssh access
- container will use a seperate physical network adapter (eth1)
== Install instructions
NOTE: "{{ ... }}" are variables to be changed by the user
- dependencies:
* fritz box
* QNAP NAS with Container Station (ssh access)
* optionally some webcams
* your domains CNAME needs to show to the static ip (or dyndns) of your trafik container
* your fritzbox ip is 192.168.178.1 and connected per first lan port
* your qnap has container station installed
* "Network and Virtual Switch" application can be used
* all your domains are free for dns rebind (fritzbox setting "dns-rebind-schutz")
* qnap is connected per its second ethernet port to the fritzbox
- ssh to nas
- copy the complete repository to a folder on your qnap nas
cd /share/Services/{{ ... }}/
cp ........
- create a network for our services
docker network create -d qnet --ipam-driver=qnet \
--ipam-opt=iface=eth1 --subnet=192.168.178.0/23 \
--gateway=192.168.178.1 qnet-static-eth1
- create directories for data persistency
#export SERVICES_HOME=share/Services/{{ ... }}/persist
mkdir -p ./persist/data/nextcloud
mkdir -p ./persist/config/nextcloud
mkdir -p ./persist/data/mysql
mkdir -p ./persist/data/graphite
mkdir -p ./persist/config/graphite
mkdir -p ./persist/data/grafana
mkdir -p ./persist/config/grafana
mkdir -p ./persist/data/minio
mkdir -p ./persist/config/minio
mkdir -p ./persist/data/etcd
mkdir -p ./persist/data/gogs
#stinky change of data folder permissions
chmod 777 ./persist -R
- configure
cp acme.json.default acme.json
chmod 600 acme.json
cp traefik.toml.default traefik.toml
cp .env.default .env
- start the services
docker-compose up -d
== Configuration examples
=== traefik.toml:
# to switch on the api and the dashboard. only for testing. ...
#[api]
#entryPoint = "traefik"
#dashboard = true
# comment the caServer - config line, if done with testing and switching to production
# now, this is using the test stage ...
[acme]
caServer = "https://acme-staging-v02.api.letsencrypt.org/directory" # comment the caServer - config line, if done with testing and switching to production
# now, this is using the test stage ...
email = "a@b.net" # change
# domains are registered with traefik at lets encrypt to gather ssl certificates.
[[acme.domains]]
main = "collab.{{replace}}.net"
main = "service.{{replace}}.net"
main = "nextcloud.{{replace}}.net"
main = "proxytraefik.{{replace}}.myfritz.net"
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
[docker]
endpoint = "unix:///var/run/docker.sock"
watch = true
=== docker-compose.yaml:
# see the grafana container as example
grafana:
image: grafana/grafana:latest # maybe want to set to a specific version?
container_name: grafana
hostname: grafana
#user: ${ID} # how to define which user it should use?
restart: always
mem_limit: ${RES_MEMLIMIT}
mem_reservation: ${RES_MEMSOFTLIMIT}
cpuset: ${RES_CPUSET} # qnap nas has 4 cpus. we dont want the containers block the nas
ports:
- "3000:3000"
volumes:
- data_grafana:/var/lib/grafana
networks:
proxy:
ipv4_address: ${IP_GRAFANA} # needed for now for simple usage within qnaps networking
labels:
- "traefik.backend=grafana" # add, if service should be accessed by a domain and ssl-proxied by traefik
- "traefik.frontend.rule=Host:${DOMAIN_SERVICE}"
- "traefik.docker.network=proxy"
- "traefik.port=3000"
- "traefik.enabled=true"
- "collectd_docker_app=grafana" # add, to get this containers stats into monitoring
- "collectd_docker_task=grafana"
#we have a qnap specific network (user defined network) defined manually
networks:
proxy:
external:
name: qnet-static-eth1
#and to store the data permanently, we use volumes
volumes:
install_nextcloud: # data will be installed into volume in /docker/bin ..or such..
data_grafana: # we bind it to a folder, so we know where the data is kept
driver: local
driver_opts:
type: 'none'
o: 'bind'
device: '$PWD/persist/data/grafana'
=== .env:
ID=$(id -u)
# domains that have a CNAME record linking to our proxy servers ip (or dyndns entry)
DOMAIN_COLLAB=collab.tigabeatz.net
# internally used ips
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
# cheap passwords
MYSQL_ROOT_PASSWORD=pa$$worD
# get md5 from htpasswd or http://www.htaccesstools.com/htpasswd-generator/
TRAEFIK_BASIC_PASS=$apr1$0As0jOd2$uRagOVpMPNzdjTLq0fGGO.
# resource settings
# after inspection, adapt each container to individual values
RES_MEMLIMIT=1024MB
RES_MEMSOFTLIMIT=256MB
#on quadcore the cpus are 0,1,2,3
RES_CPUSET=1,3
# logging settings
LOG_DRIVER=json-file
LOG_MAXSIZE=1k
LOG_MAXFILE=3
# volumes
# using workdir as a base. sadly these need, for now, to get directly
# specified in the compose file, variables didnt work #headscratch
#FOLDER_DATA=${PWD}/persist/data
#FOLDER_CONFIG=${PWD}/persist/config
```
=== Applications
==== Operator
The operator is a tool to handle incoming data .
[.float-group]
--
[.left]
.overview operator
[plantuml, operator_overview, png]
....
title operator
node device_send_metric
node udp_proxy
node decide_known_unknown
node drop_message
node forward_timeseries_db
node timeseries_db
device_send_metric --> udp_proxy
udp_proxy --> decide_known_unknown
decide_known_unknown --> drop_message : unknown
decide_known_unknown --> forward_timeseries_db : known
forward_timeseries_db --> timeseries_db
....
Zunächst muss ein Gerät registiert werden
Hier kann auch eine ID mitgegeben werden, oder geprüft werden, ob eine Id angelegt ist
# register device
http://{{ ... }}:8080
# check for a device
http://{{ ... }}:8080/register/device?deviceid=785127d89d07441c8a95595f201ee43c
# how many devices are registered
http://{{ ... }}:8080/register/device/count
Dann der Whitelist für den UDP Eingang hinzugefügt werden
Im mit dem Link aufgerufenen Fenster die vorher generierte Geräte - Id Eintragen
# append an entry to whitelist
http://{{ ... }}:8080/register/whitelist?deviceid=5dd216fe1cbf4e0f8a13ee896774e6c8
Anschließend eine Testnachricht schicken
Im mit dem Link aufgerufenen Fenster die vorher generierte Geräte - Id Eintragen und die Parameter nach Wunsch anpassen
# send test message
http://{{ ... }}:8080/send/timeseries?deviceid=86dbfa0cbe38403fa288647c86eae042&sensor=lagesensorlinks&value=123
--