Create consul server for gitlab runners
We need to do service discovery for all runner machines in order to have machines monitoring. We would add consul agent to this base image: https://dev.gitlab.org/cookbooks/packer-runner-machines.
But we need separate consul server that could be used exclusively by runners, as we consider them insecure.
Checklist:
-
prepare certificate/key pairs for consul: -
for cluster (will be used in cluster roles definitions) -
for client (will be added to base images and will be used by prometheus nodes) -
prepare gitlab-runners-consul/cluster
andgitlab-runners-consul/client
vaults with TLS secrets
-
-
prepare base gitlab-runners-consul
role usinggitlab_consul::cluster
cookbook-
configure user access -
configure base Consul cluster setup
-
-
create Consul cluster in DO NYC1 -
create machines (use 2gb
- 2 GB RAM, 2 vCPU)-
consul-01.nyc1.do.gitlab-runners.gitlab.net -
consul-02.nyc1.do.gitlab-runners.gitlab.net -
consul-03.nyc1.do.gitlab-runners.gitlab.net
-
-
add DNS entries: -
consul-01.nyc1.do.gitlab-runners.gitlab.net (IP: 165.227.93.161) -
consul-02.nyc1.do.gitlab-runners.gitlab.net (IP: 165.227.93.169) -
consul-03.nyc1.do.gitlab-runners.gitlab.net (IP: 165.227.93.170)
-
-
prepare gitlab-runners-consul-do-nyc1
role-
extend gitlab-runners-consul
role -
configure cluster nodes ( ["10.136.75.77", "10.136.75.80", "10.136.75.81"]
)
-
-
bootstrap chef on machines
-
-
create Consul cluster in GCE us-east1-c -
create machines (use n1-standard-2
- 7.5 GB RAM, 2 vCPU)-
consul-01.us-east1-c.gce.gitlab-runners.gitlab.net -
consul-02.us-east1-c.gce.gitlab-runners.gitlab.net -
consul-03.us-east1-c.gce.gitlab-runners.gitlab.net
-
-
add DNS entries: -
consul-01.us-east1-c.gce.gitlab-runners.gitlab.net (IP: 35.185.27.109) -
consul-02.us-east1-c.gce.gitlab-runners.gitlab.net (IP: 104.196.128.177) -
consul-03.us-east1-c.gce.gitlab-runners.gitlab.net (IP: 35.185.41.82)
-
-
prepare gitlab-runners-consul-gce-us-east1-c
role-
extend gitlab-runners-consul
role -
configure cluster nodes ( ["10.142.0.4", "10.142.0.6", "10.142.0.7"]
)
-
-
bootstrap chef on machines
-
-
create Consul cluster in GCE us-east1-d -
create machines (use n1-standard-2
- 7.5 GB RAM, 2 vCPU)-
consul-01.us-east1-d.gce.gitlab-runners.gitlab.net -
consul-02.us-east1-d.gce.gitlab-runners.gitlab.net -
consul-03.us-east1-d.gce.gitlab-runners.gitlab.net
-
-
add DNS entries: -
consul-01.us-east1-d.gce.gitlab-runners.gitlab.net (IP: 104.196.45.251) -
consul-02.us-east1-d.gce.gitlab-runners.gitlab.net (IP: 35.185.2.165) -
consul-03.us-east1-d.gce.gitlab-runners.gitlab.net (IP: 104.196.71.67)
-
-
prepare gitlab-runners-consul-gce-us-east1-d
role-
extend gitlab-runners-consul
role -
configure cluster nodes ( ["10.142.0.8", "10.142.0.9", "10.142.0.10"]
)
-
-
bootstrap chef on machines
-
-
prepare monitoring of consul hosts availability -
add alerting tracked by prometheus.gitlab.com (gitlab-com/runbooks!373 (merged))
-
-
configure Consul agent: -
configure node_exporter
service (waits for gitlab-cookbooks/gitlab_consul!9 (merged))
-
-
prepare ACL policy for cluster that will disable key/value store usage
Edited by Tomasz Maczukin