Incubation:APM October 11, 2021 Demo
Subscribe to the weekly demo issue
Recording
Current Update
ClickHouse replication/sharding research
- MergeTree engine replication - https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/replication/#creating-replicated-tables
- Distributed table engine in ClickHouse to abstract over cluster/shards/tables - https://clickhouse.com/docs/en/engines/table-engines/special/distributed/
- Altinity ClickHouse operator setup with replication - https://github.com/Altinity/clickhouse-operator/blob/master/docs/replication_setup.md
Sharding may only be required if the amount of data is outgrowing the storage size of the servers. I will leave this for now as we can do this later.
#16 (closed)
ClickHouse database migrations -- Using Golang migrations CLI/library for migrations - https://github.com/golang-migrate/migrate
- Migration strategy docs - https://gitlab.com/gitlab-org/incubation-engineering/apm/apm/-/blob/datadog_series/docs/migrations.md
Useful links
- DataDog Vector - think fluentd but for all observability data. Looks like a useful product for end users to get their data into datadog and/or various other sinks.
- Jaeger ClickHouse storage plugin - Interesting solution we can look at, especially for the schema.
Up Next
- Continue to iterate testing infrastructure - #18 (closed)
Edited by Joe Shaw