Implement ClickHouse operator, use in Opstrace Controller
Implement a ClickHouse operator for deploying instances that can be used both for Opstrace components and for other teams to use. Similar to the existing operator by Altinity but with tighter scope. For example the Altinity operator defers to rendering templates as-provided where for us it'd make sense to just have one or two preset configurations.
On the technical side this is pretty open-ended but one requirement is to use kubebuilder since the team already has lots of experience with that. Also just keep the code in the main opstrace
repo for now.
The main goals are:
- Design a basic/minimal CRD for defining a ClickHouse instance. Audiences are:
- Opstrace itself
- Other teams within GitLab who may not have much experience with deploying ClickHouse
- Implement the kubebuilder operator to deploy CRs
- Switch Opstrace controller from using the Altinity operator to instead using this new one.
- Ideally the new operator would also take over things like user/quota setup that are currently being done manually in the Opstrace controller.
Relates to #1642 (closed)