Integrate Opni as an optional observability unit in Sylva
What does this MR do and why?
This MR aims at integrating the Opni application into the Sylva project. Opni is an open-source software designed for multi-cluster and multi-tenant observability.
Description of what this MR covers and why has it been implemented in such a way can be found in the TL;DR section below.
Related reference(s)
Information on Opni and how to use it in the Sylva project can be found in the documentation that comes with this MR.
Test coverage
The suggested changes have been tested using Sylva's kubeadm-capd environment.
There the following manual tests were performed:
- Opni Monitoring backend functionality validation
- Opni Logging backend functionality validation
- Capability to add additional opni-agents
- Alerting tests
- AIOps Log anomaly detection tests using Opni's pretrained models
TL;DR
This MR introduces the following Sylva units:
-
opni-ns
- sets up the namespace where Opni will be deployed -
opni-crd
- deploys CRDs that are a prerequisite for the Opni application -
opni
- the Opni application itself -
opni-keycloak-eso
- external secrets that are used for Opni's Keycloak authentication automation -
opni-keycloak-resources
- automates Keycloak client and user creation for Opni's Keycloak authentication setup -
opni-ingress
- exposes relevant Opni urls, such as for: monitoring, logging, dashboard
Once the opni
unit is enabled all relevant units to the specific Opni configuration will also be enabled. Once it is disabled, all resources related to Opni will be removed, leaving the cluster clean of any leftover Opni resources.
Changes that are out of the ordinary Sylva setup
opni-ns
has not been added to the namespace-defs unit, due to:
- Opni creates additional resources while it works. If the namespace is not deleted once the
opni
unit is deleted any further enablement of theopni
unit will result in a failure, due to resources from previous setups being present. Having this as a unit bound to the "opni unit chain" ensures that the namespace will always be fresh on everyopni
unit enable/disable. - Did not want for the
opni
unit to have leftover resources if disabled.
opni-keycloak-eso
and opni-keycloak-resources
resources were not added to the keycloak-oidc-external-secrets
and keycloak-resources
respectively, because:
- Did not want for the
opni
unit to have leftover resources if disabled, or to have resources that will be deployed, but not be used in certain Opni deployment configurations. -
opni
being marked as an optional component made little sense for me to put them there.
Resources related to Opni -> Keycloak authentication automation:
Opni uses an OpenID provider in order to authenticate Grafana users, this MR comes with an automation (triggered by a configuration flag) that prepares Keycloak resources, so that Opni can use Sylva's Keycloak as this authentication mechanism.
- keycloak-resources - covered by explanation above
- eso - covered by explanation above
-
configtemplate-opni-keycloak.tpl - configurations that are specific only to the Opni -> Keycloak automation and are added to the
opni
unit only if the.Values.cluster.opni.gateway.auth.useInternalKeycloak
flag has been set totrue
. This is done in the following way due to the limitation of not being able to writeif
clauses in the rootvalues.yaml
file of the Sylva chart. -
kubernetes-keycloak-ns-secretstore.yaml - added the default namespace to the secret store conditions in order to ensure that the
opni-oidc-auth
secret could be created in the default namespace and picked up by theopni
unit's HelmRelease resource. Since doing a lookup from one unit to another (while both units are being deployed) is not possible, this was the only other way that I could get values from a secret and use it in my unit. -
kubernetes-cert-manager-ns-secretstore.yaml - added the
opni
namespace here in order to ensure that theeso-sylva-ca
resource could deploy thesylva-ca.crt
secret in theopni
namespace.
Other changes that require an explanation:
-
workload-cluster.values.yaml
- added thecert-manager
unit as an optional component in theworkload-cluster
deployment.cert-manager
is a prerequisite for a successful deployment of anopni-agent
.