Do not cache sh.helm.release secrets

vladimir braquetrequested to merge
filtered_cached_objects into main

What it does

Helm is relying on secrets for internal purpose. These secrets are heavy (up to 1.5MB in sylva), numerous (one perverion and per Helm release) and scale with the number of workload clusters and of units. By default, the present operator is caching them since secrets are watched. It leads to huge memory consumption

Therefore, the present MR is about preventing the caching of these secrets by firing them based on the "owner": "helm" label. Since they are internal, users should not use them as valuesFrom anyway. Since these secrets are not cached, API calls may rise.

Compared to the proposed solution https://github.com/kubernetes-sigs/controller-runtime/issues/244#issuecomment-2466564541

  • client.Options.Cache is not set, because it only prevent access to the cache from the client
  • No specific client is created, the usual default client is still used, with a specific cache

For objects not listed in the byObject filed, the usual caches are used

Tests

  • When several secrets with "owner": "helm" label are created, no additional memory consumption ✅
  • When several secrets without "owner": "helm" label are created, additional memory consumption is happening ✅
  • When several config maps, external secrets or other resources are created, additional memory consumption is happening ✅
  • Usual units & API tests work ✅
  • E2E Tests work ✅
  • CAPO & deployment of a workload cluster ✅

Reference

Close #37 (closed)

Edited by vladimir braquet

Merge request reports