The templating of values used in sylva units is very powerfull and flexible, but it suffers from several limitations that can be misleading for sylva users, who have to care about [nested templating and type interpretation](https://gitlab.com/sylva-projects/sylva-core/-/blob/d077499acc49234d2c16eea8615561b946c4ba56/charts/sylva-units/templates/_interpret-values.tpl#L231) Here is a sample value template as it has to be written today: ``` a: foo: bar b: '{{ .Values.a | include "preserve-type" }}' # users have to use a special template for complex types c: '{{ tuple . "b" | include "interpret" }}{{ .Values.b | get "foo" }}' # and ensure that nested templates are evaluated prior to use their values ``` While implementing this internal templating in gotpl, we are fairly limited as we can only use the `tpl` function that has following signature (pseudocode): ``` func tpl(template string, values interface{}) (string, error) { tpl.Execute(values) } ``` This nested templating could be largely improved by introducing a custom function that would return typed values instead of text: ``` func interpret(value interface{}) (interface{}, error) { // recurse on dict and lists as we do in interpret-values-gotpl if reflect.TypeOf(value) == "string" { // interpret and set it, return the result as is, preserving its type value := template(value).Execute(allValues) // All values (ie ".Values") would be provided to the function) return value } } ``` This template could be named "interpret", "getValue" or "eval" and used as follow to define the same values as in first example: ``` a: foo: bar b: "{{ eval .Values.a }}" # this would assign the dict {"foo": "bar"} to b, without having to care about its type c: "{{ eval .Values.b | get "foo" }}" # this would perform the interpretation of nested template ``` Note: As we would call interpret on all values, we will at some point call eval function to evaluate 'b' value. As this function will return a typed object, we won't have to deal with type preservation any more. Unfortunately, we don't have any solution to add such function to Helm, we could attempt to propose it upstream, but it would probably hard to introduce (and it would take a long time) But since we'll to manage the values of sylva-units in sylva-units-operator, we could perform the merge of values and interpret them in the operator using an exteended templating engine that would embeed the function described above. Sylva-units-operator would put the interpreted values in a secret, and pass it to sylva-units HelmRelease. This approach could have several benefits: - People won't have to deal with nested templating and types any more while templating values - We can expect a significant performance gain since there won't be any recursive serialisation/deserialisation of values. - The developement, testing and maintenance of values templating will be much easier (written in golang instead of goptl), and potentially easier to debug. - Templating the values prior to pass it to the HelmRelease could help operations, as we'll be able to preview values changes, or troubleshoot them even if the HelmRelease does not install - The schema will be applied to templated values - We'll probably be able to introduce that smoothly, in a backward compatible way. - As we'll be controlling how the layers of values are merged, we could introduce some custom merging features (like [merge-append](https://gitlab.com/sylva-projects/sylva-elements/helm-charts/sylva-capi-cluster/-/issues/64)) - It will also open the door for other solutions/languages to interpret values (if needed) - We'll also be able to add other templating functions if needed From an implemetation perspective, this should be fairly easy since helm engine is available as a package, with the capability to add [custom functions](https://github.com/helm/helm/blob/eb6e240cfedf688dfd407c78712a9772b72b4023/pkg/engine/engine.go#L48) The only forseenable limitation so far is that we won't have access to .Release and .Chart values any more, but we could probably overcome that: - We can either retrieve and inject the most usefull fields (Name, Namespace, Revision) - Or try to reuse helm toolkit to build the .Release/.Chart context of the sylva-units HelmRelease We'll also have to significantly increase the permission of the sylva-unit-operator controller to enable the lookups, but there shouldn't be any tradeoff compared to what we have today with Helm controller. This could even be an advantage for workload clusters, as we'll be able to restrict service accounts used for lookups.