automatically select image_key

This issue is part of sylva-projects#43 (closed) for "Topic 1"

Recap

Today users have to set image_key in their environment values. This is done under cluster.capm3.image_key, and/or control_plane.capm3.image_key machine_deployments.capm3.image_key, etc.

Improvement proposed:

  • let users only specify the OS flavor (e.g ubuntu jammy) and k8s flavor (e.g. RKE2) in their environment values, via "OS image selectors"
  • have the code automatically find OS images matching such "OS image selectors" and use them in CAPI resources

MRs in flight to implement this improvement

In rough merge order:

📑 there is also a documentation portal MR related to this change and related ones: sylva-projects.gitlab.io!144 (merged)

Historical discussion, for reference

As discussed, I'm filing this issue to follow up on #729 (comment 1651121305), a suggestion by @cristian.manda :

If I can propose something that would be to only request user values to specify the os_distro and os_hardening and let sylva-units compose the rest of the image name using bootstrap_provider and k8s_version resulting in, for example ubuntu-plain-rke2-1.26.9. I don't think it is necessary to keep the os version in the image name considering we will not support multiple versions of the same os, otherwise we will really end up with a lot of images. We would end up with:

cluster:
  capm3:
   os_distro: ubuntu
   os_hardening: false
 control_plane: 
   capm3:
     os_hardening: true

I would recap what we more recently discussed as follows: we'll let the user specify which OS and which flavor is wanted, and based on that and on the target k8s version and the bootstrap/control plane provider, Sylva code will find an OS image among images specified in sylva-units sylva_diskimagebuilder_images and os_images values that can be used.

In detail:

  1. introduce code that can loop through the os_images dict of sylva-capi-cluster and return the image_key matching "image requirements"
  2. "image requirements" will be expressed at each level of sylva-capi-cluster:
    • default
    • control-plane (under control_plane)
    • MD defaults (machine_deployment_default)
    • per-MD (machine_deployments.xxxx)
  3. "image requirements" would be:
    • expressed as a dict of parameters
    • I propose to use the os_image_selector name for this dict
    • the keys will be the same as the annotations produced on OS images by sylva diskimage-builder
  4. the code will be put in a Helm library chart, so it can be used:
    • from s-c-c to determine the image_key
    • from sylva-core to be able to build a list of image_keys relevant for the current cluster

The s-c-c values, would look like:

## populated by the tool fetching OCI artifacts (and for CAPO the tool fetching the openstack image UUID)
os_images:
  ubuntu-jammy-plain-rke2-1-27-10:
    archive-size: "1957366162"
    commit-id: feee9c481b2dfae7421c49c752b91fff6973a68c
    commit-tag: 0.2.0
    compression: gz
    filename: ubuntu-jammy-plain-rke2-1-27-10.raw
    flavor: plain
    image-format: raw
    k8s-flavor: rke2
    k8s-version: 1.27.10-rke2r1
    md5: 693a1981408afad84638dfaee34d4fcb
    openstack_glance_uuid: 91b4027a-098a-4fe8-9f8a-ced920e991e8
    os: ubuntu
    os-release: jammy
    sha256: 934abbb43b8983161de5d214f6ce8bad1f788b652673a017d4c4c8e39388dd07
    size: "6261047296"
    uri: oci://registry.gitlab.com/sylva-projects/sylva-elements/diskimage-builder/ubuntu-jammy-plain-rke2-1-27-10:0.2.0
  ubuntu-jammy-hardened-kubeadm-1-27-10:
    archive-size: "1957366162"
    commit-id: feee9c481b2dfae7421c49c752b91fff6973a68c
    commit-tag: 0.2.0
    compression: gz
    filename: ubuntu-jammy-hardened-kubeadm-1-27-10.raw
    flavor: hardened
    image-format: raw
    k8s-flavor: kubeadm
    k8s-version: 1.27.10
    md5: 693a1981408afad84638dfaee34d4fcb
    openstack_glance_uuid: 91b4027a-098a-4fe8-9f8a-ced920e991e8
    os: ubuntu
    os-release: jammy
    sha256: 934abbb43b8983161de5d214f6ce8bad1f788b652673a017d4c4c8e39388dd07
    size: "6261047296"
    uri: oci://registry.gitlab.com/sylva-projects/sylva-elements/diskimage-builder/ubuntu-jammy-plain-rke2-1-27-10:0.2.0
 ...

# user-provided (example)

os_image_selector:
  os: ubuntu
  os-release: jammy
  flavor: hardened

control_plane:
  os_image_selector:
    flavor: plain

machine_deployments:
  foo:
    os-release: noble

With the above the filter to find an image_key in os_images will be, for the control plane, assuming that the deployment is kubeadm (cabpk) and k8s 1.27.12:

os: ubuntu
os-release: jammy
flavor: hardened
k8s-flavor: kubeadm  # determined based no bootstrap_provider: cabpk
k8s-version: 1.27.12  # determined based on k8s_version: 1.27.12
Edited by Thomas Morin