Workspaces golden master test to check desired_config

Issue: Workspaces golden master test to check desired_... (#535486 - closed) • Unassigned • 17.11

What does this MR do and why?

Creates a Golden Master or Characterization test which allows validation of exact structure and content of Workspaces reconciliation response desired_config as generated by DesiredConfigGenerator logic, given an input workspace with an associated processed devfile.

Changes in this can cause regressions where workspaces will incorrectly restart due to pod resources changing after a new release.

This test also acts as a matrix-test approach, testing the combinations of different logic paths of DesiredConfigGenerator:

1. desired_state Terminated
2. include_all_resources true
3. include_all_resources false

See Decide how we will manage workspace restarts du... (#534808 - closed) • Unassigned for related discussions.

Explanation

The idea of a Golden Master or Characterization test is that it ensures that some output of your system should never change unless you expect it to, and it fails if the output does change unexpectedly.

So for our purposes, we would use it to ensure that the desired_config generated by DesiredConfigGenerator never changes unexpectedly.

But, we know that it has changed several times over the last release or so, due to recent changes. So, we were able to leverage this Golden Master testing approach to know exactly what changed and how, by running the test against different commits and points in time where the DesiredConfigGenerator and related files have changed, and then comparing how they differ.

References

• See Decide how we will manage workspace restarts du... (#534808 - closed) • Unassigned
• See this comment and the associated comment thread for more context (on Fix workspace restart blocked due to missing se... (!186697 - closed) • Safwan Ahmed • 17.11).

Screenshots or screen recordings

Historical SHAs checked

golden_master_desired_config_as_of_2025_02_22_809c92c3726f

• This is a golden master desired_config which passes against the master branch as of 2025-02-22, SHA 809c92c3726f
• This is before the recent changes to the DesiredConfigGenerator logic.

golden_master_desired_config_as_of_2025_04_09_d589bd91fbf4

• This is a golden master desired_config which passes against the master branch as of 2025-04-09, SHA d589bd91fbf4
• Compared to the master branch as of 2025-02-22, SHA 809c92c3726f, it has the following additions - annotations added to the secrets:

       Differences found in resource at index=0, kind='ConfigMap', name='workspace-991-990-fedcba-secrets-inventory':
       ["+", "metadata.annotations.workspaces.gitlab.com/host-template", "{{.port}}-workspace-991-990-fedcba.workspaces.localdev.me"]
       ["+", "metadata.annotations.workspaces.gitlab.com/id", "993"]
       ["+", "metadata.annotations.workspaces.gitlab.com/max-resources-per-workspace-sha256", "06879e20c353a4d871fb360635f6a87483987d44953ac6384af0451e8faa47ca"]

       Differences found in resource at index=1, kind='ConfigMap', name='workspace-991-990-fedcba-workspace-inventory':
       ["+", "metadata.annotations.workspaces.gitlab.com/host-template", "{{.port}}-workspace-991-990-fedcba.workspaces.localdev.me"]
       ["+", "metadata.annotations.workspaces.gitlab.com/id", "993"]
       ["+", "metadata.annotations.workspaces.gitlab.com/max-resources-per-workspace-sha256", "06879e20c353a4d871fb360635f6a87483987d44953ac6384af0451e8faa47ca"]

How to set up and validate locally

• Check against old and new SHAs.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.