Backend Design Discussion: versioning Agent Config Table

MR: Pending

Description

Following the discussion here, we are creating this issue for further discussion.

As we add new fields to the agent configuration, there is a usually a need to de-normalize the field to the workspaces table. This is to ensure that as we update the field in the agent config, it does not affect the existing workspaces. However, there are also scenarios where for existing fields of the agent config, where updating that fields affects all associated workspaces(new+existing). The issue aims to achieve 2 things -

• Make the update experience the same for all fields of the agent configuration.
• Version the agent configuration so that there is no need to de-normalize the fields to the workspaces table.

Solution

Multiple solutions have been proposed below (1, 2, 3, 4, 5). The below description is based on Solution 5.

• Make update behaviour of all the fields of the agent config consistent - updating the field would only affect the new workspaces and not the old ones.
  • TODO: Figure out how do we need to announce this?
• Add versioning to the agent config table.
  • Add a new field to the agent config table - version - of type integer.
  • For each new cluster_agent_id, this version will start from 1.
  • Every update to the agent will bump the corresponding version of the cluster_agent_id in the agent config table.
• Add reference to agent config version in the workspaces table.
  • Add a reference to the agent config version table to get the associated version of the workspace which was set during workspace creation.
  • Drop all the fields in the workspaces table which were de-normalized from the agent earlier.
• Use the latest version of the agent config while creating a new workspace.

The result would be that any updates to the agent config table would only apply to the latest agent configuration. There will no longer be a need to denormalize fields of the agent config table to the workspaces table.

We also plan to introduce paper_trail gem based on the conversation here for versioning the agent config table.

Follow ups

• Move agent config to the UI.
• Provide an agent admin option to migrate all workspaces from older agent versions to the newer ones. As it stands currently, it would result in immediate restart of the workspace. We can work on making it more delayed in nature.

Acceptance Criteria

TODO: Fill out (required)

• [Describe what must be achieved to complete this issue.]
• [Describe another requirement needed to complete this issue.]
• [Add additional acceptance criteria as needed.]