Backend: Change catalog_resource_components.inputs to spec

Problem

In !144663 (comment 1773372289) it was highlighted that currently we store inputs specifically for catalog_resource_components table. This will not be sufficient to support CI steps as it has more details under spec: (e.g. outputs).

Proposal

The suggestion was to change the inputs jsonb column to spec jsonb column so that we can store the full spec: object.

Then based on the resource_type we can interpret the spec object accordingly. For example, for template type we only expect inputs but for step type we expect inputs, outputs etc.

This design will also avoid that we add more columns in the future for such metadata.

Implementation Table

Group Issue Link
backend 👈 You are here
backend #455431 (closed)

Challenges

  • Should we introduce the new column side-by-side initially and use spec || inputs until fully migrated?
  • Can we migrate the existing metadata? For templates it would be that we wrap the inputs value around { inputs: "..." } to make it compatible with the full spec: format.
  • At what point can we remove the inputs column?
Edited by Mark Nuzzo