In !144663 (comment 1773372289) it was highlighted that currently we store inputs specifically for catalog_resource_components table. This will not be sufficient to support CI steps as it has more details under spec: (e.g. outputs).
Proposal
The suggestion was to change the inputs jsonb column to spec jsonb column so that we can store the full spec: object.
Then based on the resource_type we can interpret the spec object accordingly. For example, for template type we only expect inputs but for step type we expect inputs, outputs etc.
This design will also avoid that we add more columns in the future for such metadata.
Should we introduce the new column side-by-side initially and use spec || inputs until fully migrated?
Can we migrate the existing metadata? For templates it would be that we wrap the inputs value around { inputs: "..." } to make it compatible with the full spec: format.
@fabiopitino - sure! Does this have any breaking changes for the user? just wondering why it's a blocker for GA. Either way here are my responses:
Should we introduce the new column side-by-side initially and use spec || inputs until fully migrated?
Yes. I think we can do this in one MR and then do the migration right after.
Can we migrate the existing metadata? For templates it would be that we wrap the inputs value around { inputs: "..." } to make it compatible with the full spec: format.
@marknuzzo@dhershkovitch I'm going to take on this issue since it's a blocker for GA and it'll take several steps to complete. I'll be adding tasks for those steps soon
@fabiopitino in the description of this issue it says that templates and steps are both components. Is that true? I thought that components and steps were separate and that we'd stop using the term template (except for the old CI templates)
It sounds like the plan is to store steps in the same table as components. Is that right? Do we plan to store other future resource types in the same table?
@dhershkovitch - with the first 3 tasks complete now, @avielle and I agreed that since the remaining 2 open tasks are not related to GA, I'm going to close this issue, move those other 2 tasks into a non-GA related epic and then update the workflow status accordingly so that it's clean here. I just wanted to make you aware of these details.