Skip to content

Migrate inputs to spec on CI components table

Avielle Wolfe requested to merge 443662-migrate-inputs-to-spec into master

What does this MR do and why?

catalog_resource_components was created with an inputs column that stores the inputs value from a CI template's header. However, the components table now needs to store CI steps as well as templates, and steps have outputs as well as inputs. To handle this, a new spec column was introduced that stores the entire header.

This commit migrates the values from the inputs column of existing CI component records to the spec column so that we can remove the inputs column in the next milestone. It appends an inputs: key to the existing inputs hash so it has the correct form for the spec column.

Changelog: other

Issue: #443662 (closed)

Terminology

  1. There are 2 types of CI component: templates and steps
  2. A CI header contains a spec that has inputs for CI templates and both inputs and outputs for CI steps

Migration Output

Up
main: == [advisory_lock_connection] object_id: 122300, pg_backend_pid: 74394
main: == 20240403121221 MigrateInputsToSpecOnCatalogResourceComponents: migrating ===
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- execute("UPDATE catalog_resource_components\nSET spec = jsonb_set('{}'::jsonb, '{inputs}', inputs::jsonb)\nWHERE id BETWEEN 9 AND 18\nAND spec = '{}' AND inputs <> '{}'\n")
main:    -> 0.0022s
main: == 20240403121221 MigrateInputsToSpecOnCatalogResourceComponents: migrated (0.0207s)

main: == [advisory_lock_connection] object_id: 122300, pg_backend_pid: 74394
ci: == [advisory_lock_connection] object_id: 122600, pg_backend_pid: 74396
ci: == 20240403121221 MigrateInputsToSpecOnCatalogResourceComponents: migrating ===
ci: -- The migration is skipped since it modifies the schemas: [:gitlab_main].
ci: -- This database can only apply migrations in one of the following schemas: [:gitlab_ci, :gitlab_internal, :gitlab_shared].
ci: == 20240403121221 MigrateInputsToSpecOnCatalogResourceComponents: migrated (0.0067s)
Down

No-op

How to set up and validate locally

  1. Create a CI component that has an inputs value and no spec value. If you have no CI catalog resources in your GDK, use all of these commands:
    project = Project.first # or any project
    user = User.first
    resource = Ci::Catalog::Resource.create!(project: project)
    release = Release.create!(project: project, tag: 'test', author: user)
    version = Ci::Catalog::Resources::Version.create!(project: project, release: release, catalog_resource: resource, version: '1.0.0')
    
    component = Ci::Catalog::Resources::Component.create!(
      project: project, catalog_resource: resource, version: version, name: 'test',
      spec: {}, inputs: { test_input: nil }
    )
  2. Run the migration
  3. Check the component spec value:
    component.reload.spec
    => { 'inputs' => { 'test_input' => nil } }

Related to #443662 (closed)

Merge request reports