Migrate compliance frameworks to the new table

What does this MR do?

Related issue: #251113 (closed)

This MR connects the newly introduced ComplianceManagement::Framework model with the ComplianceManagement::ComplianceFramework::ProjectSettings model in a backward compatible manner. It also fixes a mistake with the original schema: we should associate the ComplianceManagement::Framework model with Namespace and not with the Group.

Goal (not part of this MR): allow groups to customize their compliance frameworks and migrate the existing data (enum) to the newly created table.

How to test it

There are no user-facing changes.

1. Log in to a local gitlab instance with ultimate license.
2. Go to a project / Settings / General
3. Pick a Compliance framework
4. Check the logs and verify that the framework and framework_id columns are being set.
5. Check if an INSERT happens to compliance_management_frameworks table.

Development

The current status (no connection):

This MR:

End goal (next MR/milestone):

Database migration

1. Add namespace_id column.
2. Fix the mistake where we used group_id column instead of namespace_id.
   • Remove the FK, remove the NOT NULL constraint and ignore the column (group_id)
3. Add framework_id column.
4. Migrate the data from the framework enum.
   • About 1000 records to be created.
5. Add not null constraint on the framework_id column

Note: I tested it on postgres.ai, the migrations ran without errors.

Up:

== 201005092703 AddNamespaceColumnToFrameworks: migrating =====================
-- column_exists?(:compliance_management_frameworks, :namespace_id)
   -> 0.0011s
-- add_column(:compliance_management_frameworks, :namespace_id, :integer)
   -> 0.0024s
-- change_column_null(:compliance_management_frameworks, :namespace_id, false)
   -> 0.0011s
-- transaction_open?()
   -> 0.0000s
-- foreign_keys(:compliance_management_frameworks)
   -> 0.0023s
-- execute("ALTER TABLE compliance_management_frameworks\nADD CONSTRAINT fk_b74c45b71f\nFOREIGN KEY (namespace_id)\nREFERENCES namespaces (id)\nON DELETE CASCADE\nNOT VALID;\n")
   -> 0.0010s
-- execute("SET statement_timeout TO 0")
   -> 0.0001s
-- execute("ALTER TABLE compliance_management_frameworks VALIDATE CONSTRAINT fk_b74c45b71f;")
   -> 0.0017s
-- execute("RESET ALL")
   -> 0.0002s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:compliance_management_frameworks, [:namespace_id, :name], {:unique=>true, :name=>"idx_on_compliance_management_frameworks_namespace_id_name", :algorithm=>:concurrently})
   -> 0.0017s
-- add_index(:compliance_management_frameworks, [:namespace_id, :name], {:unique=>true, :name=>"idx_on_compliance_management_frameworks_namespace_id_name", :algorithm=>:concurrently})
   -> 0.0136s
== 201005092703 AddNamespaceColumnToFrameworks: migrated (0.0321s) ============

== 201005092709 RemoveComplianceFrameworksGroupIdFk: migrating ================
-- change_column_null(:compliance_management_frameworks, :group_id, true)
   -> 0.0024s
-- foreign_keys(:compliance_management_frameworks)
   -> 0.0083s
-- remove_foreign_key(:compliance_management_frameworks, :namespaces, {:column=>:group_id})
   -> 0.0115s
-- transaction_open?()
   -> 0.0000s
-- indexes(:compliance_management_frameworks)
   -> 0.0053s
-- remove_index(:compliance_management_frameworks, {:algorithm=>:concurrently, :name=>"index_compliance_management_frameworks_on_group_id_and_name"})
   -> 0.0039s
== 201005092709 RemoveComplianceFrameworksGroupIdFk: migrated (0.0332s) =======

== 20201005092753 AddFrameworkIdToProjectFrameworkSettings: migrating =========
-- column_exists?(:project_compliance_framework_settings, :framework_id)
   -> 0.0031s
-- add_column(:project_compliance_framework_settings, :framework_id, :bigint)
   -> 0.0012s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:project_compliance_framework_settings, :framework_id, {:algorithm=>:concurrently})
   -> 0.0046s
-- add_index(:project_compliance_framework_settings, :framework_id, {:algorithm=>:concurrently})
   -> 0.0125s
-- transaction_open?()
   -> 0.0000s
-- foreign_keys(:project_compliance_framework_settings)
   -> 0.0073s
-- execute("ALTER TABLE project_compliance_framework_settings\nADD CONSTRAINT fk_be413374a9\nFOREIGN KEY (framework_id)\nREFERENCES compliance_management_frameworks (id)\nON DELETE CASCADE\nNOT VALID;\n")
   -> 0.0018s
-- execute("ALTER TABLE project_compliance_framework_settings VALIDATE CONSTRAINT fk_be413374a9;")
   -> 0.0033s
== 20201005092753 AddFrameworkIdToProjectFrameworkSettings: migrated (0.0479s)

== 20201005094331 MigrateComplianceFrameworkEnumToDatabaseFrameworkRecord: migrating
== 20201005094331 MigrateComplianceFrameworkEnumToDatabaseFrameworkRecord: migrated (0.0124s)

== 20201005153955 AddNotNullConstraintToComplianceProjectSettings: migrating ==
-- current_schema()
   -> 0.0008s
-- transaction_open?()
   -> 0.0000s
-- current_schema()
   -> 0.0007s
-- execute("ALTER TABLE project_compliance_framework_settings\nADD CONSTRAINT check_d348de9e2d\nCHECK ( framework_id IS NOT NULL )\nNOT VALID;\n")
   -> 0.0012s
-- current_schema()
   -> 0.0007s
-- execute("ALTER TABLE project_compliance_framework_settings VALIDATE CONSTRAINT check_d348de9e2d;")
   -> 0.0023s
== 20201005153955 AddNotNullConstraintToComplianceProjectSettings: migrated (0.0314s)

Down:

== 20201005153955 AddNotNullConstraintToComplianceProjectSettings: reverting ==
-- execute("ALTER TABLE project_compliance_framework_settings\nDROP CONSTRAINT IF EXISTS check_d348de9e2d\n")
   -> 0.0005s
== 20201005153955 AddNotNullConstraintToComplianceProjectSettings: reverted (0.0037s)


== 20201005094331 MigrateComplianceFrameworkEnumToDatabaseFrameworkRecord: reverting
== 20201005094331 MigrateComplianceFrameworkEnumToDatabaseFrameworkRecord: reverted (0.0000s)


== 20201005092753 AddFrameworkIdToProjectFrameworkSettings: reverting =========
-- foreign_keys(:project_compliance_framework_settings)
   -> 0.0025s
-- remove_foreign_key(:project_compliance_framework_settings, :compliance_management_frameworks, {:column=>:framework_id})
   -> 0.0037s
-- remove_column(:project_compliance_framework_settings, :framework_id)
   -> 0.0004s
== 20201005092753 AddFrameworkIdToProjectFrameworkSettings: reverted (0.0110s)

== 201005092709 RemoveComplianceFrameworksGroupIdFk: reverting ================
-- change_column_null(:compliance_management_frameworks, :group_id, false)
   -> 0.0016s
-- transaction_open?()
   -> 0.0000s
-- foreign_keys(:compliance_management_frameworks)
   -> 0.0027s
-- execute("ALTER TABLE compliance_management_frameworks\nADD CONSTRAINT fk_d3240d6339\nFOREIGN KEY (group_id)\nREFERENCES namespaces (id)\nON DELETE CASCADE\nNOT VALID;\n")
   -> 0.0011s
-- execute("SET statement_timeout TO 0")
   -> 0.0002s
-- execute("ALTER TABLE compliance_management_frameworks VALIDATE CONSTRAINT fk_d3240d6339;")
   -> 0.0021s
-- execute("RESET ALL")
   -> 0.0002s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:compliance_management_frameworks, [:group_id, :name], {:unique=>true, :name=>"index_compliance_management_frameworks_on_group_id_and_name", :algorithm=>:concurrently})
   -> 0.0018s
-- add_index(:compliance_management_frameworks, [:group_id, :name], {:unique=>true, :name=>"index_compliance_management_frameworks_on_group_id_and_name", :algorithm=>:concurrently})
   -> 0.0169s
== 201005092709 RemoveComplianceFrameworksGroupIdFk: reverted (0.0316s) =======

== 201005092703 AddNamespaceColumnToFrameworks: reverting =====================
-- transaction_open?()
   -> 0.0000s
-- indexes(:compliance_management_frameworks)
   -> 0.0021s
-- execute("SET statement_timeout TO 0")
   -> 0.0003s
-- remove_index(:compliance_management_frameworks, {:algorithm=>:concurrently, :name=>"idx_on_compliance_management_frameworks_namespace_id_name"})
   -> 0.0022s
-- execute("RESET ALL")
   -> 0.0002s
-- foreign_keys(:compliance_management_frameworks)
   -> 0.0022s
-- remove_foreign_key(:compliance_management_frameworks, :namespaces, {:column=>:namespace_id})
   -> 0.0033s
-- remove_column(:compliance_management_frameworks, :namespace_id)
   -> 0.0009s
== 201005092703 AddNamespaceColumnToFrameworks: reverted (0.0116s) ============

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team