[Group Migration MVC] Store temporary data used for a GitLab import

What does this MR do?

Addresses: #250280 (closed), as part of the overall goal of implementing the Group Migration MVC.

This will be followed up by #249159 (closed), which will start persisting the data and using it to import from another GitLab instance in Sidekiq jobs. This is in draft here: !44836 (merged)

---

🔧 Implementation

Implements the following, to fit in this class diagram (I can't include the namespace properly in the diagram so the names are a little wrong in there):

1. BulkImport
2. BulkImport::Configuration
3. BulkImport::Entity

classDiagram
    User "1" --> "*" BulkImport
    BulkImport "1" --> "*" BulkImportConfiguration
    BulkImport "1" --> "*" BulkImportEntity
    Group "1" --> "*" ImportFailure
    Project "1" --> "*" ImportFailure
    BulkImportEntity "1" --> "0..1" Group: XOR Project
    BulkImportEntity "1" --> "0..1" Project : XOR Group
    BulkImportEntity "1" --> "0..1" BulkImportEntity : Parent
    class BulkImport{
      int id
      enum source_type
      int status
    }
    class BulkImportConfiguration{
      text url
      text access_token
    }
    class BulkImportEntity{
      enum type
      text source_full_path
      int parent_id
      int group_id
      int project_id

      text destination_name
      text destination_namespace
      int status
      int jid
    }
    class ImportFailure{
    }
    class Group{
    }
    class Project{
    }

🗄 Migrations

Due to the section under https://docs.gitlab.com/ee/development/migration_style_guide.html#examples called Creating a new table when we have two foreign keys, I ended up needing to create a lot of migrations to create the ImportEntity modelling.

CreateBulkImport

rails db:migrate:up VERSION=20200922133949

== 20200922133949 CreateBulkImport: migrating =================================
-- create_table(:bulk_imports)
   -> 0.0065s
== 20200922133949 CreateBulkImport: migrated (0.0111s) ========================

rails db:migrate:down VERSION=20200922133949

== 20200922133949 CreateBulkImport: reverting =================================
-- drop_table(:bulk_imports)
   -> 0.0022s
== 20200922133949 CreateBulkImport: reverted (0.0052s) ========================

CreateBulkImportEntities

rails db:migrate:up VERSION=20200925114522

== 20200925114522 CreateBulkImportEntities: migrating =========================
-- create_table(:bulk_import_entities, {:if_not_exists=>true})
   -> 0.0134s
-- transaction_open?()
   -> 0.0000s
-- current_schema()
   -> 0.0002s
-- execute("ALTER TABLE bulk_import_entities\nADD CONSTRAINT check_13f279f7da\nCHECK ( char_length(source_full_path) <= 255 )\nNOT VALID;\n")
   -> 0.0004s
-- current_schema()
   -> 0.0002s
-- execute("SET statement_timeout TO 0")
   -> 0.0001s
-- execute("ALTER TABLE bulk_import_entities VALIDATE CONSTRAINT check_13f279f7da;")
   -> 0.0004s
-- execute("RESET ALL")
   -> 0.0002s
-- transaction_open?()
   -> 0.0000s
-- current_schema()
   -> 0.0001s
-- execute("ALTER TABLE bulk_import_entities\nADD CONSTRAINT check_715d725ea2\nCHECK ( char_length(destination_name) <= 255 )\nNOT VALID;\n")
   -> 0.0003s
-- current_schema()
   -> 0.0002s
-- execute("ALTER TABLE bulk_import_entities VALIDATE CONSTRAINT check_715d725ea2;")
   -> 0.0003s
-- transaction_open?()
   -> 0.0000s
-- current_schema()
   -> 0.0001s
-- execute("ALTER TABLE bulk_import_entities\nADD CONSTRAINT check_b834fff4d9\nCHECK ( char_length(destination_namespace) <= 255 )\nNOT VALID;\n")
   -> 0.0002s
-- current_schema()
   -> 0.0001s
-- execute("ALTER TABLE bulk_import_entities VALIDATE CONSTRAINT check_b834fff4d9;")
   -> 0.0004s
-- transaction_open?()
   -> 0.0000s
-- current_schema()
   -> 0.0001s
-- execute("ALTER TABLE bulk_import_entities\nADD CONSTRAINT check_796a4d9cc6\nCHECK ( char_length(jid) <= 255 )\nNOT VALID;\n")
   -> 0.0002s
-- current_schema()
   -> 0.0001s
-- execute("ALTER TABLE bulk_import_entities VALIDATE CONSTRAINT check_796a4d9cc6;")
   -> 0.0003s
== 20200925114522 CreateBulkImportEntities: migrated (0.0278s) ================

rails db:migrate:down VERSION=20200925114522

== 20200925114522 CreateBulkImportEntities: reverting =========================
-- drop_table(:bulk_import_entities)
   -> 0.0031s
== 20200925114522 CreateBulkImportEntities: reverted (0.0032s) ================

AddBulkImportForeignKeyToBulkImportEntities

rails db:migrate:up VERSION=20200925153423

== 20200925153423 AddBulkImportForeignKeyToBulkImportEntities: migrating ======
-- transaction_open?()
   -> 0.0000s
-- foreign_keys(:bulk_import_entities)
   -> 0.0030s
-- execute("ALTER TABLE bulk_import_entities\nADD CONSTRAINT fk_b69fa2b2df\nFOREIGN KEY (bulk_import_id)\nREFERENCES bulk_imports (id)\nON DELETE CASCADE\nNOT VALID;\n")
   -> 0.0011s
-- execute("SET statement_timeout TO 0")
   -> 0.0001s
-- execute("ALTER TABLE bulk_import_entities VALIDATE CONSTRAINT fk_b69fa2b2df;")
   -> 0.0018s
-- execute("RESET ALL")
   -> 0.0002s
== 20200925153423 AddBulkImportForeignKeyToBulkImportEntities: migrated (0.0102s)

rails db:migrate:down VERSION=20200925153423

== 20200925153423 AddBulkImportForeignKeyToBulkImportEntities: reverting ======
-- remove_foreign_key(:bulk_import_entities, {:column=>:bulk_import_id})
   -> 0.0041s
== 20200925153423 AddBulkImportForeignKeyToBulkImportEntities: reverted (0.0042s)

AddParentForeignKeyToBulkImportEntities

rails db:migrate:up VERSION=20200925193815

== 20200925193815 AddParentForeignKeyToBulkImportEntities: migrating ==========
-- transaction_open?()
   -> 0.0000s
-- foreign_keys(:bulk_import_entities)
   -> 0.0031s
-- execute("ALTER TABLE bulk_import_entities\nADD CONSTRAINT fk_a44ff95be5\nFOREIGN KEY (parent_id)\nREFERENCES bulk_import_entities (id)\nON DELETE CASCADE\nNOT VALID;\n")
   -> 0.0011s
-- execute("SET statement_timeout TO 0")
   -> 0.0001s
-- execute("ALTER TABLE bulk_import_entities VALIDATE CONSTRAINT fk_a44ff95be5;")
   -> 0.0009s
-- execute("RESET ALL")
   -> 0.0002s
== 20200925193815 AddParentForeignKeyToBulkImportEntities: migrated (0.0096s) =

rails db:migrate:down VERSION=20200925193815

== 20200925193815 AddParentForeignKeyToBulkImportEntities: reverting ==========
-- remove_foreign_key(:bulk_import_entities, {:column=>:parent_id})
   -> 0.0047s
== 20200925193815 AddParentForeignKeyToBulkImportEntities: reverted (0.0048s) =

AddNamespaceForeignKeyToBulkImportEntities

rails db:migrate:up VERSION=20200925193906

== 20200925193906 AddNamespaceForeignKeyToBulkImportEntities: migrating =======
-- transaction_open?()
   -> 0.0000s
-- foreign_keys(:bulk_import_entities)
   -> 0.0029s
-- execute("ALTER TABLE bulk_import_entities\nADD CONSTRAINT fk_88c725229f\nFOREIGN KEY (namespace_id)\nREFERENCES namespaces (id)\nON DELETE CASCADE\nNOT VALID;\n")
   -> 0.0014s
-- execute("SET statement_timeout TO 0")
   -> 0.0001s
-- execute("ALTER TABLE bulk_import_entities VALIDATE CONSTRAINT fk_88c725229f;")
   -> 0.0019s
-- execute("RESET ALL")
   -> 0.0002s
== 20200925193906 AddNamespaceForeignKeyToBulkImportEntities: migrated (0.0099s)

rails db:migrate:down VERSION=20200925193906

== 20200925193906 AddNamespaceForeignKeyToBulkImportEntities: reverting =======
-- remove_foreign_key(:bulk_import_entities, {:column=>:namespace_id})
   -> 0.0040s
== 20200925193906 AddNamespaceForeignKeyToBulkImportEntities: reverted (0.0074s)

AddProjectForeignKeyToBulkImportEntities

rails db:migrate:up VERSION=20200925194006

== 20200925194006 AddProjectForeignKeyToBulkImportEntities: migrating =========
-- transaction_open?()
   -> 0.0000s
-- foreign_keys(:bulk_import_entities)
   -> 0.0029s
-- execute("ALTER TABLE bulk_import_entities\nADD CONSTRAINT fk_d06d023c30\nFOREIGN KEY (project_id)\nREFERENCES projects (id)\nON DELETE CASCADE\nNOT VALID;\n")
   -> 0.0016s
-- execute("SET statement_timeout TO 0")
   -> 0.0002s
-- execute("ALTER TABLE bulk_import_entities VALIDATE CONSTRAINT fk_d06d023c30;")
   -> 0.0034s
-- execute("RESET ALL")
   -> 0.0002s
== 20200925194006 AddProjectForeignKeyToBulkImportEntities: migrated (0.0119s)

rails db:migrate:down VERSION=20200925194006

== 20200925194006 AddProjectForeignKeyToBulkImportEntities: reverting =========
-- remove_foreign_key(:bulk_import_entities, {:column=>:project_id})
   -> 0.0048s
== 20200925194006 AddProjectForeignKeyToBulkImportEntities: reverted (0.0086s)

CreateImportConfigurations

rails db:migrate:up VERSION=20200925112104

== 20200925112104 CreateBulkImportConfigurations: migrating ===================
-- create_table(:bulk_import_configurations, {:if_not_exists=>true})
   -> 0.0093s
== 20200925112104 CreateBulkImportConfigurations: migrated (0.0094s) ==========

rails db:migrate:down VERSION=20200925112104

== 20200925112104 CreateBulkImportConfigurations: reverting ===================
-- drop_table(:bulk_import_configurations)
   -> 0.0054s
== 20200925112104 CreateBulkImportConfigurations: reverted (0.0055s) ==========

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team