Importer: Ensure specs produce no `ImportFailure`s
<!--IssueSummary start-->
<details>
<summary>
Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards.
</summary>
- [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=207208)
</details>
<!--IssueSummary end-->
### Problem
Last year we started to track individually failing relations by inserting them as `ImportFailure`s instead of failing fast. Unfortunately this has led to several problems going under the radar. On `master`, there are several specs that currently pass, but where the imports that are being tested are actually partial failures because `ImportFailure`s exist. Since we do not assert that there are no failures currently, these tests produce false negatives since they are green when they should be red instead.
### Goal
We need to investigate and fix all existing `ImportFailure`s in tests. Once fixed, we should then have any import spec that is a functional test case (i.e. where actual imports execute) assert that `ImportFailure.all` is always empty, so that we can detect regressions quickly and easily.
### Approach
I suggest we add a test assertion such as `expect(ImportFailure.all).to eq([])` to the following specs:
* `spec/lib/gitlab/import_export/project_tree_restorer_spec.rb`
* `spec/lib/gitlab/import_export/import_export_equivalence_spec.rb`
and perhaps others that I might be unaware of.
We need to investigate and fix the following errors so that we arrive at a green build with `0` import failures.
The `project_tree_restorer_spec` would currently fail on the following items:
```ruby
+[#<ImportFailure:0x000055f16e263af8
+ id: 1,
+ relation_index: 0,
+ project_id: 1,
+ created_at: Wed, 19 Feb 2020 08:39:31 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_external_pull_requests_on_project_and_branches\"\n" +
+ "DETAIL: Key (project_id, source_branch, target_branch)=(1, feature, master) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">]
```
The `import_export_equivalence_spec` spec would currently fail on the following items:
```ruby
+[#<ImportFailure:0x000055f16cd64f80
+ id: 18,
+ relation_index: 0,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:06 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "Gitlab::Git::Repository::NoRepository",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "5:GetRepoPath: not a git repository '/home/git/gitlab/tmp/tests/repositories/@hashed/a2/18/a21855da08cb102d1d217c53dc5824a3a795c1c1a44e971bf01ab9da3a2acbbf.git'",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cd64c10
+ id: 19,
+ relation_index: 1,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:08 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveModel::UnknownAttributeError",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message: "unknown attribute 'diff_head_sha' for MergeRequest.",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cd64a80
+ id: 20,
+ relation_index: 0,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:08 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_external_pull_requests_on_project_and_branches\"\n" +
+ "DETAIL: Key (project_id, source_branch, target_branch)=(68, feature, master) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cd649b8
+ id: 21,
+ relation_index: 1,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:08 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordInvalid",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message: "Validation failed: Owner can't be blank",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cd648f0
+ id: 22,
+ relation_index: 0,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordInvalid",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "Validation failed: Sentry issue is already associated to a GitLab Issue. New issue will not be associated.",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cd64800
+ id: 23,
+ relation_index: 1,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_issues_on_project_id_and_iid\"\n" +
+ "DETAIL: Key (project_id, iid)=(68, 9) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cd64738
+ id: 24,
+ relation_index: 2,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_issues_on_project_id_and_iid\"\n" +
+ "DETAIL: Key (project_id, iid)=(68, 8) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cd64648
+ id: 25,
+ relation_index: 3,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_issues_on_project_id_and_iid\"\n" +
+ "DETAIL: Key (project_id, iid)=(68, 7) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cd64580
+ id: 26,
+ relation_index: 4,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_issues_on_project_id_and_iid\"\n" +
+ "DETAIL: Key (project_id, iid)=(68, 6) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cd644b8
+ id: 27,
+ relation_index: 5,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_issues_on_project_id_and_iid\"\n" +
+ "DETAIL: Key (project_id, iid)=(68, 5) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cd643f0
+ id: 28,
+ relation_index: 6,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_issues_on_project_id_and_iid\"\n" +
+ "DETAIL: Key (project_id, iid)=(68, 4) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cd64328
+ id: 29,
+ relation_index: 7,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_issues_on_project_id_and_iid\"\n" +
+ "DETAIL: Key (project_id, iid)=(68, 3) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cd641e8
+ id: 30,
+ relation_index: 8,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_issues_on_project_id_and_iid\"\n" +
+ "DETAIL: Key (project_id, iid)=(68, 2) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdabf98
+ id: 31,
+ relation_index: 9,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_issues_on_project_id_and_iid\"\n" +
+ "DETAIL: Key (project_id, iid)=(68, 1) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdabe58
+ id: 32,
+ relation_index: 0,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "Gitlab::Git::Repository::NoRepository",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "5:GetRepoPath: not a git repository '/home/git/gitlab/tmp/tests/repositories/@hashed/a2/18/a21855da08cb102d1d217c53dc5824a3a795c1c1a44e971bf01ab9da3a2acbbf.git'",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdabcf0
+ id: 33,
+ relation_index: 1,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_merge_requests_on_target_project_id_and_iid\"\n" +
+ "DETAIL: Key (target_project_id, iid)=(68, 7) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdabc28
+ id: 34,
+ relation_index: 2,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_merge_requests_on_target_project_id_and_iid\"\n" +
+ "DETAIL: Key (target_project_id, iid)=(68, 6) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdabae8
+ id: 35,
+ relation_index: 3,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_merge_requests_on_target_project_id_and_iid\"\n" +
+ "DETAIL: Key (target_project_id, iid)=(68, 5) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdaba20
+ id: 36,
+ relation_index: 4,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_merge_requests_on_target_project_id_and_iid\"\n" +
+ "DETAIL: Key (target_project_id, iid)=(68, 4) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdab958
+ id: 37,
+ relation_index: 5,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_merge_requests_on_target_project_id_and_iid\"\n" +
+ "DETAIL: Key (target_project_id, iid)=(68, 3) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdab818
+ id: 38,
+ relation_index: 6,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:09 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_merge_requests_on_target_project_id_and_iid\"\n" +
+ "DETAIL: Key (target_project_id, iid)=(68, 2) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdab728
+ id: 39,
+ relation_index: 7,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:10 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_merge_requests_on_target_project_id_and_iid\"\n" +
+ "DETAIL: Key (target_project_id, iid)=(68, 1) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdab610
+ id: 40,
+ relation_index: 1,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:10 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_ci_pipelines_on_project_id_and_iid\"\n" +
+ "DETAIL: Key (project_id, iid)=(68, 1) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdab548
+ id: 41,
+ relation_index: 3,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:10 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_external_pull_requests_on_project_and_branches\"\n" +
+ "DETAIL: Key (project_id, source_branch, target_branch)=(68, feature, master) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdab458
+ id: 42,
+ relation_index: 0,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:10 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordNotUnique",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message:
+ "PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_external_pull_requests_on_project_and_branches\"\n" +
+ "DETAIL: Key (project_id, source_branch, target_branch)=(68, feature, master) already exists.\n",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdab2f0
+ id: 43,
+ relation_index: 0,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:10 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordInvalid",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message: "Validation failed: Name has already been taken",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdab200
+ id: 44,
+ relation_index: 0,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:10 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordInvalid",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message: "Validation failed: Key has already been taken",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">,
+ #<ImportFailure:0x000055f16cdaaf80
+ id: 45,
+ relation_index: 1,
+ project_id: 68,
+ created_at: Wed, 19 Feb 2020 08:40:10 UTC +00:00,
+ relation_key: [FILTERED],
+ exception_class: "ActiveRecord::RecordInvalid",
+ correlation_id_value: "6c09c08669890da38ceffaa948e521eb",
+ exception_message: "Validation failed: Key has already been taken",
+ retry_count: 0,
+ group_id: nil,
+ source: "process_relation_item!">]
```
issue