Persist gists import failures
What does this MR do and why?
If a gist fails to be imported, we will save the failure in the database to be able to present any failed imports to the user in the future.
This implements Track gists import failure.
Screenshots or screen recordings
How to set up and validate locally
- Use GitLab API to try to import one or more GitHub gists with more than 10 files.
- In rails console make sure that related failures have been saved to the database:
ImportFailure.where(user: user) # 'user' - GitLab user who initiated the gists import
MR acceptance checklist
-
I have evaluated the MR acceptance checklist for this MR.
Merge request reports
Activity
changed milestone to %16.0
assigned to @cervols-ext
- A deleted user
added database databasereview pending labels
1 Warning a9fe0816: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines. Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer backend Ian Baum (
@ibaum
) (UTC-5)Mark Chao (
@lulalala
) (UTC+8)database Vitali Tatarintev (
@ck3g
) (UTC+2)Adam Hegyi (
@ahegyi
) (UTC+2)~"migration" No reviewer available No maintainer available To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
Danger- A deleted user
added Data WarehouseImpact Check label
- Resolved by Ahmed Hemdan
Hey @ghavenga, would you mind doing the initial backend review for this MR?
Hi @tianwenchen, may I ask you to do database review, please?
requested review from @ghavenga
requested review from @tianwenchen
added workflowin review label and removed workflowin dev label
- Resolved by Ahmed Hemdan
This has been identified as a contribution from a Temporary Service Provider to GitLab. It is encouraged to seek review from at least one domain expert during the review process.
added Contractor Contribution label
- Resolved by Ahmed Hemdan
Database migrations (on the main database)
1 Warnings 20230510142931 - AddForeignKeyForUserIdToImportFailures had a query that exceeded timing
guidelines. Run time should not exceed 100ms, but it was 120.09ms. Please consider possible options
to improve the query performance.ALTER TABLE import_failures VALIDATE CONSTRAINT
fk_9a9b9ba21cMigrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).
Migration Type Total runtime Result DB size change 20230510141454 - AddUserIdToImportFailures Regular 2.1 s +0.00 B 20230510142316 - AddIndexOnUserIdToImportFailures Regular 81.4 s +8.00 KiB 20230510142931 - AddForeignKeyForUserIdToImportFailures Regular 2.2 s +0.00 B Runtime Histogram for all migrations
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 12 0.1 seconds - 1 second 1 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 1 5 minutes + 0 Migration: 20230510141454 - AddUserIdToImportFailures
- Type: Regular
- Duration: 2.1 s
- Database size change: +0.00 B
Calls Total Time Max Time Mean Time Rows Query 1 5.4 ms 5.4 ms 5.4 ms 825 SELECT "features".*
FROM "features"1 4.0 ms 4.0 ms 4.0 ms 0 ALTER TABLE "import_failures" ADD "user_id" bigint
2 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for AddUserIdToImportFailures
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 4 0.1 seconds - 1 second 0 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20230510142316 - AddIndexOnUserIdToImportFailures
- Type: Regular
- Duration: 81.4 s
- Database size change: +8.00 KiB
Calls Total Time Max Time Mean Time Rows Query 1 78925.7 ms 78925.7 ms 78925.7 ms 0 CREATE INDEX CONCURRENTLY "index_import_failures_on_user_id_not_null" ON "import_failures" ("user_id")
WHERE user_id IS NOT NULL2 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for AddIndexOnUserIdToImportFailures
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 2 0.1 seconds - 1 second 0 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 1 5 minutes + 0 Migration: 20230510142931 - AddForeignKeyForUserIdToImportFailures- Type: Regular
- Duration: 2.2 s
- Database size change: +0.00 B
Calls Total Time Max Time Mean Time Rows Query 1 120.1 ms 120.1 ms 120.1 ms 0 ALTER TABLE import_failures VALIDATE CONSTRAINT fk_9a9b9ba21c
1 42.8 ms 42.8 ms 42.8 ms 0 ALTER TABLE import_failures ADD CONSTRAINT fk_9a9b9ba21c FOREIGN KEY (user_id) REFERENCES users (id) ON DELETE CASCADE NOT VALID
2 0.4 ms 0.2 ms 0.2 ms 0 SELECT "postgres_partitioned_tables".*
FROM "postgres_partitioned_tables" WHERE (identifier = concat(current_schema(), $1, $2))
LIMIT $31 0.1 ms 0.1 ms 0.1 ms 0 SELECT $1 AS one
FROM "postgres_foreign_keys" WHERE "postgres_foreign_keys"."constrained_table_name" = $2 AND "postgres_foreign_keys"."referenced_table_name" = $3 AND "postgres_foreign_keys"."name" = $4 AND "postgres_foreign_keys"."constrained_columns" = $5 AND "postgres_foreign_keys"."referenced_columns" = $6 AND "postgres_foreign_keys"."on_delete_action" = $7
LIMIT $82 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for AddForeignKeyForUserIdToImportFailures
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 6 0.1 seconds - 1 second 1 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Other information
No other migrations pending on GitLab.com
Clone details
Clone ID Clone Created At Clone Data Timestamp Expected Removal Time database-testing-1928995-10016763-main
2023-05-11T05:28:02Z 2023-05-11T03:05:00Z 2023-05-11 17:35:12 +0000 database-testing-1928995-10016763-ci
2023-05-11T05:28:02Z 2023-05-11T04:45:16Z 2023-05-11 17:35:12 +0000 Database migrations (on the ci database)
2 Warnings 20230510142931 - AddForeignKeyForUserIdToImportFailures had a query that exceeded timing
guidelines. Run time should not exceed 100ms, but it was 604.26ms. Please consider possible options
to improve the query performance.ALTER TABLE import_failures VALIDATE CONSTRAINT
fk_9a9b9ba21c20230510142931 - AddForeignKeyForUserIdToImportFailures had a query that exceeded timing
guidelines. Run time should not exceed 100ms, but it was 197.88ms. Please consider possible options
to improve the query performance.ALTER TABLE import_failures ADD CONSTRAINT fk_9a9b9ba21c
FOREIGN KEY (user_id) REFERENCES users (id) ON DELETE CASCADE NOT VALIDMigrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).
Migration Type Total runtime Result DB size change 20230510141454 - AddUserIdToImportFailures Regular 2.3 s +0.00 B 20230510142316 - AddIndexOnUserIdToImportFailures Regular 3.3 s +8.00 KiB 20230510142931 - AddForeignKeyForUserIdToImportFailures Regular 3.6 s +0.00 B Runtime Histogram for all migrations
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 10 0.1 seconds - 1 second 3 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20230510141454 - AddUserIdToImportFailures
- Type: Regular
- Duration: 2.3 s
- Database size change: +0.00 B
Calls Total Time Max Time Mean Time Rows Query 1 16.4 ms 16.4 ms 16.4 ms 0 ALTER TABLE "import_failures" ADD "user_id" bigint
2 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for AddUserIdToImportFailures
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 3 0.1 seconds - 1 second 0 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20230510142316 - AddIndexOnUserIdToImportFailures
- Type: Regular
- Duration: 3.3 s
- Database size change: +8.00 KiB
Calls Total Time Max Time Mean Time Rows Query 1 33.4 ms 33.4 ms 33.4 ms 0 CREATE INDEX CONCURRENTLY "index_import_failures_on_user_id_not_null" ON "import_failures" ("user_id")
WHERE user_id IS NOT NULL2 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for AddIndexOnUserIdToImportFailures
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 3 0.1 seconds - 1 second 0 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20230510142931 - AddForeignKeyForUserIdToImportFailures- Type: Regular
- Duration: 3.6 s
- Database size change: +0.00 B
Calls Total Time Max Time Mean Time Rows Query 1 604.3 ms 604.3 ms 604.3 ms 0 ALTER TABLE import_failures VALIDATE CONSTRAINT fk_9a9b9ba21c
1 197.9 ms 197.9 ms 197.9 ms 0 ALTER TABLE import_failures ADD CONSTRAINT fk_9a9b9ba21c FOREIGN KEY (user_id) REFERENCES users (id) ON DELETE CASCADE NOT VALID
1 10.3 ms 10.3 ms 10.3 ms 0 SELECT $1 AS one
FROM "postgres_foreign_keys" WHERE "postgres_foreign_keys"."constrained_table_name" = $2 AND "postgres_foreign_keys"."referenced_table_name" = $3 AND "postgres_foreign_keys"."name" = $4 AND "postgres_foreign_keys"."constrained_columns" = $5 AND "postgres_foreign_keys"."referenced_columns" = $6 AND "postgres_foreign_keys"."on_delete_action" = $7
LIMIT $82 0.5 ms 0.3 ms 0.3 ms 0 SELECT "postgres_partitioned_tables".*
FROM "postgres_partitioned_tables" WHERE (identifier = concat(current_schema(), $1, $2))
LIMIT $32 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for AddForeignKeyForUserIdToImportFailures
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 4 0.1 seconds - 1 second 3 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Other information
No other migrations pending on GitLab.com
Clone details
Clone ID Clone Created At Clone Data Timestamp Expected Removal Time database-testing-1928995-10016763-main
2023-05-11T05:28:02Z 2023-05-11T03:05:00Z 2023-05-11 17:35:12 +0000 database-testing-1928995-10016763-ci
2023-05-11T05:28:02Z 2023-05-11T04:45:16Z 2023-05-11 17:35:12 +0000
Brought to you by gitlab-org/database-team/gitlab-com-database-testing. Epic
- A deleted user
added database-testing-automation label
- Resolved by Ahmed Hemdan
@tianwenchen
, thanks for approving this merge request.This is the first time the merge request is approved. To ensure full test coverage, please start a new pipeline before merging.
For more info, please refer to the following links:
added pipeline:mr-approved label
removed review request for @tianwenchen
requested review from @ahegyi
- Resolved by Ahmed Hemdan
- Resolved by Ahmed Hemdan
- Resolved by Ahmed Hemdan
- Resolved by Ahmed Hemdan
removed review request for @minac
- Resolved by Adam Hegyi
- Resolved by Tetiana Zavediuk
added databaseapproved label and removed databasereview pending label
added Data WarehouseNot Impacted label and removed Data WarehouseImpact Check label
added 1686 commits
-
3b92f82b...96c0c642 - 1684 commits from branch
master
- df60e824 - Persist gists import failures
- 7743c371 - Apply reviewer's suggestions
-
3b92f82b...96c0c642 - 1684 commits from branch
requested review from @ahmed.hemdan
changed milestone to %16.1
added missed:16.0 label
- Resolved by Tetiana Zavediuk
- Resolved by Ahmed Hemdan
- Resolved by Tetiana Zavediuk
- Resolved by Tetiana Zavediuk
added typemaintenance label and removed typefeature label
added 444 commits
-
7743c371...dcb66ee6 - 441 commits from branch
master
- 97e38004 - Persist gists import failures
- a9fe0816 - Apply reviewer's suggestions
- 4a61fbf6 - Apply new suggestions to backend
Toggle commit list-
7743c371...dcb66ee6 - 441 commits from branch
added maintenancerefactor label
enabled an automatic merge when the pipeline for b30e344a succeeds
Hello @cervols-ext
The database team is looking for ways to improve the database review process and we would love your help!
If you'd be open to someone on the database team reaching out to you for a chat, or if you'd like to leave some feedback asynchronously, just post a reply to this comment mentioning:
@gitlab-org/database-team
And someone will be by shortly!
Thanks for your help!
This message was generated automatically. You're welcome to improve it.
mentioned in commit bc8c22f1
added workflowstaging-canary label and removed workflowin review label
added workflowcanary label and removed workflowstaging-canary label
added workflowstaging label and removed workflowcanary label
added workflowproduction label and removed workflowstaging label
added releasedcandidate label
mentioned in merge request kubitus-project/kubitus-installer!2224 (merged)