Seed Default Organization in Production and Development
What does this MR do and why?
In !120118 (merged), we added the concept of a Default Organization. This organization was added using a database migration so gitlab.com does have this organization.
However, we also need to be sure that this organization is created on new installations. We do not run old migrations during the installation of a GitLab instance
This MR adds creation of the default organization to both development and production.
It also includes a migration that conditionally creates the Default Organization, for self-managed instances that were installed after the original migration.
Installed using GitLab version... | Has the default organization? | Solution |
---|---|---|
Before %16.0 | Yes | - |
%16.0 | No | Will run migration from this MR |
%16.1 and newer | No | Add Default Organization as part of DB seeding |
Related issue #411441 (closed)
How to set up and validate locally
Rails Env Development:
git checkout master
gdk reset-data
-
gdk psql -c "SELECT * FROM organizations WHERE id = 1"
Result: no organizations git checkout rw-seed-default-organization
gdk reset-data
-
gdk psql -c "SELECT * FROM organizations WHERE id = 1"
Result: one organization
Rails Env Production:
I ran this on my local development, while having redis listening on the default port:
git checkout master
DISABLE_DATABASE_ENVIRONMENT_CHECK=1 RAILS_ENV=production rake db:environment:set db:drop db:create db:schema:load db:seed_fu
-
gdk psql -c "SELECT * FROM organizations WHERE id = 1"
Result: no organizations git checkout rw-seed-default-organization
DISABLE_DATABASE_ENVIRONMENT_CHECK=1 RAILS_ENV=production rake db:environment:set db:drop db:create db:schema:load db:seed_fu
-
gdk psql -c "SELECT * FROM organizations WHERE id = 1"
Result: one organization
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Merge request reports
Activity
changed milestone to %16.1
added backend grouptenant scale typefeature workflowin dev labels
assigned to @rutgerwessels
added devopsdata stores sectioncore platform labels
2 Warnings New migrations added but db/structure.sql wasn't updated Usually, when adding new migrations, db/structure.sql should be
updated too (unless the migration isn't changing the DB schema
and isn't the most recent one).featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart.
For more information, see:
- The Handbook page on merge request types.
- The definition of done documentation.
Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer backend Hinam Mehra (
@hmehra
) (UTC+10, 8 hours ahead of@rutgerwessels
)Douglas Barbosa Alexandre (
@dbalexandre
) (UTC+0, 2 hours behind@rutgerwessels
)database Jon Jenkins (
@jon_jenkins
) (UTC-5, 7 hours behind@rutgerwessels
)Omar Qunsul (
@OmarQunsulGitlab
) (UTC+3, 1 hour ahead of@rutgerwessels
)~"migration" No reviewer available No maintainer available To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
Dangeradded featureenhancement label
added 1 commit
- 6d76c560 - Seed default organization in development and production databases
Allure report
allure-report-publisher
generated test report!e2e-test-on-gdk:
test report for bbc96ee0expand test summary
+-----------------------------------------------------------------------+ | suites summary | +------------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +------------------+--------+--------+---------+-------+-------+--------+ | Govern | 2 | 0 | 0 | 0 | 2 | ✅ | | Plan | 4 | 0 | 0 | 0 | 4 | ✅ | | Create | 8 | 0 | 1 | 0 | 9 | ✅ | | Monitor | 4 | 0 | 0 | 0 | 4 | ✅ | | Framework sanity | 0 | 0 | 1 | 0 | 1 | ➖ | | Manage | 1 | 0 | 0 | 0 | 1 | ✅ | | Data Stores | 2 | 0 | 0 | 1 | 2 | ❗ | +------------------+--------+--------+---------+-------+-------+--------+ | Total | 21 | 0 | 2 | 1 | 23 | ❗ | +------------------+--------+--------+---------+-------+-------+--------+
added Category:Organization label
@alexpooley What do you think about this?
I am thinking we need a better way to have a guarantee that the default organization does exists
- Resolved by Manoj M J
added pipeline:run-all-rspec label
removed pipeline:run-all-rspec label
added 220 commits
-
6d76c560...36a7e207 - 218 commits from branch
master
- d1611fb1 - Seed default organization in development and production databases
- 48a0d4f1 - Create default organization
-
6d76c560...36a7e207 - 218 commits from branch
- A deleted user
added database databasereview pending labels
- Resolved by Sincheol (David) Kim
@dgruzd can you please review this from database perspective?
And @alexpooley, can you please review from backend perspective?
requested review from @alexpooley and @dgruzd
- Resolved by Dmitry Gruzd
- Resolved by Alex Pooley
removed review request for @dgruzd
- Resolved by Alex Pooley
- Resolved by Alex Pooley
added 396 commits
Toggle commit listrequested review from @dgruzd
- Resolved by Pavel Shutsin
added databasereviewed label and removed databasereview pending label
@dgruzd
, thanks for approving this merge request.This is the first time the merge request is approved. To ensure full test coverage, a new pipeline will be started shortly.
For more info, please refer to the following links:
added pipeline:mr-approved label
Database migrations (on the main database)
Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).
Migration Type Total runtime Result DB size change 20230605095810 - EnsureDefaultOrganization Regular 2.3 s +0.00 B Runtime Histogram for all migrations
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 3 0.1 seconds - 1 second 0 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20230605095810 - EnsureDefaultOrganization
- Type: Regular
- Duration: 2.3 s
- Database size change: +0.00 B
Calls Total Time Max Time Mean Time Rows Query 1 0.2 ms 0.2 ms 0.2 ms 1 SELECT $1 AS one
FROM "organizations" WHERE "organizations"."id" = $2
LIMIT $32 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for EnsureDefaultOrganization
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 3 0.1 seconds - 1 second 0 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Background Migration: MigrateLinksForVulnerabilityFindings
Sampled 78 batches. Estimated Time to complete: 3 weeks, 5 days, 2 hours, and 42 minutes
- Interval: 120s
- Total tuple count: 75206930
- Max batch size:
- Estimated seconds to complete: 2256120s
- Estimated number of batches: 18801
- Average batch time: 10.52s
- Batch size: 4000
- N. of batches sampled: 78
- N. of failed batches: 0
Time estimation is conservative and based on sampling production data in a test environment. It represents the max time that migration could take. The actual time may differ from this estimation.
_Consider changing max_batch_size and interval if this estimate is unacceptable._
Calls Total Time Max Time Mean Time Rows Query 539 501067.0 ms 88395.1 ms 929.6 ms 269500 SELECT "vulnerability_occurrences"."id", "vulnerability_occurrences"."raw_metadata" FROM "vulnerability_occurrences" WHERE "vulnerability_occurrences"."id" BETWEEN $1 AND $2 AND "vulnerability_occurrences"."id" >= $3 AND "vulnerability_occurrences"."id" < $4
78 56673.0 ms 1742.1 ms 726.6 ms 38508 SELECT "vulnerability_occurrences"."id", "vulnerability_occurrences"."raw_metadata" FROM "vulnerability_occurrences" WHERE "vulnerability_occurrences"."id" BETWEEN $1 AND $2 AND "vulnerability_occurrences"."id" >= $3
616 14407.6 ms 427.4 ms 23.4 ms 1335527f SELECT COUNT(*) AS count_all, "vulnerability_finding_links"."vulnerability_occurrence_id" AS vulnerability_finding_links_vulnerability_occurrence_id, "vulnerability_finding_links"."name" AS vulnerability_finding_links_name, "vulnerability_finding_links"."url" AS vulnerability_finding_links_url
FROM "vulnerability_finding_links" WHERE "vulnerability_finding_links"."vulnerability_occurrence_id" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37, $38, $39, $40, $41, $42, $43, $44, $45, $46, $47, $48, $49, $50, $51, $52, $53, $54, $55, $56, $57, $58, $59, $60, $61, $62, $63, $64, $65, $66, $67, $68, $69, $70, $71, $72, $73, $74, $75, $76, $77, $78, $79, $80, $81, $82, $83, $84, $85, $86, $87, $88, $89, $90, $91, $92, $93, $94, $95, $96, $97, $98, $99, $100, $101, $102, $103, $104, $105, $106, $107, $108, $109, $110, $111, $112, $113, $114, $115, $116, $117, $118, $119, $120, $121, $122, $123, $124, $125, $126, $127, $128, $129, $130, $131, $132, $133, $134, $135, $136, $137, $138, $139, $140, $141, $142, $143, $144, $145, $146, $147, $148, $149, $150, $151, $152, $153, $154, $155, $156, $157, $158, $159, $160, $161, $162, $163, $164, $165, $166, $167, $168, $169, $170, $171, $172, $173, $174, $175, $176, $177, $178, $179, $180, $181, $182, $183, $184, $185, $186, $187, $188, $189, $190, $191, $192, $193, $194, $195, $196, $197, $198, $199, $200, $201, $202, $203, $204, $205, $206, $207, $208, $209, $210, $211, $212, $213, $214, $215, $216, $217, $218, $219, $220, $221, $222, $223, $224, $225, $226, $227, $228, $229, $230, $231, $232, $233, $234, $235, $236, $237, $238, $239, $240, $241, $242, $243, $244, $245, $246, $247, $248, $249, $250, $251, $252, $253, $254, $255, $256, $257, $258, $259, $260, $261, $262, $263, $264, $265, $266, $267, $268, $269, $270, $271, $272, $273, $274, $275, $276, $277, $278, $279, $280, $281, $282, $283, $284, $285, $286, $287, $288, $289, $290, $291, $292, $293, $294, $295, $296, $297, $298, $299, $300, $301, $302, $303, $304, $305, $306, $307, $308, $309, $310, $311, $312, $313, $314, $315, $316, $317, $318, $319, $320, $321, $322, $323, $324, $325, $326, $327, $328, $329, $330, $331, $332, $333, $334, $335, $336, $337, $338, $339, $340, $341, $342, $343, $344, $345, $346, $347, $348, $349, $350, $351, $352, $353, $354, $355, $356, $357, $358, $359, $360, $361, $362, $363, $364, $365, $366, $367, $368, $369, $370, $371, $372, $373, $374, $375, $376, $377, $378, $379, $380, $381, $382, $383, $384, $385, $386, $387, $388, $389, $390, $391, $392, $393, $394, $395, $396, $397, $398, $399, $400, $401, $402, $403, $404, $405, $406, $407, $408, $409, $410, $411, $412, $413, $414, $415, $416, $417, $418, $419, $420, $421, $422, $423, $424, $425, $426, $427, $428, $429, $430, $431, $432, $433, $434, $435, $436, $437, $438, $439, $440, $441, $442, $443, $444, $445, $446, $447, $448, $449, $450, $451, $452, $453, $454, $455, $456, $457, $458, $459, $460, $461, $462, $463, $464, $465, $466, $467, $468, $469, $470, $471, $472, $473, $474, $475, $476, $477, $478, $479, $480, $481, $482, $483, $484, $485, $486, $487, $488, $489, $490, $491, $492, $493, $494, $495, $496, $497, $498, $499, $500)
GROUP BY "vulnerability_finding_links"."vulnerability_occurrence_id", "vulnerability_finding_links"."name", "vulnerability_finding_links"."url"1 168.9 ms 168.9 ms 168.9 ms 1 INSERT INTO "vulnerability_finding_links" ("vulnerability_occurrence_id","name","url","created_at","updated_at") VALUES ($1, $2, $3, $4, $5) ON CONFLICT ("id") DO
UPDATE
SET "vulnerability_occurrence_id"=excluded."vulnerability_occurrence_id","name"=excluded."name","url"=excluded."url","created_at"=excluded."created_at","updated_at"=excluded."updated_at"156 49.4 ms 17.9 ms 0.3 ms 156 INSERT INTO "batched_background_migration_job_transition_logs" ("batched_background_migration_job_id", "created_at", "updated_at", "previous_status", "next_status") VALUES ($1, $2, $3, $4, $5) RETURNING "id"
1 14.8 ms 14.8 ms 14.8 ms 0 SELECT COUNT(*) AS count_all, "vulnerability_finding_links"."vulnerability_occurrence_id" AS vulnerability_finding_links_vulnerability_occurrence_id, "vulnerability_finding_links"."name" AS vulnerability_finding_links_name, "vulnerability_finding_links"."url" AS vulnerability_finding_links_url
FROM "vulnerability_finding_links" WHERE "vulnerability_finding_links"."vulnerability_occurrence_id" IN ($1, $2, $3, $4, $5, $6, $7, $8)
GROUP BY "vulnerability_finding_links"."vulnerability_occurrence_id", "vulnerability_finding_links"."name", "vulnerability_finding_links"."url"78 127.8 ms 11.0 ms 1.6 ms 78 UPDATE "batched_background_migration_jobs" SET "updated_at" = $1, "finished_at" = $2, "status" = $3, "metrics" = $4
WHERE "batched_background_migration_jobs"."id" = $53 12.2 ms 10.2 ms 4.1 ms 72 INSERT INTO "vulnerability_finding_links" ("vulnerability_occurrence_id","name","url","created_at","updated_at") VALUES ($1, $2, $3, $4, $5), ($6, $7, $8, $9, $10), ($11, $12, $13, $14, $15), ($16, $17, $18, $19, $20), ($21, $22, $23, $24, $25), ($26, $27, $28, $29, $30), ($31, $32, $33, $34, $35), ($36, $37, $38, $39, $40), ($41, $42, $43, $44, $45), ($46, $47, $48, $49, $50), ($51, $52, $53, $54, $55), ($56, $57, $58, $59, $60), ($61, $62, $63, $64, $65), ($66, $67, $68, $69, $70), ($71, $72, $73, $74, $75), ($76, $77, $78, $79, $80), ($81, $82, $83, $84, $85), ($86, $87, $88, $89, $90), ($91, $92, $93, $94, $95), ($96, $97, $98, $99, $100), ($101, $102, $103, $104, $105), ($106, $107, $108, $109, $110), ($111, $112, $113, $114, $115), ($116, $117, $118, $119, $120) ON CONFLICT ("id") DO
UPDATE
SET "vulnerability_occurrence_id"=excluded."vulnerability_occurrence_id","name"=excluded."name","url"=excluded."url","created_at"=excluded."created_at","updated_at"=excluded."updated_at"78 56.4 ms 8.7 ms 0.7 ms 78 UPDATE "batched_background_migration_jobs" SET "updated_at" = $1, "started_at" = $2, "status" = $3, "attempts" = $4
WHERE "batched_background_migration_jobs"."id" = $52 9.3 ms 8.6 ms 4.6 ms 12 INSERT INTO "vulnerability_finding_links" ("vulnerability_occurrence_id","name","url","created_at","updated_at") VALUES ($1, $2, $3, $4, $5), ($6, $7, $8, $9, $10), ($11, $12, $13, $14, $15), ($16, $17, $18, $19, $20), ($21, $22, $23, $24, $25), ($26, $27, $28, $29, $30) ON CONFLICT ("id") DO
UPDATE
SET "vulnerability_occurrence_id"=excluded."vulnerability_occurrence_id","name"=excluded."name","url"=excluded."url","created_at"=excluded."created_at","updated_at"=excluded."updated_at"1 7.8 ms 7.8 ms 7.8 ms 2 INSERT INTO "vulnerability_finding_links" ("vulnerability_occurrence_id","name","url","created_at","updated_at") VALUES ($1, $2, $3, $4, $5), ($6, $7, $8, $9, $10) ON CONFLICT ("id") DO
UPDATE
SET "vulnerability_occurrence_id"=excluded."vulnerability_occurrence_id","name"=excluded."name","url"=excluded."url","created_at"=excluded."created_at","updated_at"=excluded."updated_at"1 6.5 ms 6.5 ms 6.5 ms 8 INSERT INTO "vulnerability_finding_links" ("vulnerability_occurrence_id","name","url","created_at","updated_at") VALUES ($1, $2, $3, $4, $5), ($6, $7, $8, $9, $10), ($11, $12, $13, $14, $15), ($16, $17, $18, $19, $20), ($21, $22, $23, $24, $25), ($26, $27, $28, $29, $30), ($31, $32, $33, $34, $35), ($36, $37, $38, $39, $40) ON CONFLICT ("id") DO
UPDATE
SET "vulnerability_occurrence_id"=excluded."vulnerability_occurrence_id","name"=excluded."name","url"=excluded."url","created_at"=excluded."created_at","updated_at"=excluded."updated_at"617 159.6 ms 2.9 ms 0.3 ms 539 SELECT "vulnerability_occurrences"."id" FROM "vulnerability_occurrences" WHERE "vulnerability_occurrences"."id" BETWEEN $1 AND $2 AND "vulnerability_occurrences"."id" >= $3
ORDER BY "vulnerability_occurrences"."id" ASC
LIMIT $4
OFFSET $578 11.3 ms 0.4 ms 0.1 ms 78 SELECT SUM("batched_background_migration_jobs"."batch_size")
FROM "batched_background_migration_jobs" WHERE "batched_background_migration_jobs"."batched_background_migration_id" = $1 AND ("batched_background_migration_jobs"."status" IN ($2))156 7.7 ms 0.2 ms 0.0 ms 156 SELECT "batched_background_migration_jobs".*
FROM "batched_background_migration_jobs" WHERE "batched_background_migration_jobs"."id" = $1
LIMIT $278 3.2 ms 0.1 ms 0.0 ms 78 SELECT "batched_background_migrations".*
FROM "batched_background_migrations" WHERE "batched_background_migrations"."id" = $1
LIMIT $278 3.5 ms 0.1 ms 0.0 ms 78 SELECT "vulnerability_occurrences"."id" FROM "vulnerability_occurrences" WHERE "vulnerability_occurrences"."id" BETWEEN $1 AND $2
ORDER BY "vulnerability_occurrences"."id" ASC
LIMIT $3Histogram of batch runtimes for MigrateLinksForVulnerabilityFindings
Batch Runtime Count 0 seconds - 10 seconds 52 10 seconds - 1 minute 25 1 minute - 2 minutes 1 2 minutes - 3 minutes 0 3 minutes - 5 minutes 0 5 minutes + 0 Histogram across all sampled batches of MigrateLinksForVulnerabilityFindings
Query Runtime Count 0 seconds - 0.1 seconds 1841 0.1 seconds - 0.5 seconds 224 0.5 seconds - 1 second 327 1 second - 2 seconds 166 2 seconds - 5 seconds 1 5 seconds + 2 Background Migration: BackfillRootStorageStatisticsForkStorageSizes
Sampled 8 batches. Estimated Time to complete: 6 days, 17 hours, and 18 minutes
- Interval: 120s
- Total tuple count: 4839935
- Max batch size:
- Estimated seconds to complete: 580680s
- Estimated number of batches: 4839
- Average batch time: 115.69s
- Batch size: 1000
- N. of batches sampled: 8
- N. of failed batches: 0
Time estimation is conservative and based on sampling production data in a test environment. It represents the max time that migration could take. The actual time may differ from this estimation.
_Consider changing max_batch_size and interval if this estimate is unacceptable._
Calls Total Time Max Time Mean Time Rows Query 4282 127378.7 ms 67362.9 ms 29.7 ms 638 SELECT SUM("project_statistics"."storage_size") AS sum_project_statistics_storage_size, "projects"."visibility_level" AS projects_visibility_level
FROM "projects" INNER JOIN "project_statistics" ON "project_statistics"."project_id" = "projects"."id" INNER JOIN "fork_network_members" ON "fork_network_members"."project_id" = "projects"."id" INNER JOIN "fork_networks" ON "fork_networks"."id" = "fork_network_members"."fork_network_id" WHERE "projects"."namespace_id" = $1 AND (fork_networks.root_project_id != projects.id)
GROUP BY "projects"."visibility_level"2672 36092.7 ms 1345.5 ms 13.5 ms 82 SELECT SUM("project_statistics"."storage_size") AS sum_project_statistics_storage_size, "projects"."visibility_level" AS projects_visibility_level
FROM "projects" INNER JOIN "project_statistics" ON "project_statistics"."project_id" = "projects"."id" INNER JOIN "fork_network_members" ON "fork_network_members"."project_id" = "projects"."id" INNER JOIN "fork_networks" ON "fork_networks"."id" = "fork_network_members"."fork_network_id" WHERE "projects"."namespace_id" IN (
SELECT namespaces.traversal_ids[array_length(namespaces.traversal_ids, $1)] AS id
FROM "namespaces" WHERE "namespaces"."type" = $2 AND (traversal_ids @> ($3))
) AND (fork_networks.root_project_id != projects.id)
GROUP BY "projects"."visibility_level"6954 19795.1 ms 640.5 ms 2.8 ms 6954 SELECT type
FROM namespaces
WHERE id = $163 10291.9 ms 353.1 ms 163.4 ms 6300 SELECT "namespace_root_storage_statistics".*
FROM "namespace_root_storage_statistics" WHERE "namespace_root_storage_statistics"."namespace_id" BETWEEN $1 AND $2 AND "namespace_root_storage_statistics"."namespace_id" >= $3 AND "namespace_root_storage_statistics"."namespace_id" < $427 277.5 ms 245.9 ms 10.3 ms 27 UPDATE "namespace_root_storage_statistics" SET "updated_at" = $1, "public_forks_storage_size" = $2, "private_forks_storage_size" = $3
WHERE "namespace_root_storage_statistics"."namespace_id" = $48 784.2 ms 182.1 ms 98.0 ms 748 SELECT "namespace_root_storage_statistics".*
FROM "namespace_root_storage_statistics" WHERE "namespace_root_storage_statistics"."namespace_id" BETWEEN $1 AND $2 AND "namespace_root_storage_statistics"."namespace_id" >= $3291 266.2 ms 36.6 ms 0.9 ms 291 UPDATE "namespace_root_storage_statistics" SET "updated_at" = $1, "public_forks_storage_size" = $2
WHERE "namespace_root_storage_statistics"."namespace_id" = $3345 229.1 ms 24.7 ms 0.7 ms 345 UPDATE "namespace_root_storage_statistics" SET "updated_at" = $1, "private_forks_storage_size" = $2
WHERE "namespace_root_storage_statistics"."namespace_id" = $38 15.3 ms 14.5 ms 1.9 ms 8 UPDATE "batched_background_migration_jobs" SET "updated_at" = $1, "finished_at" = $2, "status" = $3, "metrics" = $4
WHERE "batched_background_migration_jobs"."id" = $516 14.3 ms 11.5 ms 0.9 ms 16 INSERT INTO "batched_background_migration_job_transition_logs" ("batched_background_migration_job_id", "created_at", "updated_at", "previous_status", "next_status") VALUES ($1, $2, $3, $4, $5) RETURNING "id"
1 5.2 ms 5.2 ms 5.2 ms 1 UPDATE "namespace_root_storage_statistics" SET "updated_at" = $1, "public_forks_storage_size" = $2, "internal_forks_storage_size" = $3
WHERE "namespace_root_storage_statistics"."namespace_id" = $44 6.1 ms 4.0 ms 1.5 ms 4 UPDATE "namespace_root_storage_statistics" SET "updated_at" = $1, "internal_forks_storage_size" = $2
WHERE "namespace_root_storage_statistics"."namespace_id" = $31 3.4 ms 3.4 ms 3.4 ms 1 UPDATE "namespace_root_storage_statistics" SET "updated_at" = $1, "public_forks_storage_size" = $2, "internal_forks_storage_size" = $3, "private_forks_storage_size" = $4
WHERE "namespace_root_storage_statistics"."namespace_id" = $571 7.8 ms 0.4 ms 0.1 ms 63 SELECT "namespace_root_storage_statistics"."namespace_id" FROM "namespace_root_storage_statistics" WHERE "namespace_root_storage_statistics"."namespace_id" BETWEEN $1 AND $2 AND "namespace_root_storage_statistics"."namespace_id" >= $3
ORDER BY "namespace_root_storage_statistics"."namespace_id" ASC
LIMIT $4
OFFSET $58 0.9 ms 0.2 ms 0.1 ms 8 UPDATE "batched_background_migration_jobs" SET "updated_at" = $1, "started_at" = $2, "status" = $3, "attempts" = $4
WHERE "batched_background_migration_jobs"."id" = $58 0.4 ms 0.1 ms 0.1 ms 8 SELECT SUM("batched_background_migration_jobs"."batch_size")
FROM "batched_background_migration_jobs" WHERE "batched_background_migration_jobs"."batched_background_migration_id" = $1 AND ("batched_background_migration_jobs"."status" IN ($2))16 0.8 ms 0.1 ms 0.0 ms 16 SELECT "batched_background_migration_jobs".*
FROM "batched_background_migration_jobs" WHERE "batched_background_migration_jobs"."id" = $1
LIMIT $28 0.3 ms 0.1 ms 0.0 ms 8 SELECT "namespace_root_storage_statistics"."namespace_id" FROM "namespace_root_storage_statistics" WHERE "namespace_root_storage_statistics"."namespace_id" BETWEEN $1 AND $2
ORDER BY "namespace_root_storage_statistics"."namespace_id" ASC
LIMIT $38 0.3 ms 0.1 ms 0.0 ms 8 SELECT "batched_background_migrations".*
FROM "batched_background_migrations" WHERE "batched_background_migrations"."id" = $1
LIMIT $2Histogram of batch runtimes for BackfillRootStorageStatisticsForkStorageSizes
Batch Runtime Count 0 seconds - 10 seconds 1 10 seconds - 1 minute 0 1 minute - 2 minutes 5 2 minutes - 3 minutes 1 3 minutes - 5 minutes 1 5 minutes + 0 Histogram across all sampled batches of BackfillRootStorageStatisticsForkStorageSizes
Query Runtime Count 0 seconds - 0.1 seconds 14327 0.1 seconds - 0.5 seconds 454 0.5 seconds - 1 second 5 1 second - 2 seconds 4 2 seconds - 5 seconds 0 5 seconds + 1 Other information
Other migrations pending on GitLab.com
Migration Type Total runtime Result DB size change 20230517163300 - QueueBackfillRootStorageStatisticsForkStorageSizes Post deploy 2.2 s +0.00 B 20230522111534 - RescheduleMigrationForLinksFromMetadata Post deploy 4.3 s +0.00 B 20230530114845 - CleanupOrganizationsWithNullPath Post deploy 1.5 s +0.00 B 20230530115830 - RemoveDefaultOnOrganizationPath Post deploy 1.5 s +0.00 B 20230601184234 - DropUnnecessaryAsyncIndexFromEvents Post deploy 2.6 s +0.00 B 20230602063059 - RemoveBroadcastMessagesNamespaceIdColumn Post deploy 1.7 s -16.00 KiB 20230602112904 - ChangeUserPreferencesDefaults Post deploy 1.8 s +0.00 B Clone details
Clone ID Clone Created At Clone Data Timestamp Expected Removal Time database-testing-1998686-10252908-main
2023-06-06T13:41:50Z 2023-06-06T12:13:04Z 2023-06-07 02:17:07 +0000 database-testing-1998686-10252908-ci
2023-06-06T13:41:50Z 2023-06-06T12:41:06Z 2023-06-07 02:17:07 +0000 Database migrations (on the ci database)
Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).
Migration Type Total runtime Result DB size change 20230605095810 - EnsureDefaultOrganization Regular 2.3 s +0.00 B Runtime Histogram for all migrations
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 2 0.1 seconds - 1 second 0 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20230605095810 - EnsureDefaultOrganization
- Type: Regular
- Duration: 2.3 s
- Database size change: +0.00 B
Calls Total Time Max Time Mean Time Rows Query 2 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for EnsureDefaultOrganization
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 2 0.1 seconds - 1 second 0 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Other information
Other migrations pending on GitLab.com
Migration Type Total runtime Result DB size change 20230517163300 - QueueBackfillRootStorageStatisticsForkStorageSizes Post deploy 1.9 s +0.00 B 20230522111534 - RescheduleMigrationForLinksFromMetadata Post deploy 2.1 s +0.00 B 20230530114845 - CleanupOrganizationsWithNullPath Post deploy 1.9 s +0.00 B 20230530115830 - RemoveDefaultOnOrganizationPath Post deploy 2.1 s +0.00 B 20230601184234 - DropUnnecessaryAsyncIndexFromEvents Post deploy 3.1 s +0.00 B 20230602063059 - RemoveBroadcastMessagesNamespaceIdColumn Post deploy 2.0 s -8.00 KiB 20230602112904 - ChangeUserPreferencesDefaults Post deploy 2.4 s +0.00 B Clone details
Clone ID Clone Created At Clone Data Timestamp Expected Removal Time database-testing-1998686-10252908-main
2023-06-06T13:41:50Z 2023-06-06T12:13:04Z 2023-06-07 02:17:07 +0000 database-testing-1998686-10252908-ci
2023-06-06T13:41:50Z 2023-06-06T12:41:06Z 2023-06-07 02:17:07 +0000
Brought to you by gitlab-org/database-team/gitlab-com-database-testing. Epic
- Resolved by Rutger Wessels
added databaseapproved label and removed databasereviewed label
removed review request for @pshutsin
- A deleted user
added database-testing-automation label
- Resolved by Sincheol (David) Kim
requested review from @dskim_gitlab
marked the checklist item I have evaluated the MR acceptance checklist for this MR. as completed
mentioned in issue #414930 (closed)
enabled an automatic merge when the pipeline for 175fa4bb succeeds
@dskim_gitlab, did you forget to run a pipeline before you merged this work? Based on our code review process, if the latest pipeline was created more than 6 hours ago OR finished more than 2 hours ago, you should:
- Ensure the merge request is not in Draft status.
- Start a pipeline (especially important for Community contribution merge requests).
- Set the merge request to auto-merge.
This is a guideline, not a rule. Please consider replying to this comment for transparency.
This message was generated automatically. You're welcome to improve it.
@dskim_gitlab Thanks for noticing! So pressing the 'Run Pipeline' button is the way to go in this case?
@rutgerwessels Yes, that's what I normally do, but I just reran individual failures this time.
Hello @rutgerwessels
The database team is looking for ways to improve the database review process and we would love your help!
If you'd be open to someone on the database team reaching out to you for a chat, or if you'd like to leave some feedback asynchronously, just post a reply to this comment mentioning:
@gitlab-org/database-team
And someone will be by shortly!
Thanks for your help!
This message was generated automatically. You're welcome to improve it.
mentioned in commit 76a67b8a
added workflowstaging-canary label and removed workflowin dev label
added workflowcanary label and removed workflowstaging-canary label
mentioned in issue #411441 (closed)
added workflowstaging label and removed workflowcanary label
added workflowproduction label and removed workflowstaging label
added workflowpost-deploy-db-production label and removed workflowproduction label
added releasedcandidate label
mentioned in merge request kubitus-project/kubitus-installer!2224 (merged)
added releasedpublished label and removed releasedcandidate label
mentioned in merge request !144997 (merged)
mentioned in merge request !145606 (merged)
mentioned in issue omnibus-gitlab#8484 (closed)
mentioned in merge request !155762 (merged)
mentioned in merge request !156380 (closed)
added devopstenant scale grouporganizations sectioninfrastructure platforms labels and removed devopsdata stores grouptenant scale [DEPRECATED] sectioncore platform labels