Early Access Program tracking init
What does this MR do and why?
Creates some base product usage activity tracking mechanism for future GitLab Early Access program initiative.
Issue for more details.
Screenshots or screen recordings
Negative scenario 2
Example check for each existing user that didn't do anything but feature flag is enabled:
Positive scenario
Example check for user that participates in Early access program AND agreed on product usage data collection AND feature flag is enabled:
How to set up and validate locally
Negative scenario 1
Example check for each existing user that didn't do anything:
- As signed in user go to some owned project page.
- On left-sidebar click on
Code > Snippets > New snippet
. - Create snippet with any code you like.
- In rails console check that no tracking event created:
EarlyAccessProgram::TrackingEvent.none?
should returntrue
.
Negative scenario 2
Example check for each existing user that didn't do anything but feature flag is enabled:
- In rails console enable feature flag for your user
Feature.enable(:early_access_program, User.find(<your-id>))
- As signed in user go to some owned project page.
- On left-sidebar click on
Code > Snippets > New snippet
. - Create snippet with any code you like. (optionally you can edit existing one)
- In rails console check that no tracking event created:
EarlyAccessProgram::TrackingEvent.none?
should returntrue
.
Positive scenario
Example check for user that participates in Early access program AND agreed on product usage data collection AND feature flag is enabled:
- In rails console enable feature flag for your user
Feature.enable(:early_access_program, User.find(<your-id>))
- In rails console mark your user as Early access program participant and agreed on product usage data collection by
User.find(<your-id>).user_preference.update!(early_access_program_participant: true, early_access_program_tracking: true)
- As signed in user go to some owned project page.
- On left-sidebar click on
Code > Snippets > New snippet
. - Create snippet with any code you like. (optionally you can edit existing one)
- In rails console check that no tracking event created:
EarlyAccessProgram::TrackingEvent.none?
should returnfalse
. - See list of tracking events by typing
EarlyAccessProgram::TrackingEvent.all
in rails console.
Merge request reports
Activity
assigned to @knockfog-ext
- A deleted user
added Contributor Success databasereview pending feature flag labels
- Resolved by Rostyslav Safonov
2 Warnings This MR changes code in ee/
, but its Changelog commit is missing theEE: true
trailer. Consider adding it to your Changelog commits.37bb37df: The commit subject must contain at least 3 words. For more information, take a look at our Commit message guidelines. Reviewer roulette
Category Reviewer Maintainer backend @subashis
(UTC-6)
@wandering_person
(UTC+7)
database @bwill
(UTC-5)
@pshutsin
(UTC+2)
test for spec/features/*
@subashis
(UTC-6)
Maintainer review is optional for test for spec/features/*
~"Backend Static Code Analysis" Reviewer review is optional for ~"Backend Static Code Analysis" @ddieulivol
(UTC+2)
Please check reviewer's status!
Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
Danger- A deleted user
added Data WarehouseImpact Check label
- Resolved by 🤖 GitLab Bot 🤖
Proper labels assigned to this merge request. Please ignore me.
@knockfog-ext - please see the following guidance and update this merge request.1 Error Please add typebug typefeature, or typemaintenance label to this merge request. Edited by 🤖 GitLab Bot 🤖
added typefeature label
- Resolved by Michael Becker
This has been identified as a contribution from a Temporary Service Provider to GitLab. It is encouraged to seek review from at least one domain expert during the review process.
added Contractor Contribution label
added 600 commits
-
0d282e20...3e41b355 - 599 commits from branch
gitlab-org:master
- 041a69d2 - Early Access Program tracking init
-
0d282e20...3e41b355 - 599 commits from branch
Generated bygitlab_quality-test_tooling
.
Slow tests detected in this merge request. These slow tests might be related to this merge request's changes.Click to expand
Job File Name Duration Expected duration #6593848212 spec/features/dashboard/snippets_spec.rb#L27
Dashboard snippets when the project has snippets behaves like paginated snippets clicking on the link to the second page shows the remaining snippets 62.86 s < 50.13 s #6593848212 spec/features/dashboard/snippets_spec.rb#L90
Dashboard snippets filtering by visibility contains all private snippets of logged user when clicking on private 62.5 s < 50.13 s #6593848212 spec/features/dashboard/snippets_spec.rb#L97
Dashboard snippets filtering by visibility contains all internal snippets of logged user when clicking on internal 62.47 s < 50.13 s #6593848212 spec/features/dashboard/snippets_spec.rb#L104
Dashboard snippets filtering by visibility contains all public snippets of logged user when clicking on public 62.55 s < 50.13 s #6593848108 spec/features/dashboard/snippets_spec.rb#L27
Dashboard snippets when the project has snippets behaves like paginated snippets clicking on the link to the second page shows the remaining snippets 63.41 s < 50.13 s #6593848108 spec/features/dashboard/snippets_spec.rb#L90
Dashboard snippets filtering by visibility contains all private snippets of logged user when clicking on private 62.98 s < 50.13 s #6593848108 spec/features/dashboard/snippets_spec.rb#L97
Dashboard snippets filtering by visibility contains all internal snippets of logged user when clicking on internal 63.31 s < 50.13 s #6593848108 spec/features/dashboard/snippets_spec.rb#L104
Dashboard snippets filtering by visibility contains all public snippets of logged user when clicking on public 63.36 s < 50.13 s #6595417870 spec/features/dashboard/snippets_spec.rb#L27
Dashboard snippets when the project has snippets behaves like paginated snippets clicking on the link to the second page shows the remaining snippets 62.86 s < 50.13 s #6595417870 spec/features/dashboard/snippets_spec.rb#L90
Dashboard snippets filtering by visibility contains all private snippets of logged user when clicking on private 62.51 s < 50.13 s #6595417870 spec/features/dashboard/snippets_spec.rb#L97
Dashboard snippets filtering by visibility contains all internal snippets of logged user when clicking on internal 62.49 s < 50.13 s #6595417870 spec/features/dashboard/snippets_spec.rb#L104
Dashboard snippets filtering by visibility contains all public snippets of logged user when clicking on public 62.66 s < 50.13 s #6595417729 spec/features/dashboard/snippets_spec.rb#L27
Dashboard snippets when the project has snippets behaves like paginated snippets clicking on the link to the second page shows the remaining snippets 63.23 s < 50.13 s #6595417729 spec/features/dashboard/snippets_spec.rb#L90
Dashboard snippets filtering by visibility contains all private snippets of logged user when clicking on private 62.61 s < 50.13 s #6595417729 spec/features/dashboard/snippets_spec.rb#L97
Dashboard snippets filtering by visibility contains all internal snippets of logged user when clicking on internal 62.63 s < 50.13 s #6595417729 spec/features/dashboard/snippets_spec.rb#L104
Dashboard snippets filtering by visibility contains all public snippets of logged user when clicking on public 62.67 s < 50.13 s #6628594300 ee/spec/models/epic_spec.rb#L917
Epic relative positioning there is a parent behaves like a class that supports relative positioning .move_nulls_to_end can move many nulls 47.89 s < 45.4 s #6639869538 ee/spec/models/epic_spec.rb#L917
Epic relative positioning there is a parent behaves like a class that supports relative positioning .move_nulls_to_end can move many nulls 47.97 s < 45.4 s #6667514329 spec/features/users/snippets_spec.rb#L23
Snippets tab on a user profile when the user has snippets pagination behaves like paginated snippets is limited to 20 items per page 62.23 s < 50.13 s #6667514329 spec/features/users/snippets_spec.rb#L32
Snippets tab on a user profile when the user has snippets list content contains only internal and public snippets of a user when a user is logged in 63.09 s < 50.13 s #6667514329 spec/features/users/snippets_spec.rb#L44
Snippets tab on a user profile when the user has snippets list content contains only public snippets of a user when a user is not logged in 62.3 s < 50.13 s #6667514570 spec/features/users/snippets_spec.rb#L23
Snippets tab on a user profile when the user has snippets pagination behaves like paginated snippets is limited to 20 items per page 62.56 s < 50.13 s #6667514570 spec/features/users/snippets_spec.rb#L32
Snippets tab on a user profile when the user has snippets list content contains only internal and public snippets of a user when a user is logged in 63.35 s < 50.13 s #6667514570 spec/features/users/snippets_spec.rb#L44
Snippets tab on a user profile when the user has snippets list content contains only public snippets of a user when a user is not logged in 62.28 s < 50.13 s #6667521287 spec/features/users/snippets_spec.rb#L23
Snippets tab on a user profile when the user has snippets pagination behaves like paginated snippets is limited to 20 items per page 62.47 s < 50.13 s #6667521287 spec/features/users/snippets_spec.rb#L32
Snippets tab on a user profile when the user has snippets list content contains only internal and public snippets of a user when a user is logged in 63.24 s < 50.13 s #6667521287 spec/features/users/snippets_spec.rb#L44
Snippets tab on a user profile when the user has snippets list content contains only public snippets of a user when a user is not logged in 62.65 s < 50.13 s #6683571703 spec/features/users/snippets_spec.rb#L32
Snippets tab on a user profile when the user has snippets list content contains only internal and public snippets of a user when a user is logged in 62.2 s < 50.13 s #6683571703 spec/features/users/snippets_spec.rb#L44
Snippets tab on a user profile when the user has snippets list content contains only public snippets of a user when a user is not logged in 62.34 s < 50.13 s #6683572411 spec/features/users/snippets_spec.rb#L32
Snippets tab on a user profile when the user has snippets list content contains only internal and public snippets of a user when a user is logged in 62.28 s < 50.13 s #6683572411 spec/features/users/snippets_spec.rb#L44
Snippets tab on a user profile when the user has snippets list content contains only public snippets of a user when a user is not logged in 62.16 s < 50.13 s #6683571530 spec/features/users/snippets_spec.rb#L32
Snippets tab on a user profile when the user has snippets list content contains only internal and public snippets of a user when a user is logged in 62.42 s < 50.13 s #6683571530 spec/features/users/snippets_spec.rb#L44
Snippets tab on a user profile when the user has snippets list content contains only public snippets of a user when a user is not logged in 62.45 s < 50.13 s #6683572599 ee/spec/models/epic_spec.rb#L932
Epic relative positioning there is a parent behaves like a class that supports relative positioning .move_nulls_to_end can move many nulls 46.15 s < 45.4 s #6694140186 ee/spec/features/boards/swimlanes/epics_swimlanes_drag_drop_spec.rb#L33
epics swimlanes when no epic is displayed user can drag and drop between columns 50.44 s < 50.13 s #6694132201 ee/spec/features/boards/swimlanes/epics_swimlanes_drag_drop_spec.rb#L33
epics swimlanes when no epic is displayed user can drag and drop between columns 57.51 s < 50.13 s #6694131996 ee/spec/models/epic_spec.rb#L932
Epic relative positioning there is a parent behaves like a class that supports relative positioning .move_nulls_to_end can move many nulls 46.62 s < 45.4 s #6696930491 ee/spec/features/boards/swimlanes/epics_swimlanes_drag_drop_spec.rb#L33
epics swimlanes when no epic is displayed user can drag and drop between columns 73.22 s < 50.13 s #6696873849 ee/spec/features/boards/swimlanes/epics_swimlanes_drag_drop_spec.rb#L33
epics swimlanes when no epic is displayed user can drag and drop between columns 58.65 s < 50.13 s - A deleted user
added rspec:slow test detected label
- Resolved by Rostyslav Safonov
Database migrations (on the main database)
Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).
Migration Type Total runtime Result DB size change 20240404191440 - AddEarlyAccessProgramParticipantToUserPreferences Regular 1.8 s +0.00 B 20240404192955 - CreateEarlyAccessProgramTrackingEvents Regular 2.0 s +64.00 KiB Runtime Histogram for all migrations
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 11 0.1 seconds - 1 second 1 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20240404191440 - AddEarlyAccessProgramParticipantToUserPreferences
- Type: Regular
- Duration: 1.8 s
- Database size change: +0.00 B
Calls Total Time Max Time Mean Time Rows Query 1 18.4 ms 18.4 ms 18.4 ms 0 ALTER TABLE "user_preferences" ADD "early_access_program_participant" boolean DEFAULT FALSE NOT NULL, ADD "early_access_program_tracking" boolean DEFAULT FALSE NOT NULL
2 0.0 ms 0.0 ms 0.0 ms 2 SELECT $1::regtype::oid
2 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for AddEarlyAccessProgramParticipantToUserPreferences
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 5 0.1 seconds - 1 second 0 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20240404192955 - CreateEarlyAccessProgramTrackingEvents
- Type: Regular
- Duration: 2.0 s
- Database size change: +64.00 KiB
Calls Total Time Max Time Mean Time Rows Query 1 77.3 ms 77.3 ms 77.3 ms 0 CREATE TABLE "early_access_program_tracking_events" ("id" bigserial primary key, "user_id" bigint NOT NULL, "event_name" text NOT NULL, "event_label" text, "category" text, "created_at" timestamptz NOT NULL, "updated_at" timestamptz NOT NULL, CONSTRAINT "fk_rails_3e8c32b3dd" FOREIGN KEY ("user_id") REFERENCES "users" ("id") ON DELETE CASCADE, CONSTRAINT check_573cf46c14 CHECK (char_length("event_name") <= 255), CONSTRAINT check_4faf711d09 CHECK (char_length("event_label") <= 255), CONSTRAINT check_e631e07806 CHECK (char_length("category") <= 255))
1 4.6 ms 4.6 ms 4.6 ms 0 CREATE INDEX "index_early_access_program_tracking_events_on_category" ON "early_access_program_tracking_events" ("category")
1 3.3 ms 3.3 ms 3.3 ms 0 CREATE INDEX "index_early_access_program_tracking_events_on_user_id" ON "early_access_program_tracking_events" ("user_id")
1 1.4 ms 1.4 ms 1.4 ms 0 CREATE INDEX "index_early_access_program_tracking_events_on_event_name" ON "early_access_program_tracking_events" ("event_name")
1 0.9 ms 0.9 ms 0.9 ms 0 CREATE INDEX "index_early_access_program_tracking_events_on_event_label" ON "early_access_program_tracking_events" ("event_label")
2 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Column ordering suggestionsOrdering columns efficiently (as in below tables) will help us save more space, further details can be found here.
CREATE TABLE "early_access_program_tracking_events" ("id" bigserial primary key, "user_id" bigint NOT NULL, "event_name" text NOT NULL, "event_label" text, "category" text, "created_at" timestamptz NOT NULL, "updated_at" timestamptz NOT NULL, CONSTRAINT "fk_rails_3e8c32b3dd" FOREIGN KEY ("user_id") REFERENCES "users" ("id") ON DELETE CASCADE, CONSTRAINT check_573cf46c14 CHECK (char_length("event_name") <= 255), CONSTRAINT check_4faf711d09 CHECK (char_length("event_label") <= 255), CONSTRAINT check_e631e07806 CHECK (char_length("category") <= 255))
Column name Data Type id bigserial user_id int8 created_at timestamptz updated_at timestamptz event_name text event_label text category text Histogram for CreateEarlyAccessProgramTrackingEvents
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 6 0.1 seconds - 1 second 1 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Background Migration: BackfillDeploymentApprovalsProjectId
Sampled 31 batches. Estimated Time to complete: 6 hours and 52 minutes
- Interval: 120s
- Total tuple count: 206197
- Max batch size: 0
- Estimated seconds to complete: 24720s
- Estimated number of batches: 206
- Average batch time: 6.43s
- Batch size: 1000
- N. of batches sampled: 31
- N. of failed batches: 0
Time estimation is conservative and based on sampling production data in a test environment. It represents the max time that migration could take. The actual time may differ from this estimation.
_Consider changing max_batch_size and interval if this estimate is unacceptable._
Calls Total Time Max Time Mean Time Rows Query 279 107233.6 ms 1620.2 ms 384.3 ms 27900 UPDATE deployment_approvals
SET project_id = deployments.project_id
FROM deployments
WHERE deployments.id = deployment_approvals.deployment_id AND deployment_approvals.id IN (
SELECT deployment_approvals.id
FROM deployment_approvals
WHERE deployment_approvals.id BETWEEN $1 AND $2 AND deployment_approvals.project_id IS NULL AND deployment_approvals.id >= $3 AND deployment_approvals.id < $4
)31 6917.4 ms 458.5 ms 223.1 ms 3100 UPDATE deployment_approvals
SET project_id = deployments.project_id
FROM deployments
WHERE deployments.id = deployment_approvals.deployment_id AND deployment_approvals.id IN (
SELECT deployment_approvals.id
FROM deployment_approvals
WHERE deployment_approvals.id BETWEEN $1 AND $2 AND deployment_approvals.project_id IS NULL AND deployment_approvals.id >= $3
)310 76.9 ms 9.7 ms 0.2 ms 279 SELECT deployment_approvals.id
FROM deployment_approvals
WHERE deployment_approvals.id BETWEEN $1 AND $2 AND deployment_approvals.project_id IS NULL AND deployment_approvals.id >= $3
ORDER BY deployment_approvals.id ASC
LIMIT $4
OFFSET $562 19.1 ms 7.7 ms 0.3 ms 62 INSERT INTO batched_background_migration_job_transition_logs (batched_background_migration_job_id, created_at, updated_at, previous_status, next_status) VALUES ($1, $2, $3, $4, $5) RETURNING id
31 13.4 ms 3.7 ms 0.4 ms 31 UPDATE batched_background_migration_jobs
SET updated_at = $1, started_at = $2, status = $3, attempts = $4
WHERE batched_background_migration_jobs.id = $562 5.9 ms 2.3 ms 0.1 ms 62 SELECT batched_background_migration_jobs.*
FROM batched_background_migration_jobs
WHERE batched_background_migration_jobs.id = $1
LIMIT $231 5.1 ms 0.4 ms 0.2 ms 31 UPDATE batched_background_migration_jobs
SET updated_at = $1, finished_at = $2, status = $3, metrics = $4
WHERE batched_background_migration_jobs.id = $531 1.5 ms 0.1 ms 0.0 ms 31 SELECT deployment_approvals.id
FROM deployment_approvals
WHERE deployment_approvals.id BETWEEN $1 AND $2 AND deployment_approvals.project_id IS NULL
ORDER BY deployment_approvals.id ASC
LIMIT $331 2.3 ms 0.1 ms 0.1 ms 31 SELECT sum(batched_background_migration_jobs.batch_size)
FROM batched_background_migration_jobs
WHERE batched_background_migration_jobs.batched_background_migration_id = $1 AND batched_background_migration_jobs.status IN ($2)31 1.5 ms 0.1 ms 0.0 ms 31 SELECT batched_background_migrations.*
FROM batched_background_migrations
WHERE batched_background_migrations.id = $1
LIMIT $2Histogram of batch runtimes for BackfillDeploymentApprovalsProjectId
Batch Runtime Count 0 seconds - 10 seconds 30 10 seconds - 1 minute 1 1 minute - 2 minutes 0 2 minutes - 3 minutes 0 3 minutes - 5 minutes 0 5 minutes + 0 Histogram across all sampled batches of BackfillDeploymentApprovalsProjectId
Query Runtime Count 0 seconds - 0.1 seconds 589 0.1 seconds - 0.5 seconds 225 0.5 seconds - 1 second 78 1 second - 2 seconds 7 2 seconds - 5 seconds 0 5 seconds + 0 Background Migration: BackfillArchivedAndTraversalIdsToVulnerabilityReads
Sampled 11 batches. Estimated Time to complete: 1 week, 6 days, 4 hours, and 44 minutes
- Interval: 120s
- Total tuple count: 95020256
- Max batch size: 0
- Estimated seconds to complete: 1140240s
- Estimated number of batches: 9502
- Average batch time: 79.05s
- Batch size: 10000
- N. of batches sampled: 11
- N. of failed batches: 0
Time estimation is conservative and based on sampling production data in a test environment. It represents the max time that migration could take. The actual time may differ from this estimation.
_Consider changing max_batch_size and interval if this estimate is unacceptable._
Calls Total Time Max Time Mean Time Rows Query 1089 600734.5 ms 13295.2 ms 551.6 ms 108900 UPDATE vulnerability_reads
SET traversal_ids = namespaces.traversal_ids, archived = projects.archived
FROM projects
JOIN namespaces ON namespaces.id = projects.namespace_id
WHERE vulnerability_reads.id IN (
SELECT vulnerability_reads.id
FROM vulnerability_reads
WHERE vulnerability_reads.id BETWEEN $1 AND $2 AND vulnerability_reads.id >= $3 AND vulnerability_reads.id < $4
) AND vulnerability_reads.project_id = projects.id11 5198.1 ms 1362.9 ms 472.6 ms 1005 UPDATE vulnerability_reads
SET traversal_ids = namespaces.traversal_ids, archived = projects.archived
FROM projects
JOIN namespaces ON namespaces.id = projects.namespace_id
WHERE vulnerability_reads.id IN (
SELECT vulnerability_reads.id
FROM vulnerability_reads
WHERE vulnerability_reads.id BETWEEN $1 AND $2 AND vulnerability_reads.id >= $3
) AND vulnerability_reads.project_id = projects.id22 20.8 ms 16.8 ms 0.9 ms 22 INSERT INTO batched_background_migration_job_transition_logs (batched_background_migration_job_id, created_at, updated_at, previous_status, next_status) VALUES ($1, $2, $3, $4, $5) RETURNING id
1100 176.0 ms 11.4 ms 0.2 ms 1089 SELECT vulnerability_reads.id
FROM vulnerability_reads
WHERE vulnerability_reads.id BETWEEN $1 AND $2 AND vulnerability_reads.id >= $3
ORDER BY vulnerability_reads.id ASC
LIMIT $4
OFFSET $511 8.0 ms 4.1 ms 0.7 ms 11 UPDATE batched_background_migration_jobs
SET updated_at = $1, started_at = $2, status = $3, attempts = $4
WHERE batched_background_migration_jobs.id = $511 9.8 ms 3.4 ms 0.9 ms 11 UPDATE batched_background_migration_jobs
SET updated_at = $1, finished_at = $2, status = $3, metrics = $4
WHERE batched_background_migration_jobs.id = $522 2.0 ms 0.4 ms 0.1 ms 22 SELECT batched_background_migration_jobs.*
FROM batched_background_migration_jobs
WHERE batched_background_migration_jobs.id = $1
LIMIT $211 0.5 ms 0.1 ms 0.0 ms 11 SELECT vulnerability_reads.id
FROM vulnerability_reads
WHERE vulnerability_reads.id BETWEEN $1 AND $2
ORDER BY vulnerability_reads.id ASC
LIMIT $311 0.5 ms 0.1 ms 0.0 ms 11 SELECT sum(batched_background_migration_jobs.batch_size)
FROM batched_background_migration_jobs
WHERE batched_background_migration_jobs.batched_background_migration_id = $1 AND batched_background_migration_jobs.status IN ($2)11 0.5 ms 0.1 ms 0.0 ms 11 SELECT batched_background_migrations.*
FROM batched_background_migrations
WHERE batched_background_migrations.id = $1
LIMIT $2Histogram of batch runtimes for BackfillArchivedAndTraversalIdsToVulnerabilityReads
Batch Runtime Count 0 seconds - 10 seconds 0 10 seconds - 1 minute 2 1 minute - 2 minutes 9 2 minutes - 3 minutes 0 3 minutes - 5 minutes 0 5 minutes + 0 Histogram across all sampled batches of BackfillArchivedAndTraversalIdsToVulnerabilityReads
Query Runtime Count 0 seconds - 0.1 seconds 1197 0.1 seconds - 0.5 seconds 587 0.5 seconds - 1 second 403 1 second - 2 seconds 94 2 seconds - 5 seconds 8 5 seconds + 10 Other information
Other migrations pending on GitLab.com
Migration Type Total runtime Result DB size change 20240404115853 - AddPrefixedToCatalogResourceVersions Regular 2.2 s +0.00 B 20240408105626 - AddSendBotMessageToPolicies Regular 1.7 s +0.00 B 20240410061546 - AddIndexForMemberApprovalsMemberNamespaceIdStatus Regular 2.7 s +8.00 KiB [note] 20240403020614 - PrepareTmpBackfillIndexForPipelineIdsToVulnerabilityOccurrences Post deploy 3.3 s +0.00 B 20240408135326 - RemoveForeignKeysFromExternalApprovalRulesProtectedBranches Post deploy 2.3 s +0.00 B 20240408135652 - DropExternalApprovalRulesProtectedBranchesTable Post deploy 1.9 s -32.00 KiB 20240409070036 - SyncIndexForPipelinesUniqueIdBigint Post deploy 2.9 s +0.00 B 20240409140739 - RequeueBackfillArchivedAndTraversalIdsToVulnerabilityReads Post deploy 7.6 s +0.00 B 20240410004336 - AddDeploymentApprovalsProjectIdTrigger Post deploy 2.2 s +0.00 B 20240410004337 - QueueBackfillDeploymentApprovalsProjectId Post deploy 2.3 s +0.00 B 20240410070036 - PrepareAsyncIndexesForPipelinesIdBigint Post deploy 14.4 s +0.00 B 20240411052531 - DropIdxMergeRequestsOnTargetProjectIdAndIidOpened Post deploy 3.3 s +0.00 B 20240411070036 - AsyncFkIdBigint4CiPipelinesPCiBuildsCiPipelinesConfigPCiStages Post deploy 18.4 s +16.00 KiB 20240412000002 - PrepareAsyncIndexForBuildsPart5 Post deploy 9.8 s +0.00 B Clone details
Clone ID Clone Created At Clone Data Timestamp Expected Removal Time database-testing-3125776-13557265-main
2024-04-16T15:53:52Z 2024-04-15T10:51:44Z 2024-04-17 04:20:10 +0000 database-testing-3125776-13557265-ci
2024-04-16T15:53:52Z 2024-04-16T12:41:29Z 2024-04-17 04:20:10 +0000 Database migrations (on the ci database)
1 Warnings 20240404192955 - CreateEarlyAccessProgramTrackingEvents had a query that exceeded timing
guidelines. Run time should not exceed 100ms, but it was 532.34ms. Please consider possible options
to improve the query performance.CREATE TABLE "early_access_program_tracking_events"
("id" bigserial primary key, "user_id" bigint NOT NULL, "event_name" text NOT NULL, "event_label"
text, "category" text, "created_at" timestamptz NOT NULL, "updated_at" timestamptz NOT NULL,
CONSTRAINT "fk_rails_3e8c32b3dd" FOREIGN KEY ("user_id") REFERENCES "users" ("id") ON DELETE
CASCADE, CONSTRAINT check_573cf46c14 CHECK (char_length("event_name") <= 255), CONSTRAINT
check_4faf711d09 CHECK (char_length("event_label") <= 255), CONSTRAINT check_e631e07806 CHECK
(char_length("category") <= 255))Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).
Migration Type Total runtime Result DB size change 20240404191440 - AddEarlyAccessProgramParticipantToUserPreferences Regular 2.5 s +0.00 B 20240404192955 - CreateEarlyAccessProgramTrackingEvents Regular 3.2 s +64.00 KiB Runtime Histogram for all migrations
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 12 0.1 seconds - 1 second 2 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20240404191440 - AddEarlyAccessProgramParticipantToUserPreferences
- Type: Regular
- Duration: 2.5 s
- Database size change: +0.00 B
Calls Total Time Max Time Mean Time Rows Query 1 52.4 ms 52.4 ms 52.4 ms 0 ALTER TABLE "user_preferences" ADD "early_access_program_participant" boolean DEFAULT FALSE NOT NULL, ADD "early_access_program_tracking" boolean DEFAULT FALSE NOT NULL
2 0.0 ms 0.0 ms 0.0 ms 2 SELECT $1::regtype::oid
2 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for AddEarlyAccessProgramParticipantToUserPreferences
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 5 0.1 seconds - 1 second 0 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20240404192955 - CreateEarlyAccessProgramTrackingEvents- Type: Regular
- Duration: 3.2 s
- Database size change: +64.00 KiB
Calls Total Time Max Time Mean Time Rows Query 1 532.3 ms 532.3 ms 532.3 ms 0 CREATE TABLE "early_access_program_tracking_events" ("id" bigserial primary key, "user_id" bigint NOT NULL, "event_name" text NOT NULL, "event_label" text, "category" text, "created_at" timestamptz NOT NULL, "updated_at" timestamptz NOT NULL, CONSTRAINT "fk_rails_3e8c32b3dd" FOREIGN KEY ("user_id") REFERENCES "users" ("id") ON DELETE CASCADE, CONSTRAINT check_573cf46c14 CHECK (char_length("event_name") <= 255), CONSTRAINT check_4faf711d09 CHECK (char_length("event_label") <= 255), CONSTRAINT check_e631e07806 CHECK (char_length("category") <= 255))
1 27.2 ms 27.2 ms 27.2 ms 0 CREATE INDEX "index_early_access_program_tracking_events_on_category" ON "early_access_program_tracking_events" ("category")
1 24.1 ms 24.1 ms 24.1 ms 0 CREATE TRIGGER gitlab_schema_write_trigger_for_early_access_program_tracking_events BEFORE INSERT OR
UPDATE OR DELETE OR TRUNCATE ON early_access_program_tracking_events FOR EACH STATEMENT EXECUTE FUNCTION gitlab_schema_prevent_write()1 4.1 ms 4.1 ms 4.1 ms 0 CREATE INDEX "index_early_access_program_tracking_events_on_event_name" ON "early_access_program_tracking_events" ("event_name")
1 1.2 ms 1.2 ms 1.2 ms 0 CREATE INDEX "index_early_access_program_tracking_events_on_event_label" ON "early_access_program_tracking_events" ("event_label")
1 1.0 ms 1.0 ms 1.0 ms 0 CREATE INDEX "index_early_access_program_tracking_events_on_user_id" ON "early_access_program_tracking_events" ("user_id")
1 0.1 ms 0.1 ms 0.1 ms 1 SELECT COUNT(*) from information_schema.triggers
WHERE event_object_table = $1 AND trigger_name = $22 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Column ordering suggestionsOrdering columns efficiently (as in below tables) will help us save more space, further details can be found here.
CREATE TABLE "early_access_program_tracking_events" ("id" bigserial primary key, "user_id" bigint NOT NULL, "event_name" text NOT NULL, "event_label" text, "category" text, "created_at" timestamptz NOT NULL, "updated_at" timestamptz NOT NULL, CONSTRAINT "fk_rails_3e8c32b3dd" FOREIGN KEY ("user_id") REFERENCES "users" ("id") ON DELETE CASCADE, CONSTRAINT check_573cf46c14 CHECK (char_length("event_name") <= 255), CONSTRAINT check_4faf711d09 CHECK (char_length("event_label") <= 255), CONSTRAINT check_e631e07806 CHECK (char_length("category") <= 255))
Column name Data Type id bigserial user_id int8 created_at timestamptz updated_at timestamptz event_name text event_label text category text Histogram for CreateEarlyAccessProgramTrackingEvents
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 7 0.1 seconds - 1 second 2 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Other information
Other migrations pending on GitLab.com
Migration Type Total runtime Result DB size change 20240404115853 - AddPrefixedToCatalogResourceVersions Regular 2.6 s +0.00 B 20240408135326 - RemoveForeignKeysFromExternalApprovalRulesProtectedBranches Post deploy 2.6 s +0.00 B 20240408135652 - DropExternalApprovalRulesProtectedBranchesTable Post deploy 2.4 s -32.00 KiB 20240409070036 - SyncIndexForPipelinesUniqueIdBigint Post deploy 3.1 s +0.00 B 20240410070036 - PrepareAsyncIndexesForPipelinesIdBigint Post deploy 13.5 s +0.00 B 20240411070036 - AsyncFkIdBigint4CiPipelinesPCiBuildsCiPipelinesConfigPCiStages Post deploy 17.6 s +24.00 KiB 20240412000002 - PrepareAsyncIndexForBuildsPart5 Post deploy 9.8 s +0.00 B Clone details
Clone ID Clone Created At Clone Data Timestamp Expected Removal Time database-testing-3125776-13557265-main
2024-04-16T15:53:52Z 2024-04-15T10:51:44Z 2024-04-17 04:20:10 +0000 database-testing-3125776-13557265-ci
2024-04-16T15:53:52Z 2024-04-16T12:41:29Z 2024-04-17 04:20:10 +0000
Brought to you by gitlab-org/database-team/gitlab-com-database-testing. Epic
- A deleted user
added database-testing-automation label
added 456 commits
-
a95be5d0...b0cee131 - 453 commits from branch
gitlab-org:master
- 1f2aef3e - Early Access Program tracking init
- da2c5e1c - Fix NoMethodError in DB docs specs
- b652c83f - Adjust query limit to allow Early Access program tracking
Toggle commit list-
a95be5d0...b0cee131 - 453 commits from branch
requested review from @ddieulivol
requested review from @dgruzd
- Resolved by Pavel Shutsin
Could you take a
at this one?I don't know why but few specs around DB schema fails. Don't have a clue why. I generated schema by dropping+creating+migrating test DB.
Edited by Rostyslav Safonov
- Resolved by Rostyslav Safonov
I think CONSTRAINT CHECK will slow the table insert, if we can do the check in the script instead of passing it to the database to validate, it will help with the query.
requested review from @wandering_person and removed review request for @ddieulivol
added pipeline:mr-approved label
- Resolved by Rostyslav Safonov
@ddieulivol
, thanks for approving this merge request.This is the first time the merge request has been approved. To ensure we don't only run predictive pipelines, and we don't break
master
, please start a new pipeline before merging.Please wait for the pipeline to start before resolving this discussion and set auto-merge for the new pipeline. See merging a merge request for more details.
- Resolved by Rostyslav Safonov
- Resolved by Rostyslav Safonov
- Resolved by Rostyslav Safonov
- Resolved by Rostyslav Safonov
- Resolved by Rostyslav Safonov
removed review request for @dgruzd
changed milestone to %17.0
- Resolved by Rostyslav Safonov
- Resolved by Michael Becker
added 2 commits
reset approvals from @ddieulivol by pushing to the branch
removed pipeline:mr-approved label
mentioned in incident gitlab-org/quality/engineering-productivity/approved-mr-pipeline-incidents#88 (closed)
E2E Test Result Summary
allure-report-publisher
generated test report!e2e-test-on-gdk:
test report for 2315c1d2expand test summary
+------------------------------------------------------------------+ | suites summary | +-------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +-------------+--------+--------+---------+-------+-------+--------+ | Govern | 66 | 0 | 0 | 0 | 66 | ✅ | | Plan | 51 | 0 | 2 | 0 | 53 | ✅ | | Create | 96 | 0 | 9 | 0 | 105 | ✅ | | Data Stores | 31 | 0 | 0 | 0 | 31 | ✅ | | Verify | 35 | 0 | 1 | 0 | 36 | ✅ | | Package | 24 | 0 | 6 | 0 | 30 | ✅ | | Monitor | 7 | 0 | 0 | 0 | 7 | ✅ | | Release | 5 | 0 | 0 | 0 | 5 | ✅ | | Analytics | 2 | 0 | 0 | 0 | 2 | ✅ | | Manage | 0 | 0 | 1 | 0 | 1 | ➖ | +-------------+--------+--------+---------+-------+-------+--------+ | Total | 317 | 0 | 19 | 0 | 336 | ✅ | +-------------+--------+--------+---------+-------+-------+--------+
e2e-package-and-test:
test report for 57897625expand test summary
+---------------------------------------------------------------------+ | suites summary | +----------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +----------------+--------+--------+---------+-------+-------+--------+ | Monitor | 36 | 0 | 13 | 0 | 49 | ✅ | | Plan | 285 | 0 | 23 | 0 | 308 | ✅ | | Verify | 168 | 0 | 30 | 0 | 198 | ✅ | | Govern | 300 | 0 | 13 | 9 | 313 | ✅ | | Manage | 38 | 0 | 12 | 1 | 50 | ✅ | | Create | 600 | 0 | 81 | 5 | 681 | ✅ | | Secure | 9 | 0 | 3 | 0 | 12 | ✅ | | Systems | 10 | 0 | 0 | 0 | 10 | ✅ | | Package | 211 | 0 | 44 | 7 | 255 | ✅ | | GitLab Metrics | 2 | 0 | 1 | 0 | 3 | ✅ | | ModelOps | 0 | 0 | 3 | 0 | 3 | ➖ | | Fulfillment | 8 | 0 | 75 | 1 | 83 | ✅ | | Analytics | 7 | 0 | 0 | 0 | 7 | ✅ | | Data Stores | 137 | 0 | 28 | 0 | 165 | ✅ | | Configure | 1 | 0 | 9 | 0 | 10 | ✅ | | Release | 17 | 0 | 3 | 0 | 20 | ✅ | | Growth | 0 | 0 | 6 | 0 | 6 | ➖ | | Ai-powered | 0 | 0 | 3 | 0 | 3 | ➖ | +----------------+--------+--------+---------+-------+-------+--------+ | Total | 1829 | 0 | 347 | 23 | 2176 | ✅ | +----------------+--------+--------+---------+-------+-------+--------+
added 518 commits
-
7c69b8cc...64a4fefc - 513 commits from branch
gitlab-org:master
- d8e31df4 - Early Access Program tracking init
- f1992b95 - Fix NoMethodError in DB docs specs
- e021ea33 - Adjust query limit to allow Early Access program tracking
- fdf0b467 - Remove useless comments; Change milestone
- 0b6e5c40 - Regenerated structure.sql
Toggle commit list-
7c69b8cc...64a4fefc - 513 commits from branch
requested review from @dgruzd
added databasereviewed workflowin review labels and removed databasereview pending label
added pipeline:mr-approved label
added 183 commits
-
0b6e5c40...9848f4e8 - 182 commits from branch
gitlab-org:master
- f6475965 - Resolve conflict for `lib/gitlab/internal_events.rb`
-
0b6e5c40...9848f4e8 - 182 commits from branch
reset approvals from @dgruzd by pushing to the branch
added databaseapproved label and removed databasereviewed label
removed review request for @pshutsin
added 386 commits
-
f6475965...ab7e297c - 380 commits from branch
gitlab-org:master
- 40e3609c - Early Access Program tracking init
- 607e78ed - Fix NoMethodError in DB docs specs
- a9828019 - Adjust query limit to allow Early Access program tracking
- db5ab79c - Remove useless comments; Change milestone
- 37bb37df - Regenerated structure.sql
- 4f63c123 - Fix Gitlab::InternalEvents specs memoization
Toggle commit list-
f6475965...ab7e297c - 380 commits from branch
reset approvals from @pshutsin and @wandering_person by pushing to the branch
- Resolved by Michael Becker
- Resolved by Michael Becker
reset approvals from @pshutsin by pushing to the branch
removed pipeline:mr-approved label
requested review from @wandering_person
added pipeline:mr-approved label
added Data WarehouseNot Impacted label and removed Data WarehouseImpact Check label
added 509 commits
-
2f584223...0037e571 - 507 commits from branch
gitlab-org:master
- b241ffe0 - Merge branch 'master' of gitlab.com:gitlab-community/gitlab into...
- 57897625 - Adjust queries threshold in epics_swimlanes_drag_drop_spec.rb
-
2f584223...0037e571 - 507 commits from branch
- Resolved by Pavel Shutsin
@pshutsin it happened again
MWPS failed because one more (new) related spec with threshold failed. Plz, schedule new one.
added 241 commits
-
57897625...406cc7f7 - 239 commits from branch
gitlab-org:master
- 2c8d22a5 - Merge branch 'master' of gitlab.com:gitlab-community/gitlab into...
- 2315c1d2 - Fix structure.sql TZ
-
57897625...406cc7f7 - 239 commits from branch
removed pipeline:mr-approved label
added pipeline:mr-approved label
Hello @knockfog-ext
The database team is looking for ways to improve the database review process and we would love your help!
If you'd be open to someone on the database team reaching out to you for a chat, or if you'd like to leave some feedback asynchronously, just post a reply to this comment mentioning:
@gitlab-org/database-team
And someone will be by shortly!
Thanks for your help!
This message was generated automatically. You're welcome to improve it.
mentioned in incident gitlab-org/quality/engineering-productivity/approved-mr-pipeline-incidents#107 (closed)
mentioned in commit 3fc001c2
added workflowstaging-canary label and removed workflowin review label
added workflowcanary label and removed workflowstaging-canary label
added workflowstaging label and removed workflowcanary label
added workflowproduction label and removed workflowstaging label
added workflowpost-deploy-db-production label and removed workflowproduction label
added releasedcandidate label
mentioned in merge request kubitus-project/kubitus-installer!3058 (merged)
added releasedpublished label and removed releasedcandidate label
added pipelinetier-3 label
mentioned in issue gitlab-org/quality/triage-reports#20642 (closed)
mentioned in issue gitlab-org/quality/triage-reports#21003 (closed)
mentioned in issue gitlab-org/quality/triage-reports#21572 (closed)
mentioned in issue gitlab-org/quality/triage-reports#22062