Skip to content
Snippets Groups Projects

Early Access Program tracking init

All threads resolved!

What does this MR do and why?

Creates some base product usage activity tracking mechanism for future GitLab Early Access program initiative.

Issue for more details.

Screenshots or screen recordings

Negative scenario 1

Example check for each existing user that didn't do anything:

Screenshot_2024-04-10_at_17.40.00

Negative scenario 2

Example check for each existing user that didn't do anything but feature flag is enabled:

Screenshot_2024-04-10_at_17.42.20

Positive scenario

Example check for user that participates in Early access program AND agreed on product usage data collection AND feature flag is enabled:

Screenshot_2024-04-10_at_17.43.32

How to set up and validate locally

Negative scenario 1

Example check for each existing user that didn't do anything:

  1. As signed in user go to some owned project page.
  2. On left-sidebar click on Code > Snippets > New snippet.
  3. Create snippet with any code you like.
  4. In rails console check that no tracking event created: EarlyAccessProgram::TrackingEvent.none? should return true.
Negative scenario 2

Example check for each existing user that didn't do anything but feature flag is enabled:

  1. In rails console enable feature flag for your user Feature.enable(:early_access_program, User.find(<your-id>))
  2. As signed in user go to some owned project page.
  3. On left-sidebar click on Code > Snippets > New snippet.
  4. Create snippet with any code you like. (optionally you can edit existing one)
  5. In rails console check that no tracking event created: EarlyAccessProgram::TrackingEvent.none? should return true.
Positive scenario

Example check for user that participates in Early access program AND agreed on product usage data collection AND feature flag is enabled:

  1. In rails console enable feature flag for your user Feature.enable(:early_access_program, User.find(<your-id>))
  2. In rails console mark your user as Early access program participant and agreed on product usage data collection by User.find(<your-id>).user_preference.update!(early_access_program_participant: true, early_access_program_tracking: true)
  3. As signed in user go to some owned project page.
  4. On left-sidebar click on Code > Snippets > New snippet.
  5. Create snippet with any code you like. (optionally you can edit existing one)
  6. In rails console check that no tracking event created: EarlyAccessProgram::TrackingEvent.none? should return false.
  7. See list of tracking events by typing EarlyAccessProgram::TrackingEvent.all in rails console.
Edited by Rostyslav Safonov

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • 2 Warnings
    :warning: This MR changes code in ee/, but its Changelog commit is missing the EE: true trailer. Consider adding it to your Changelog commits.
    :warning: 37bb37df: The commit subject must contain at least 3 words. For more information, take a look at our Commit message guidelines.

    Reviewer roulette

    Category Reviewer Maintainer
    backend @subashis profile link current availability (UTC-6) @wandering_person profile link current availability (UTC+7)
    database @bwill profile link current availability (UTC-5) @pshutsin profile link current availability (UTC+2)
    test for spec/features/* @subashis profile link current availability (UTC-6) Maintainer review is optional for test for spec/features/*
    ~"Backend Static Code Analysis" Reviewer review is optional for ~"Backend Static Code Analysis" @ddieulivol profile link current availability (UTC+2)

    Please check reviewer's status!

    • available Reviewer is available!
    • unavailable Reviewer is unavailable!

    Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

    If needed, you can retry the :repeat: danger-review job that generated this comment.

    Generated by :no_entry_sign: Danger

  • Rostyslav Safonov changed the description

    changed the description

  • added 1 commit

    • 0d282e20 - Early Access Program tracking init

    Compare with previous version

  • Rostyslav Safonov changed the description

    changed the description

  • Rostyslav Safonov added 600 commits

    added 600 commits

    Compare with previous version

  • Rostyslav Safonov changed the description

    changed the description

  • Rostyslav Safonov marked this merge request as ready

    marked this merge request as ready

  • :tools: Generated by gitlab_quality-test_tooling.


    :snail: Slow tests detected in this merge request. These slow tests might be related to this merge request's changes.

    Click to expand
    Job File Name Duration Expected duration
    #6593848212 spec/features/dashboard/snippets_spec.rb#L27 Dashboard snippets when the project has snippets behaves like paginated snippets clicking on the link to the second page shows the remaining snippets 62.86 s < 50.13 s
    #6593848212 spec/features/dashboard/snippets_spec.rb#L90 Dashboard snippets filtering by visibility contains all private snippets of logged user when clicking on private 62.5 s < 50.13 s
    #6593848212 spec/features/dashboard/snippets_spec.rb#L97 Dashboard snippets filtering by visibility contains all internal snippets of logged user when clicking on internal 62.47 s < 50.13 s
    #6593848212 spec/features/dashboard/snippets_spec.rb#L104 Dashboard snippets filtering by visibility contains all public snippets of logged user when clicking on public 62.55 s < 50.13 s
    #6593848108 spec/features/dashboard/snippets_spec.rb#L27 Dashboard snippets when the project has snippets behaves like paginated snippets clicking on the link to the second page shows the remaining snippets 63.41 s < 50.13 s
    #6593848108 spec/features/dashboard/snippets_spec.rb#L90 Dashboard snippets filtering by visibility contains all private snippets of logged user when clicking on private 62.98 s < 50.13 s
    #6593848108 spec/features/dashboard/snippets_spec.rb#L97 Dashboard snippets filtering by visibility contains all internal snippets of logged user when clicking on internal 63.31 s < 50.13 s
    #6593848108 spec/features/dashboard/snippets_spec.rb#L104 Dashboard snippets filtering by visibility contains all public snippets of logged user when clicking on public 63.36 s < 50.13 s
    #6595417870 spec/features/dashboard/snippets_spec.rb#L27 Dashboard snippets when the project has snippets behaves like paginated snippets clicking on the link to the second page shows the remaining snippets 62.86 s < 50.13 s
    #6595417870 spec/features/dashboard/snippets_spec.rb#L90 Dashboard snippets filtering by visibility contains all private snippets of logged user when clicking on private 62.51 s < 50.13 s
    #6595417870 spec/features/dashboard/snippets_spec.rb#L97 Dashboard snippets filtering by visibility contains all internal snippets of logged user when clicking on internal 62.49 s < 50.13 s
    #6595417870 spec/features/dashboard/snippets_spec.rb#L104 Dashboard snippets filtering by visibility contains all public snippets of logged user when clicking on public 62.66 s < 50.13 s
    #6595417729 spec/features/dashboard/snippets_spec.rb#L27 Dashboard snippets when the project has snippets behaves like paginated snippets clicking on the link to the second page shows the remaining snippets 63.23 s < 50.13 s
    #6595417729 spec/features/dashboard/snippets_spec.rb#L90 Dashboard snippets filtering by visibility contains all private snippets of logged user when clicking on private 62.61 s < 50.13 s
    #6595417729 spec/features/dashboard/snippets_spec.rb#L97 Dashboard snippets filtering by visibility contains all internal snippets of logged user when clicking on internal 62.63 s < 50.13 s
    #6595417729 spec/features/dashboard/snippets_spec.rb#L104 Dashboard snippets filtering by visibility contains all public snippets of logged user when clicking on public 62.67 s < 50.13 s
    #6628594300 ee/spec/models/epic_spec.rb#L917 Epic relative positioning there is a parent behaves like a class that supports relative positioning .move_nulls_to_end can move many nulls 47.89 s < 45.4 s
    #6639869538 ee/spec/models/epic_spec.rb#L917 Epic relative positioning there is a parent behaves like a class that supports relative positioning .move_nulls_to_end can move many nulls 47.97 s < 45.4 s
    #6667514329 spec/features/users/snippets_spec.rb#L23 Snippets tab on a user profile when the user has snippets pagination behaves like paginated snippets is limited to 20 items per page 62.23 s < 50.13 s
    #6667514329 spec/features/users/snippets_spec.rb#L32 Snippets tab on a user profile when the user has snippets list content contains only internal and public snippets of a user when a user is logged in 63.09 s < 50.13 s
    #6667514329 spec/features/users/snippets_spec.rb#L44 Snippets tab on a user profile when the user has snippets list content contains only public snippets of a user when a user is not logged in 62.3 s < 50.13 s
    #6667514570 spec/features/users/snippets_spec.rb#L23 Snippets tab on a user profile when the user has snippets pagination behaves like paginated snippets is limited to 20 items per page 62.56 s < 50.13 s
    #6667514570 spec/features/users/snippets_spec.rb#L32 Snippets tab on a user profile when the user has snippets list content contains only internal and public snippets of a user when a user is logged in 63.35 s < 50.13 s
    #6667514570 spec/features/users/snippets_spec.rb#L44 Snippets tab on a user profile when the user has snippets list content contains only public snippets of a user when a user is not logged in 62.28 s < 50.13 s
    #6667521287 spec/features/users/snippets_spec.rb#L23 Snippets tab on a user profile when the user has snippets pagination behaves like paginated snippets is limited to 20 items per page 62.47 s < 50.13 s
    #6667521287 spec/features/users/snippets_spec.rb#L32 Snippets tab on a user profile when the user has snippets list content contains only internal and public snippets of a user when a user is logged in 63.24 s < 50.13 s
    #6667521287 spec/features/users/snippets_spec.rb#L44 Snippets tab on a user profile when the user has snippets list content contains only public snippets of a user when a user is not logged in 62.65 s < 50.13 s
    #6683571703 spec/features/users/snippets_spec.rb#L32 Snippets tab on a user profile when the user has snippets list content contains only internal and public snippets of a user when a user is logged in 62.2 s < 50.13 s
    #6683571703 spec/features/users/snippets_spec.rb#L44 Snippets tab on a user profile when the user has snippets list content contains only public snippets of a user when a user is not logged in 62.34 s < 50.13 s
    #6683572411 spec/features/users/snippets_spec.rb#L32 Snippets tab on a user profile when the user has snippets list content contains only internal and public snippets of a user when a user is logged in 62.28 s < 50.13 s
    #6683572411 spec/features/users/snippets_spec.rb#L44 Snippets tab on a user profile when the user has snippets list content contains only public snippets of a user when a user is not logged in 62.16 s < 50.13 s
    #6683571530 spec/features/users/snippets_spec.rb#L32 Snippets tab on a user profile when the user has snippets list content contains only internal and public snippets of a user when a user is logged in 62.42 s < 50.13 s
    #6683571530 spec/features/users/snippets_spec.rb#L44 Snippets tab on a user profile when the user has snippets list content contains only public snippets of a user when a user is not logged in 62.45 s < 50.13 s
    #6683572599 ee/spec/models/epic_spec.rb#L932 Epic relative positioning there is a parent behaves like a class that supports relative positioning .move_nulls_to_end can move many nulls 46.15 s < 45.4 s
    #6694140186 ee/spec/features/boards/swimlanes/epics_swimlanes_drag_drop_spec.rb#L33 epics swimlanes when no epic is displayed user can drag and drop between columns 50.44 s < 50.13 s
    #6694132201 ee/spec/features/boards/swimlanes/epics_swimlanes_drag_drop_spec.rb#L33 epics swimlanes when no epic is displayed user can drag and drop between columns 57.51 s < 50.13 s
    #6694131996 ee/spec/models/epic_spec.rb#L932 Epic relative positioning there is a parent behaves like a class that supports relative positioning .move_nulls_to_end can move many nulls 46.62 s < 45.4 s
    #6696930491 ee/spec/features/boards/swimlanes/epics_swimlanes_drag_drop_spec.rb#L33 epics swimlanes when no epic is displayed user can drag and drop between columns 73.22 s < 50.13 s
    #6696873849 ee/spec/features/boards/swimlanes/epics_swimlanes_drag_drop_spec.rb#L33 epics swimlanes when no epic is displayed user can drag and drop between columns 58.65 s < 50.13 s
  • A deleted user added rspec:slow test detected label
  • This is part of the beta program issue.

  • added 1 commit

    • b2cd07a4 - Early Access Program tracking init

    Compare with previous version

  • Ghost User
  • added 1 commit

    • e91068f9 - Early Access Program tracking init

    Compare with previous version

  • Database migrations (on the main database)

    Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).

    Migration Type Total runtime Result DB size change
    20240404191440 - AddEarlyAccessProgramParticipantToUserPreferences Regular 1.8 s :white_check_mark: +0.00 B
    20240404192955 - CreateEarlyAccessProgramTrackingEvents Regular 2.0 s :white_check_mark: +64.00 KiB
    Runtime Histogram for all migrations
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 11
    0.1 seconds - 1 second 1
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20240404191440 - AddEarlyAccessProgramParticipantToUserPreferences

    • Type: Regular
    • Duration: 1.8 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    1 18.4 ms 18.4 ms 18.4 ms 0
    ALTER TABLE "user_preferences" ADD "early_access_program_participant" boolean DEFAULT FALSE NOT NULL, ADD "early_access_program_tracking" boolean DEFAULT FALSE NOT NULL
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT $1::regtype::oid
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT pg_backend_pid()
    Histogram for AddEarlyAccessProgramParticipantToUserPreferences
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 5
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20240404192955 - CreateEarlyAccessProgramTrackingEvents

    • Type: Regular
    • Duration: 2.0 s
    • Database size change: +64.00 KiB
    Calls Total Time Max Time Mean Time Rows Query
    1 77.3 ms 77.3 ms 77.3 ms 0
    CREATE TABLE "early_access_program_tracking_events" ("id" bigserial primary key, "user_id" bigint NOT NULL, "event_name" text NOT NULL, "event_label" text, "category" text, "created_at" timestamptz NOT NULL, "updated_at" timestamptz NOT NULL, CONSTRAINT "fk_rails_3e8c32b3dd" FOREIGN KEY ("user_id") REFERENCES "users" ("id") ON DELETE CASCADE, CONSTRAINT check_573cf46c14 CHECK (char_length("event_name") <= 255), CONSTRAINT check_4faf711d09 CHECK (char_length("event_label") <= 255), CONSTRAINT check_e631e07806 CHECK (char_length("category") <= 255))
    1 4.6 ms 4.6 ms 4.6 ms 0
    CREATE INDEX "index_early_access_program_tracking_events_on_category" ON "early_access_program_tracking_events" ("category")
    1 3.3 ms 3.3 ms 3.3 ms 0
    CREATE INDEX "index_early_access_program_tracking_events_on_user_id" ON "early_access_program_tracking_events" ("user_id")
    1 1.4 ms 1.4 ms 1.4 ms 0
    CREATE INDEX "index_early_access_program_tracking_events_on_event_name" ON "early_access_program_tracking_events" ("event_name")
    1 0.9 ms 0.9 ms 0.9 ms 0
    CREATE INDEX "index_early_access_program_tracking_events_on_event_label" ON "early_access_program_tracking_events" ("event_label")
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT pg_backend_pid()
    :warning: Column ordering suggestions

    Ordering columns efficiently (as in below tables) will help us save more space, further details can be found here.

     CREATE TABLE "early_access_program_tracking_events" ("id" bigserial primary key, "user_id" bigint NOT NULL, "event_name" text NOT NULL, "event_label" text, "category" text, "created_at" timestamptz NOT NULL, "updated_at" timestamptz NOT NULL, CONSTRAINT "fk_rails_3e8c32b3dd" FOREIGN KEY ("user_id") REFERENCES "users" ("id") ON DELETE CASCADE, CONSTRAINT check_573cf46c14 CHECK (char_length("event_name") <= 255), CONSTRAINT check_4faf711d09 CHECK (char_length("event_label") <= 255), CONSTRAINT check_e631e07806 CHECK (char_length("category") <= 255)) 
    Column name Data Type
    id bigserial
    user_id int8
    created_at timestamptz
    updated_at timestamptz
    event_name text
    event_label text
    category text
    Histogram for CreateEarlyAccessProgramTrackingEvents
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 6
    0.1 seconds - 1 second 1
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Background Migration: BackfillDeploymentApprovalsProjectId

    Sampled 31 batches. Estimated Time to complete: 6 hours and 52 minutes
    • Interval: 120s
    • Total tuple count: 206197
    • Max batch size: 0
    • Estimated seconds to complete: 24720s
    • Estimated number of batches: 206
    • Average batch time: 6.43s
    • Batch size: 1000
    • N. of batches sampled: 31
    • N. of failed batches: 0

    Time estimation is conservative and based on sampling production data in a test environment. It represents the max time that migration could take. The actual time may differ from this estimation.

    _Consider changing max_batch_size and interval if this estimate is unacceptable._

    Calls Total Time Max Time Mean Time Rows Query
    279 107233.6 ms 1620.2 ms 384.3 ms 27900
    UPDATE deployment_approvals
    SET project_id = deployments.project_id
    FROM deployments
    WHERE deployments.id = deployment_approvals.deployment_id AND deployment_approvals.id IN (
    SELECT deployment_approvals.id
    FROM deployment_approvals
    WHERE deployment_approvals.id BETWEEN $1 AND $2 AND deployment_approvals.project_id IS NULL AND deployment_approvals.id >= $3 AND deployment_approvals.id < $4
    )
    31 6917.4 ms 458.5 ms 223.1 ms 3100
    UPDATE deployment_approvals
    SET project_id = deployments.project_id
    FROM deployments
    WHERE deployments.id = deployment_approvals.deployment_id AND deployment_approvals.id IN (
    SELECT deployment_approvals.id
    FROM deployment_approvals
    WHERE deployment_approvals.id BETWEEN $1 AND $2 AND deployment_approvals.project_id IS NULL AND deployment_approvals.id >= $3
    )
    310 76.9 ms 9.7 ms 0.2 ms 279
    SELECT deployment_approvals.id
    FROM deployment_approvals
    WHERE deployment_approvals.id BETWEEN $1 AND $2 AND deployment_approvals.project_id IS NULL AND deployment_approvals.id >= $3
    ORDER BY deployment_approvals.id ASC
    LIMIT $4
    OFFSET $5
    62 19.1 ms 7.7 ms 0.3 ms 62
    INSERT INTO batched_background_migration_job_transition_logs (batched_background_migration_job_id, created_at, updated_at, previous_status, next_status) VALUES ($1, $2, $3, $4, $5) RETURNING id
    31 13.4 ms 3.7 ms 0.4 ms 31
    UPDATE batched_background_migration_jobs
    SET updated_at = $1, started_at = $2, status = $3, attempts = $4
    WHERE batched_background_migration_jobs.id = $5
    62 5.9 ms 2.3 ms 0.1 ms 62
    SELECT batched_background_migration_jobs.*
    FROM batched_background_migration_jobs
    WHERE batched_background_migration_jobs.id = $1
    LIMIT $2
    31 5.1 ms 0.4 ms 0.2 ms 31
    UPDATE batched_background_migration_jobs
    SET updated_at = $1, finished_at = $2, status = $3, metrics = $4
    WHERE batched_background_migration_jobs.id = $5
    31 1.5 ms 0.1 ms 0.0 ms 31
    SELECT deployment_approvals.id
    FROM deployment_approvals
    WHERE deployment_approvals.id BETWEEN $1 AND $2 AND deployment_approvals.project_id IS NULL
    ORDER BY deployment_approvals.id ASC
    LIMIT $3
    31 2.3 ms 0.1 ms 0.1 ms 31
    SELECT sum(batched_background_migration_jobs.batch_size)
    FROM batched_background_migration_jobs
    WHERE batched_background_migration_jobs.batched_background_migration_id = $1 AND batched_background_migration_jobs.status IN ($2)
    31 1.5 ms 0.1 ms 0.0 ms 31
    SELECT batched_background_migrations.*
    FROM batched_background_migrations
    WHERE batched_background_migrations.id = $1
    LIMIT $2
    Histogram of batch runtimes for BackfillDeploymentApprovalsProjectId
    Batch Runtime Count
    0 seconds - 10 seconds 30
    10 seconds - 1 minute 1
    1 minute - 2 minutes 0
    2 minutes - 3 minutes 0
    3 minutes - 5 minutes 0
    5 minutes + 0
    Histogram across all sampled batches of BackfillDeploymentApprovalsProjectId
    Query Runtime Count
    0 seconds - 0.1 seconds 589
    0.1 seconds - 0.5 seconds 225
    0.5 seconds - 1 second 78
    1 second - 2 seconds 7
    2 seconds - 5 seconds 0
    5 seconds + 0

    Background Migration: BackfillArchivedAndTraversalIdsToVulnerabilityReads

    Sampled 11 batches. Estimated Time to complete: 1 week, 6 days, 4 hours, and 44 minutes
    • Interval: 120s
    • Total tuple count: 95020256
    • Max batch size: 0
    • Estimated seconds to complete: 1140240s
    • Estimated number of batches: 9502
    • Average batch time: 79.05s
    • Batch size: 10000
    • N. of batches sampled: 11
    • N. of failed batches: 0

    Time estimation is conservative and based on sampling production data in a test environment. It represents the max time that migration could take. The actual time may differ from this estimation.

    _Consider changing max_batch_size and interval if this estimate is unacceptable._

    Calls Total Time Max Time Mean Time Rows Query
    1089 600734.5 ms 13295.2 ms 551.6 ms 108900
    UPDATE vulnerability_reads
    SET traversal_ids = namespaces.traversal_ids, archived = projects.archived
    FROM projects
    JOIN namespaces ON namespaces.id = projects.namespace_id
    WHERE vulnerability_reads.id IN (
    SELECT vulnerability_reads.id
    FROM vulnerability_reads
    WHERE vulnerability_reads.id BETWEEN $1 AND $2 AND vulnerability_reads.id >= $3 AND vulnerability_reads.id < $4
    ) AND vulnerability_reads.project_id = projects.id
    11 5198.1 ms 1362.9 ms 472.6 ms 1005
    UPDATE vulnerability_reads
    SET traversal_ids = namespaces.traversal_ids, archived = projects.archived
    FROM projects
    JOIN namespaces ON namespaces.id = projects.namespace_id
    WHERE vulnerability_reads.id IN (
    SELECT vulnerability_reads.id
    FROM vulnerability_reads
    WHERE vulnerability_reads.id BETWEEN $1 AND $2 AND vulnerability_reads.id >= $3
    ) AND vulnerability_reads.project_id = projects.id
    22 20.8 ms 16.8 ms 0.9 ms 22
    INSERT INTO batched_background_migration_job_transition_logs (batched_background_migration_job_id, created_at, updated_at, previous_status, next_status) VALUES ($1, $2, $3, $4, $5) RETURNING id
    1100 176.0 ms 11.4 ms 0.2 ms 1089
    SELECT vulnerability_reads.id
    FROM vulnerability_reads
    WHERE vulnerability_reads.id BETWEEN $1 AND $2 AND vulnerability_reads.id >= $3
    ORDER BY vulnerability_reads.id ASC
    LIMIT $4
    OFFSET $5
    11 8.0 ms 4.1 ms 0.7 ms 11
    UPDATE batched_background_migration_jobs
    SET updated_at = $1, started_at = $2, status = $3, attempts = $4
    WHERE batched_background_migration_jobs.id = $5
    11 9.8 ms 3.4 ms 0.9 ms 11
    UPDATE batched_background_migration_jobs
    SET updated_at = $1, finished_at = $2, status = $3, metrics = $4
    WHERE batched_background_migration_jobs.id = $5
    22 2.0 ms 0.4 ms 0.1 ms 22
    SELECT batched_background_migration_jobs.*
    FROM batched_background_migration_jobs
    WHERE batched_background_migration_jobs.id = $1
    LIMIT $2
    11 0.5 ms 0.1 ms 0.0 ms 11
    SELECT vulnerability_reads.id
    FROM vulnerability_reads
    WHERE vulnerability_reads.id BETWEEN $1 AND $2
    ORDER BY vulnerability_reads.id ASC
    LIMIT $3
    11 0.5 ms 0.1 ms 0.0 ms 11
    SELECT sum(batched_background_migration_jobs.batch_size)
    FROM batched_background_migration_jobs
    WHERE batched_background_migration_jobs.batched_background_migration_id = $1 AND batched_background_migration_jobs.status IN ($2)
    11 0.5 ms 0.1 ms 0.0 ms 11
    SELECT batched_background_migrations.*
    FROM batched_background_migrations
    WHERE batched_background_migrations.id = $1
    LIMIT $2
    Histogram of batch runtimes for BackfillArchivedAndTraversalIdsToVulnerabilityReads
    Batch Runtime Count
    0 seconds - 10 seconds 0
    10 seconds - 1 minute 2
    1 minute - 2 minutes 9
    2 minutes - 3 minutes 0
    3 minutes - 5 minutes 0
    5 minutes + 0
    Histogram across all sampled batches of BackfillArchivedAndTraversalIdsToVulnerabilityReads
    Query Runtime Count
    0 seconds - 0.1 seconds 1197
    0.1 seconds - 0.5 seconds 587
    0.5 seconds - 1 second 403
    1 second - 2 seconds 94
    2 seconds - 5 seconds 8
    5 seconds + 10

    Other information

    Other migrations pending on GitLab.com
    Migration Type Total runtime Result DB size change
    20240404115853 - AddPrefixedToCatalogResourceVersions Regular 2.2 s :white_check_mark: +0.00 B
    20240408105626 - AddSendBotMessageToPolicies Regular 1.7 s :white_check_mark: +0.00 B
    20240410061546 - AddIndexForMemberApprovalsMemberNamespaceIdStatus Regular 2.7 s :white_check_mark: +8.00 KiB [note]
    20240403020614 - PrepareTmpBackfillIndexForPipelineIdsToVulnerabilityOccurrences Post deploy 3.3 s :white_check_mark: +0.00 B
    20240408135326 - RemoveForeignKeysFromExternalApprovalRulesProtectedBranches Post deploy 2.3 s :white_check_mark: +0.00 B
    20240408135652 - DropExternalApprovalRulesProtectedBranchesTable Post deploy 1.9 s :white_check_mark: -32.00 KiB
    20240409070036 - SyncIndexForPipelinesUniqueIdBigint Post deploy 2.9 s :white_check_mark: +0.00 B
    20240409140739 - RequeueBackfillArchivedAndTraversalIdsToVulnerabilityReads Post deploy 7.6 s :warning: +0.00 B
    20240410004336 - AddDeploymentApprovalsProjectIdTrigger Post deploy 2.2 s :white_check_mark: +0.00 B
    20240410004337 - QueueBackfillDeploymentApprovalsProjectId Post deploy 2.3 s :white_check_mark: +0.00 B
    20240410070036 - PrepareAsyncIndexesForPipelinesIdBigint Post deploy 14.4 s :white_check_mark: +0.00 B
    20240411052531 - DropIdxMergeRequestsOnTargetProjectIdAndIidOpened Post deploy 3.3 s :white_check_mark: +0.00 B
    20240411070036 - AsyncFkIdBigint4CiPipelinesPCiBuildsCiPipelinesConfigPCiStages Post deploy 18.4 s :warning: +16.00 KiB
    20240412000002 - PrepareAsyncIndexForBuildsPart5 Post deploy 9.8 s :white_check_mark: +0.00 B
    Clone details
    Clone ID Clone Created At Clone Data Timestamp Expected Removal Time
    database-testing-3125776-13557265-main 2024-04-16T15:53:52Z 2024-04-15T10:51:44Z 2024-04-17 04:20:10 +0000
    database-testing-3125776-13557265-ci 2024-04-16T15:53:52Z 2024-04-16T12:41:29Z 2024-04-17 04:20:10 +0000

    Job artifacts

    Database migrations (on the ci database)

    1 Warnings
    :warning: 20240404192955 - CreateEarlyAccessProgramTrackingEvents had a query that exceeded timing
    guidelines
    . Run time should not exceed 100ms, but it was 532.34ms. Please consider possible options
    to improve the query performance.
    CREATE TABLE "early_access_program_tracking_events"
    ("id" bigserial primary key, "user_id" bigint NOT NULL, "event_name" text NOT NULL, "event_label"
    text, "category" text, "created_at" timestamptz NOT NULL, "updated_at" timestamptz NOT NULL,
    CONSTRAINT "fk_rails_3e8c32b3dd" FOREIGN KEY ("user_id") REFERENCES "users" ("id") ON DELETE
    CASCADE, CONSTRAINT check_573cf46c14 CHECK (char_length("event_name") <= 255), CONSTRAINT
    check_4faf711d09 CHECK (char_length("event_label") <= 255), CONSTRAINT check_e631e07806 CHECK
    (char_length("category") <= 255))

    Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).

    Migration Type Total runtime Result DB size change
    20240404191440 - AddEarlyAccessProgramParticipantToUserPreferences Regular 2.5 s :white_check_mark: +0.00 B
    20240404192955 - CreateEarlyAccessProgramTrackingEvents Regular 3.2 s :warning: +64.00 KiB
    Runtime Histogram for all migrations
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 12
    0.1 seconds - 1 second 2
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20240404191440 - AddEarlyAccessProgramParticipantToUserPreferences

    • Type: Regular
    • Duration: 2.5 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    1 52.4 ms 52.4 ms 52.4 ms 0
    ALTER TABLE "user_preferences" ADD "early_access_program_participant" boolean DEFAULT FALSE NOT NULL, ADD "early_access_program_tracking" boolean DEFAULT FALSE NOT NULL
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT $1::regtype::oid
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT pg_backend_pid()
    Histogram for AddEarlyAccessProgramParticipantToUserPreferences
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 5
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    :warning: Migration: 20240404192955 - CreateEarlyAccessProgramTrackingEvents

    • Type: Regular
    • Duration: 3.2 s
    • Database size change: +64.00 KiB
    Calls Total Time Max Time Mean Time Rows Query
    1 532.3 ms 532.3 ms 532.3 ms 0
    CREATE TABLE "early_access_program_tracking_events" ("id" bigserial primary key, "user_id" bigint NOT NULL, "event_name" text NOT NULL, "event_label" text, "category" text, "created_at" timestamptz NOT NULL, "updated_at" timestamptz NOT NULL, CONSTRAINT "fk_rails_3e8c32b3dd" FOREIGN KEY ("user_id") REFERENCES "users" ("id") ON DELETE CASCADE, CONSTRAINT check_573cf46c14 CHECK (char_length("event_name") <= 255), CONSTRAINT check_4faf711d09 CHECK (char_length("event_label") <= 255), CONSTRAINT check_e631e07806 CHECK (char_length("category") <= 255))
    1 27.2 ms 27.2 ms 27.2 ms 0
    CREATE INDEX "index_early_access_program_tracking_events_on_category" ON "early_access_program_tracking_events" ("category")
    1 24.1 ms 24.1 ms 24.1 ms 0
    CREATE TRIGGER gitlab_schema_write_trigger_for_early_access_program_tracking_events BEFORE INSERT OR
    UPDATE OR DELETE OR TRUNCATE ON early_access_program_tracking_events FOR EACH STATEMENT EXECUTE FUNCTION gitlab_schema_prevent_write()
    1 4.1 ms 4.1 ms 4.1 ms 0
    CREATE INDEX "index_early_access_program_tracking_events_on_event_name" ON "early_access_program_tracking_events" ("event_name")
    1 1.2 ms 1.2 ms 1.2 ms 0
    CREATE INDEX "index_early_access_program_tracking_events_on_event_label" ON "early_access_program_tracking_events" ("event_label")
    1 1.0 ms 1.0 ms 1.0 ms 0
    CREATE INDEX "index_early_access_program_tracking_events_on_user_id" ON "early_access_program_tracking_events" ("user_id")
    1 0.1 ms 0.1 ms 0.1 ms 1
    SELECT COUNT(*) from information_schema.triggers
    WHERE event_object_table = $1 AND trigger_name = $2
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT pg_backend_pid()
    :warning: Column ordering suggestions

    Ordering columns efficiently (as in below tables) will help us save more space, further details can be found here.

     CREATE TABLE "early_access_program_tracking_events" ("id" bigserial primary key, "user_id" bigint NOT NULL, "event_name" text NOT NULL, "event_label" text, "category" text, "created_at" timestamptz NOT NULL, "updated_at" timestamptz NOT NULL, CONSTRAINT "fk_rails_3e8c32b3dd" FOREIGN KEY ("user_id") REFERENCES "users" ("id") ON DELETE CASCADE, CONSTRAINT check_573cf46c14 CHECK (char_length("event_name") <= 255), CONSTRAINT check_4faf711d09 CHECK (char_length("event_label") <= 255), CONSTRAINT check_e631e07806 CHECK (char_length("category") <= 255)) 
    Column name Data Type
    id bigserial
    user_id int8
    created_at timestamptz
    updated_at timestamptz
    event_name text
    event_label text
    category text
    Histogram for CreateEarlyAccessProgramTrackingEvents
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 7
    0.1 seconds - 1 second 2
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Other information

    Other migrations pending on GitLab.com
    Migration Type Total runtime Result DB size change
    20240404115853 - AddPrefixedToCatalogResourceVersions Regular 2.6 s :white_check_mark: +0.00 B
    20240408135326 - RemoveForeignKeysFromExternalApprovalRulesProtectedBranches Post deploy 2.6 s :white_check_mark: +0.00 B
    20240408135652 - DropExternalApprovalRulesProtectedBranchesTable Post deploy 2.4 s :white_check_mark: -32.00 KiB
    20240409070036 - SyncIndexForPipelinesUniqueIdBigint Post deploy 3.1 s :white_check_mark: +0.00 B
    20240410070036 - PrepareAsyncIndexesForPipelinesIdBigint Post deploy 13.5 s :white_check_mark: +0.00 B
    20240411070036 - AsyncFkIdBigint4CiPipelinesPCiBuildsCiPipelinesConfigPCiStages Post deploy 17.6 s :warning: +24.00 KiB
    20240412000002 - PrepareAsyncIndexForBuildsPart5 Post deploy 9.8 s :white_check_mark: +0.00 B
    Clone details
    Clone ID Clone Created At Clone Data Timestamp Expected Removal Time
    database-testing-3125776-13557265-main 2024-04-16T15:53:52Z 2024-04-15T10:51:44Z 2024-04-17 04:20:10 +0000
    database-testing-3125776-13557265-ci 2024-04-16T15:53:52Z 2024-04-16T12:41:29Z 2024-04-17 04:20:10 +0000

    Job artifacts


    Brought to you by gitlab-org/database-team/gitlab-com-database-testing. Epic

  • Rostyslav Safonov changed the description

    changed the description

  • added 3 commits

    • b8d26ba5 - Early Access Program tracking init
    • 09e9afeb - Fix NoMethodError in DB docs specs
    • a95be5d0 - Adjust query limit to allow Early Access program tracking

    Compare with previous version

  • Rostyslav Safonov added 456 commits

    added 456 commits

    • a95be5d0...b0cee131 - 453 commits from branch gitlab-org:master
    • 1f2aef3e - Early Access Program tracking init
    • da2c5e1c - Fix NoMethodError in DB docs specs
    • b652c83f - Adjust query limit to allow Early Access program tracking

    Compare with previous version

  • requested review from @ddieulivol

  • Rostyslav Safonov requested review from @dgruzd

    requested review from @dgruzd

  • David Dieulivol approved this merge request

    approved this merge request

  • David Dieulivol requested review from @wandering_person and removed review request for @ddieulivol

    requested review from @wandering_person and removed review request for @ddieulivol

  • Dmitry Gruzd removed review request for @dgruzd

    removed review request for @dgruzd

  • Dmitry Gruzd changed milestone to %17.0

    changed milestone to %17.0

  • Michael Becker
  • added 2 commits

    • ab2f66b6 - Remove useless comments; Change milestone
    • ce7ae81a - Regenerated structure.sql

    Compare with previous version

  • Rostyslav Safonov reset approvals from @ddieulivol by pushing to the branch

    reset approvals from @ddieulivol by pushing to the branch

  • added 1 commit

    Compare with previous version

  • E2E Test Result Summary

    allure-report-publisher generated test report!

    e2e-test-on-gdk: :white_check_mark: test report for 2315c1d2

    expand test summary
    +------------------------------------------------------------------+
    |                          suites summary                          |
    +-------------+--------+--------+---------+-------+-------+--------+
    |             | passed | failed | skipped | flaky | total | result |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Govern      | 66     | 0      | 0       | 0     | 66    | ✅     |
    | Plan        | 51     | 0      | 2       | 0     | 53    | ✅     |
    | Create      | 96     | 0      | 9       | 0     | 105   | ✅     |
    | Data Stores | 31     | 0      | 0       | 0     | 31    | ✅     |
    | Verify      | 35     | 0      | 1       | 0     | 36    | ✅     |
    | Package     | 24     | 0      | 6       | 0     | 30    | ✅     |
    | Monitor     | 7      | 0      | 0       | 0     | 7     | ✅     |
    | Release     | 5      | 0      | 0       | 0     | 5     | ✅     |
    | Analytics   | 2      | 0      | 0       | 0     | 2     | ✅     |
    | Manage      | 0      | 0      | 1       | 0     | 1     | ➖     |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Total       | 317    | 0      | 19      | 0     | 336   | ✅     |
    +-------------+--------+--------+---------+-------+-------+--------+

    e2e-package-and-test: :white_check_mark: test report for 57897625

    expand test summary
    +---------------------------------------------------------------------+
    |                           suites summary                            |
    +----------------+--------+--------+---------+-------+-------+--------+
    |                | passed | failed | skipped | flaky | total | result |
    +----------------+--------+--------+---------+-------+-------+--------+
    | Monitor        | 36     | 0      | 13      | 0     | 49    | ✅     |
    | Plan           | 285    | 0      | 23      | 0     | 308   | ✅     |
    | Verify         | 168    | 0      | 30      | 0     | 198   | ✅     |
    | Govern         | 300    | 0      | 13      | 9     | 313   | ✅     |
    | Manage         | 38     | 0      | 12      | 1     | 50    | ✅     |
    | Create         | 600    | 0      | 81      | 5     | 681   | ✅     |
    | Secure         | 9      | 0      | 3       | 0     | 12    | ✅     |
    | Systems        | 10     | 0      | 0       | 0     | 10    | ✅     |
    | Package        | 211    | 0      | 44      | 7     | 255   | ✅     |
    | GitLab Metrics | 2      | 0      | 1       | 0     | 3     | ✅     |
    | ModelOps       | 0      | 0      | 3       | 0     | 3     | ➖     |
    | Fulfillment    | 8      | 0      | 75      | 1     | 83    | ✅     |
    | Analytics      | 7      | 0      | 0       | 0     | 7     | ✅     |
    | Data Stores    | 137    | 0      | 28      | 0     | 165   | ✅     |
    | Configure      | 1      | 0      | 9       | 0     | 10    | ✅     |
    | Release        | 17     | 0      | 3       | 0     | 20    | ✅     |
    | Growth         | 0      | 0      | 6       | 0     | 6     | ➖     |
    | Ai-powered     | 0      | 0      | 3       | 0     | 3     | ➖     |
    +----------------+--------+--------+---------+-------+-------+--------+
    | Total          | 1829   | 0      | 347     | 23    | 2176  | ✅     |
    +----------------+--------+--------+---------+-------+-------+--------+
  • Rostyslav Safonov added 518 commits

    added 518 commits

    • 7c69b8cc...64a4fefc - 513 commits from branch gitlab-org:master
    • d8e31df4 - Early Access Program tracking init
    • f1992b95 - Fix NoMethodError in DB docs specs
    • e021ea33 - Adjust query limit to allow Early Access program tracking
    • fdf0b467 - Remove useless comments; Change milestone
    • 0b6e5c40 - Regenerated structure.sql

    Compare with previous version

  • Rostyslav Safonov requested review from @dgruzd

    requested review from @dgruzd

  • Dmitry Gruzd approved this merge request

    approved this merge request

  • Dmitry Gruzd requested review from @pshutsin and removed review request for @dgruzd

    requested review from @pshutsin and removed review request for @dgruzd

  • Michael Becker added 183 commits

    added 183 commits

    Compare with previous version

  • Michael Becker reset approvals from @dgruzd by pushing to the branch

    reset approvals from @dgruzd by pushing to the branch

  • Michael Becker approved this merge request

    approved this merge request

  • added databaseapproved label and removed databasereviewed label

  • Pavel Shutsin approved this merge request

    approved this merge request

  • Pavel Shutsin removed review request for @pshutsin

    removed review request for @pshutsin

  • Rostyslav Safonov added 386 commits

    added 386 commits

    • f6475965...ab7e297c - 380 commits from branch gitlab-org:master
    • 40e3609c - Early Access Program tracking init
    • 607e78ed - Fix NoMethodError in DB docs specs
    • a9828019 - Adjust query limit to allow Early Access program tracking
    • db5ab79c - Remove useless comments; Change milestone
    • 37bb37df - Regenerated structure.sql
    • 4f63c123 - Fix Gitlab::InternalEvents specs memoization

    Compare with previous version

  • Rostyslav Safonov reset approvals from @pshutsin and @wandering_person by pushing to the branch

    reset approvals from @pshutsin and @wandering_person by pushing to the branch

  • Pavel Shutsin approved this merge request

    approved this merge request

  • added 1 commit

    • 2f584223 - Bump a couple of milestones to 17.0

    Compare with previous version

  • Michael Becker reset approvals from @pshutsin by pushing to the branch

    reset approvals from @pshutsin by pushing to the branch

  • Michael Becker resolved all threads

    resolved all threads

  • requested review from @wandering_person

  • Dmitry Gruzd approved this merge request

    approved this merge request

  • Pavel Shutsin approved this merge request

    approved this merge request

  • Pavel Shutsin resolved all threads

    resolved all threads

  • Rostyslav Safonov added 509 commits

    added 509 commits

    • 2f584223...0037e571 - 507 commits from branch gitlab-org:master
    • b241ffe0 - Merge branch 'master' of gitlab.com:gitlab-community/gitlab into...
    • 57897625 - Adjust queries threshold in epics_swimlanes_drag_drop_spec.rb

    Compare with previous version

  • Rostyslav Safonov reset approvals from @pshutsin and @dgruzd by pushing to the branch

    reset approvals from @pshutsin and @dgruzd by pushing to the branch

  • Rostyslav Safonov added 241 commits

    added 241 commits

    • 57897625...406cc7f7 - 239 commits from branch gitlab-org:master
    • 2c8d22a5 - Merge branch 'master' of gitlab.com:gitlab-community/gitlab into...
    • 2315c1d2 - Fix structure.sql TZ

    Compare with previous version

  • added 1 commit

    Compare with previous version

  • Pavel Shutsin approved this merge request

    approved this merge request

  • Pavel Shutsin resolved all threads

    resolved all threads

  • Hello @knockfog-ext :wave:

    The database team is looking for ways to improve the database review process and we would love your help!

    If you'd be open to someone on the database team reaching out to you for a chat, or if you'd like to leave some feedback asynchronously, just post a reply to this comment mentioning:

    @gitlab-org/database-team

    And someone will be by shortly!

    Thanks for your help! :heart:

    This message was generated automatically. You're welcome to improve it.

  • Pavel Shutsin mentioned in commit 3fc001c2

    mentioned in commit 3fc001c2

  • added workflowstaging label and removed workflowcanary label

  • Please register or sign in to reply
    Loading