Skip to content
Snippets Groups Projects

Add Dependency list to Explore page

Merged mo khan requested to merge mokhax/432264/explore-dependencies into master
1 unresolved thread

What does this MR do and why?

This MR contains a spike implementation of a global dependency list that is available under /explore/dependencies. This initial version of the Dependency list is gated behind a feature flag and only available to instance admins. It requires membership to the default organization and removes features such as exporting, filtering, sorting, grouping, and advanced pagination.

SELECT 
  "sbom_occurrences"."id",
  "sbom_occurrences"."created_at",
  "sbom_occurrences"."updated_at",
  "sbom_occurrences"."component_version_id",
  "sbom_occurrences"."project_id",
  "sbom_occurrences"."pipeline_id",
  "sbom_occurrences"."source_id",
  "sbom_occurrences"."commit_sha",
  "sbom_occurrences"."component_id",
  "sbom_occurrences"."uuid",
  "sbom_occurrences"."package_manager",
  "sbom_occurrences"."component_name",
  "sbom_occurrences"."input_file_path",
  "sbom_occurrences"."licenses",
  "sbom_occurrences"."highest_severity",
  "sbom_occurrences"."vulnerability_count",
  "sbom_occurrences"."source_package_id"
FROM "sbom_occurrences"
INNER JOIN "projects" ON "sbom_occurrences"."project_id" = "projects"."id"
WHERE "projects"."organization_id" = 1 
ORDER BY "sbom_occurrences"."id" ASC
LIMIT 20
OFFSET 0;
Time: 25.281 ms
  - planning: 7.076 ms
  - execution: 18.205 ms
    - I/O read: 17.753 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 82 (~656.00 KiB) from the buffer pool
  - reads: 14 (~112.00 KiB) from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/25779/commands/81260

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

image image

How to set up and validate locally

  1. Enable explore_dependencies feature flag. Feature.enable(:explore_dependencies)
  2. Log in as an instance admin.
  3. Ensure membership in the default organization. ::Organizations::OrganizationUser.create_default_organization_record_for(1, user_is_admin: true)
  4. Open http://gdk.test:3000/explore/dependencies
Edited by mo khan

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • assigned to @mokhax

  • A deleted user added frontend label

    added frontend label

  • Contributor
    3 Warnings
    :warning: This merge request is quite big (755 lines changed), please consider splitting it into multiple merge requests.
    :warning: 8df2137e: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines.
    :warning:

    featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart.

    For more information, see:

    1 Message
    :book: CHANGELOG missing:

    If this merge request needs a changelog entry, add the Changelog trailer to the commit message you want to add to the changelog.

    If this merge request doesn't need a CHANGELOG entry, feel free to ignore this message.

    Reviewer roulette

    Category Reviewer Maintainer
    backend @robyrne profile link current availability (UTC+0, 7 hours ahead of author) @alexpooley profile link current availability (UTC+8, 15 hours ahead of author)
    database @dstull profile link current availability (UTC-5, 2 hours ahead of author) @dfrazao-gitlab profile link current availability (UTC+1, 8 hours ahead of author)
    frontend @elwyn-gitlab profile link current availability (UTC+13, 20 hours ahead of author) @blabuschagne profile link current availability (UTC+1, 8 hours ahead of author)
    ~"Authorization" Reviewer review is optional for ~"Authorization" @hmehra profile link current availability (UTC+11, 18 hours ahead of author)

    Please check reviewer's status!

    • available Reviewer is available!
    • unavailable Reviewer is unavailable!

    Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

    If needed, you can retry the :repeat: danger-review job that generated this comment.

    Generated by :no_entry_sign: Danger

  • Contributor

    E2E Test Result Summary

    allure-report-publisher generated test report!

    e2e-test-on-gdk: :white_check_mark: test report for 5d0b87e3

    expand test summary
    +------------------------------------------------------------------+
    |                          suites summary                          |
    +-------------+--------+--------+---------+-------+-------+--------+
    |             | passed | failed | skipped | flaky | total | result |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Create      | 8      | 0      | 3       | 0     | 11    | ✅     |
    | Govern      | 65     | 0      | 1       | 0     | 66    | ✅     |
    | Plan        | 4      | 0      | 0       | 0     | 4     | ✅     |
    | Data Stores | 2      | 0      | 0       | 0     | 2     | ✅     |
    | Monitor     | 4      | 0      | 0       | 0     | 4     | ✅     |
    | Package     | 0      | 0      | 1       | 0     | 1     | ➖     |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Total       | 83     | 0      | 5       | 0     | 88    | ✅     |
    +-------------+--------+--------+---------+-------+-------+--------+

    e2e-package-and-test: :white_check_mark: test report for 5d0b87e3

    expand test summary
    +------------------------------------------------------------------+
    |                          suites summary                          |
    +-------------+--------+--------+---------+-------+-------+--------+
    |             | passed | failed | skipped | flaky | total | result |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Govern      | 166    | 0      | 14      | 2     | 180   | ✅     |
    | Create      | 16     | 0      | 6       | 0     | 22    | ✅     |
    | Plan        | 8      | 0      | 0       | 0     | 8     | ✅     |
    | Data Stores | 4      | 0      | 0       | 0     | 4     | ✅     |
    | Monitor     | 8      | 0      | 0       | 0     | 8     | ✅     |
    | Package     | 0      | 0      | 2       | 0     | 2     | ➖     |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Total       | 202    | 0      | 22      | 2     | 224   | ✅     |
    +-------------+--------+--------+---------+-------+-------+--------+

    e2e-review-qa: :white_check_mark: test report for 5d0b87e3

    expand test summary
    +-------------------------------------------------------------+
    |                       suites summary                        |
    +--------+--------+--------+---------+-------+-------+--------+
    |        | passed | failed | skipped | flaky | total | result |
    +--------+--------+--------+---------+-------+-------+--------+
    | Govern | 3      | 0      | 0       | 0     | 3     | ✅     |
    +--------+--------+--------+---------+-------+-------+--------+
    | Total  | 3      | 0      | 0       | 0     | 3     | ✅     |
    +--------+--------+--------+---------+-------+-------+--------+
  • mo khan added 3 commits

    added 3 commits

    • 068c19a1 - Bootstrap the dependencies app
    • ce440aec - Load SBOM occurrences through the default organization
    • f4d2d979 - Undo accidental change

    Compare with previous version

  • mo khan changed the description

    changed the description

  • Contributor

    Bundle size analysis [beta]

    This compares changes in bundle size for entry points between the commits 13e55430 and 5d0b87e3

    :sparkles: Special assets

    Entrypoint / Name Size before Size after Diff Diff in percent
    average 4.14 MB 4.13 MB - -0.0 %
    mainChunk 3.13 MB 3.13 MB - 0.0 %

    :new: New entry points: 1

    Expand
    Entrypoint / Name Size before Size after Diff Diff in percent
    pages.explore.dependencies 0 Bytes 559.58 KB +559.58 KB 100.0 %

    Note: We do not have exact data for 13e55430. So we have used data from: ae1f8767.
    The target commit was too new, so we used the latest commit from master we have info on.
    It might help to rerun the bundle-size-review job
    This might mean that you have a few false positives in this report. If something unrelated to your code changes is reported, you can check this comparison in order to see if they caused this change.

    Please look at the full report for more details


    Read more about how this report works.

    Generated by :no_entry_sign: Danger

  • mo khan added 1 commit

    added 1 commit

    • 19f2348e - Rename NAMESPACE_EXPLORE TO NAMESPACE_ORGANIZATION

    Compare with previous version

  • mo khan mentioned in issue #432264 (closed)

    mentioned in issue #432264 (closed)

  • mo khan mentioned in issue #438729 (closed)

    mentioned in issue #438729 (closed)

  • mo khan mentioned in issue #438743 (closed)

    mentioned in issue #438743 (closed)

  • mo khan mentioned in epic &12460

    mentioned in epic &12460

  • mo khan changed the description

    changed the description

  • mo khan added 250 commits

    added 250 commits

    • 19f2348e...4bd3b171 - 244 commits from branch master
    • 741f0ac6 - Scaffold out explore/dependencies
    • 0654d495 - Bootstrap the dependencies app
    • 212f51bf - Load SBOM occurrences through the default organization
    • 0aa12dd0 - Undo accidental change
    • 69637f95 - Rename NAMESPACE_EXPLORE TO NAMESPACE_ORGANIZATION
    • 09075ce0 - Restrict access to /explore/dependencies to self managed

    Compare with previous version

  • mo khan
  • mo khan
  • mo khan added 228 commits

    added 228 commits

    • 09075ce0...c4eb4dba - 221 commits from branch master
    • f4e3b5a0 - Scaffold out explore/dependencies
    • 93926a0a - Bootstrap the dependencies app
    • 9842e442 - Load SBOM occurrences through the default organization
    • c98221c9 - Undo accidental change
    • 5fafbaf7 - Rename NAMESPACE_EXPLORE TO NAMESPACE_ORGANIZATION
    • c2a693e4 - Restrict access to /explore/dependencies to self managed
    • 68e19d02 - Move explore/dependencies menu to /ee

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • c2e2d7c7 - Display explore/dependencies via licensed feature

    Compare with previous version

  • mo khan added 2 commits

    added 2 commits

    • 01fd9feb - Move explore/dependencies routes to /ee
    • c2dfc942 - Move explore/dependencies controller to /ee

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    Compare with previous version

  • mo khan added 198 commits

    added 198 commits

    • a32809fa...f8aa321b - 186 commits from branch master
    • f8aa321b...56f269b6 - 2 earlier commits
    • 82128a02 - Load SBOM occurrences through the default organization
    • 3526644b - Undo accidental change
    • cf4336f8 - Rename NAMESPACE_EXPLORE TO NAMESPACE_ORGANIZATION
    • e02da478 - Restrict access to /explore/dependencies to self managed
    • 219d5c8e - Move explore/dependencies menu to /ee
    • 80791607 - Display explore/dependencies via licensed feature
    • 535978ee - Move explore/dependencies routes to /ee
    • 607b7d53 - Move explore/dependencies controller to /ee
    • f97ae3ee - Test out the JSON endpoint
    • 45e2e0bc - Remove zero occurrences unless an organization, group or project is provided

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 652781be - Render dependencies at an organization level

    Compare with previous version

  • A deleted user added feature flag label

    added feature flag label

  • mo khan added 2 commits

    added 2 commits

    • 9be5d6ec - Move has_many :sbom_occurrences to ee/
    • 59b22573 - Tag spec with dependency_management category

    Compare with previous version

  • mo khan added 2 commits

    added 2 commits

    • 06c6f0cc - Remove license column from table
    • cc16692b - Move javascript initializer to /ee

    Compare with previous version

  • mo khan added 3 commits

    added 3 commits

    • 465941ed - Display location column if a count is not provided
    • 469d5600 - Display licenses if provided via JSON
    • 1c9024a4 - Display licenses if provided via JSON

    Compare with previous version

  • mo khan added 2 commits

    added 2 commits

    • 7879436d - Paginate without totals to speed up page
    • e2bc2522 - Update finder spec to filter by organization

    Compare with previous version

  • mo khan added 2 commits

    added 2 commits

    • c978232f - Fix up entity serializer
    • cd99cfa6 - Check for project on request context

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 510ca6cc - Use try instead of ternary operator

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    Compare with previous version

  • mo khan changed milestone to %16.9

    changed milestone to %16.9

  • mo khan changed title from Draft: Scaffold out explore/dependencies to Draft: {+Add /+}explore/dependencies{++}

    changed title from Draft: Scaffold out explore/dependencies to Draft: {+Add /+}explore/dependencies{++}

  • mo khan added 1 commit

    added 1 commit

    • cc8cbabb - Attempt to load the results efficiently

    Compare with previous version

  • added groupthreat insights label and removed groupauthorization label

  • mo khan added 1 commit

    added 1 commit

    • 23fd3cba - Add spec for rendering dependencies at organization level

    Compare with previous version

  • mo khan added 332 commits

    added 332 commits

    • 23fd3cba...f28c402c - 302 commits from branch master
    • f28c402c...ebb78665 - 20 earlier commits
    • a1531bd3 - Paginate without totals to speed up page
    • a350fddd - Update finder spec to filter by organization
    • a01965ed - Fix up entity serializer
    • 9621215b - Check for project on request context
    • dc7f525b - Use try instead of ternary operator
    • 2bdddda0 - Add N+1 spec
    • 19ff8664 - Attempt to load the results efficiently
    • 17036822 - Add spec for rendering dependencies at organization level
    • 53d66191 - Extract organizationFields property
    • 3500d48d - Add spec for Organizations::Organization#sbom_occurrences

    Compare with previous version

  • mo khan added 2 commits

    added 2 commits

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 09634408 - Update spec to include through association

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 1f7b8303 - Conditionally render dependencies menu

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 14287499 - Extract include_menu matcher

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 537d04fa - Use matcher to remove duplication

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 9dc5daa6 - Check feature flag in frontend code

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 8090f169 - Hide the export button when the export endpoint is not defined

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 2ee5733b - Disable sorting at organization level

    Compare with previous version

  • mo khan added 846 commits

    added 846 commits

    • 2ee5733b...e555da1f - 805 commits from branch master
    • e555da1f...44efa7bb - 31 earlier commits
    • 08f42519 - Add rollout issue url
    • 52a8b6b4 - Update spec to include through association
    • ae8d3287 - Conditionally render dependencies menu
    • cc112e72 - Extract include_menu matcher
    • 915cd384 - Use matcher to remove duplication
    • fe5420d5 - Check feature flag in frontend code
    • ec423659 - Hide the export button when the export endpoint is not defined
    • 329d98a6 - Disable sorting at organization level
    • ce6e9d98 - Bulk create projects in specs
    • 9d605238 - Fix N+1 on organization->project->namespace->route

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 8b15a1df - Authorize access to /explore/dependencies

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • eafcd688 - Move read_dependency permission spec to ee/

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 948ed82c - Move ee specific policy to ee/

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • d11cd3e7 - Add read_licenses policy for Organization

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • bb91b427 - Remove duplicate predicate in before_action

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • cca0b2bb - Update serializer to look for namespace key

    Compare with previous version

  • mo khan changed the description

    changed the description

  • mo khan marked this merge request as ready

    marked this merge request as ready

  • mo khan changed the description

    changed the description

  • mo khan
  • mo khan
  • mo khan
  • mo khan
  • mo khan
    • Author Developer
      Resolved by Savas Vedova

      This is a big MR because it introduces a new page under /explore. I wanted to make sure that the frontend and backend pieces fit nicely together so I worked on both sets of changes in 1 MR. I know that this might not be ideal and I hope you can bare with me for this first MR. Pretty please with sugar on top. :pray:

      @dpisek do you mind doing a frontend review?

      @zmartins do you mind doing a backend review?

      @minac do you mind doing a database review?

  • Zamir Martins requested review from @zmartins

    requested review from @zmartins

  • Zamir Martins requested review from @dpisek and @minac

    requested review from @dpisek and @minac

  • Zamir Martins
  • Zamir Martins removed review request for @zmartins

    removed review request for @zmartins

  • Mehmet Emin INAC
  • Mehmet Emin INAC
  • Mehmet Emin INAC
  • Mehmet Emin INAC requested review from @ghavenga and removed review request for @minac

    requested review from @ghavenga and removed review request for @minac

  • David Pisek
  • David Pisek
  • mo khan changed title from Add {-/explore/dependencies-} to Add Dependency list to Explore page

    changed title from Add {-/explore/dependencies-} to Add Dependency list to Explore page

  • mo khan added 1 commit

    added 1 commit

    • bbdb5ac9 - Rename namepsace to organization

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • c4c01761 - Add specs for DependencyEntity with an organization

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • d63abf75 - Test that read_licenses is disabled

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 6cf4256c - Add spec when signed in as admin with feature not available

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 38861a65 - Test sign in as admin with feature disabled against JSON endpoint

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 8df2137e - Disable actions bar for organization in App component

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    Compare with previous version

  • mo khan requested review from @zmartins

    requested review from @zmartins

  • David Pisek approved this merge request

    approved this merge request

  • David Pisek
  • Zamir Martins approved this merge request

    approved this merge request

  • Zamir Martins requested review from @bwill

    requested review from @bwill

  • Zamir Martins removed review request for @zmartins

    removed review request for @zmartins

  • mo khan added 1 commit

    added 1 commit

    • 9bc05f0b - Apply 1 suggestion(s) to 1 file(s)

    Compare with previous version

  • mo khan revoked approvals from @dpisek by pushing to the branch

    revoked approvals from @dpisek by pushing to the branch

  • mo khan added 1 commit

    added 1 commit

    • 627b7b47 - Use shield icon instead of package icon

    Compare with previous version

  • David Pisek approved this merge request

    approved this merge request

  • mo khan added 1 commit

    added 1 commit

    • 532bed8e - Apply 1 suggestion(s) to 1 file(s)

    Compare with previous version

  • mo khan revoked approvals from @dpisek by pushing to the branch

    revoked approvals from @dpisek by pushing to the branch

  • mo khan added 1 commit

    added 1 commit

    Compare with previous version

  • Brian Williams
  • Brian Williams
  • Brian Williams removed review request for @bwill

    removed review request for @bwill

  • added groupauthorization label and removed groupthreat insights label

  • mo khan added 1 commit

    added 1 commit

    • df9cd8ad - Change feature flag ownership to Authz group

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 06722ea8 - Return HTTP Forbidden status

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 7646d9c7 - Rename project_or_group to dependable

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • bff95e89 - Remove check for blank organization

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • b42bd65f - Apply 1 suggestion(s) to 1 file(s)

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    Compare with previous version

  • mo khan added 1 commit

    added 1 commit

    • 1a1084b7 - Render menu when user has access to default organization

    Compare with previous version

  • mo khan added 1171 commits

    added 1171 commits

    Compare with previous version

  • mo khan requested review from @mokhax

    requested review from @mokhax

  • mo khan requested review from @bwill and removed review request for @mokhax

    requested review from @bwill and removed review request for @mokhax

  • Brian Williams approved this merge request

    approved this merge request

  • Brian Williams removed review request for @bwill

    removed review request for @bwill

  • mo khan requested review from @alexbuijs

    requested review from @alexbuijs

  • Gregory Havenga approved this merge request

    approved this merge request

  • added databaseapproved label and removed databasereviewed label

  • Gregory Havenga
  • Alex Buijs
  • Alex Buijs
  • Alex Buijs removed review request for @alexbuijs

    removed review request for @alexbuijs

  • Alex Buijs approved this merge request

    approved this merge request

  • David Pisek requested review from @svedova

    requested review from @svedova

  • Savas Vedova approved this merge request

    approved this merge request

  • Savas Vedova enabled an automatic merge when all merge checks for 5d0b87e3 pass

    enabled an automatic merge when all merge checks for 5d0b87e3 pass

  • Savas Vedova resolved all threads

    resolved all threads

  • merged

  • Hello @mokhax :wave:

    The database team is looking for ways to improve the database review process and we would love your help!

    If you'd be open to someone on the database team reaching out to you for a chat, or if you'd like to leave some feedback asynchronously, just post a reply to this comment mentioning:

    @gitlab-org/database-team

    And someone will be by shortly!

    Thanks for your help! :heart:

    This message was generated automatically. You're welcome to improve it.

  • Savas Vedova mentioned in commit 130b5f87

    mentioned in commit 130b5f87

  • added workflowstaging label and removed workflowcanary label

  • mo khan mentioned in issue #441683 (closed)

    mentioned in issue #441683 (closed)

  • Please register or sign in to reply
    Loading