Skip to content

Update naming validation for Conan recipes

Steve Abrams requested to merge 37335-conan-name-validation into master

What does this MR do?

Updates package name validation to exclude conan, and adds a custom validation to validate conan recipe uniqueness.

Database data

Validation method

The query will look like:

SELECT 1 AS one
  FROM packages_packages
 INNER JOIN packages_conan_metadata ON packages_conan_metadata.package_id = packages_packages.id
 WHERE packages_packages.project_id = 1
   AND packages_packages.package_type = 3
   AND packages_packages.name = 'foo'
   AND packages_packages.version = '1.0'
   AND packages_conan_metadata.package_channel = 'stable'
   AND packages_conan_metadata.package_username = 'mygroup+myorg'
   AND packages_packages.id != 1

Here is the resulting explain when run on #database-lab with a project known to have conan packages using the existing indexes (with execution):

 Nested Loop  (cost=0.29..8.51 rows=1 width=4) (actual time=5.770..5.770 rows=0 loops=1)
   Buffers: shared hit=3 read=4
   I/O Timings: read=5.648
   ->  Seq Scan on public.packages_conan_metadata  (cost=0.00..4.18 rows=1 width=8) (actual time=1.152..1.156 rows=2 loops=1)
         Filter: (((packages_conan_metadata.package_channel)::text = 'stable'::text) AND ((packages_conan_metadata.package_username)::text = 'mygroup+myorg'::text))
         Rows Removed by Filter: 10
         Buffers: shared read=1
         I/O Timings: read=1.116
   ->  Index Scan using packages_packages_pkey on public.packages_packages  (cost=0.29..4.32 rows=1 width=8) (actual time=2.300..2.300 rows=0 loops=2)
         Index Cond: (packages_packages.id = packages_conan_metadata.package_id)
         Filter: ((packages_packages.id <> 1) AND (packages_packages.project_id = 1) AND (packages_packages.package_type = 3) AND ((packages_packages.name)::text = 'foo'::text) AND ((packages_packages.version)::text = '1.0'::text))
         Rows Removed by Filter: 1
         Buffers: shared hit=3 read=3
         I/O Timings: read=4.532

visualized explain plan

Summary:

Time: 8.069 ms
  - planning: 2.194 ms
  - execution: 5.875 ms
    - I/O read: 5.648 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 3 (~24.00 KiB) from the buffer pool
  - reads: 4 (~32.00 KiB) from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

Here is the same query after updating with the new index:

Nested Loop  (cost=0.55..6.61 rows=1 width=4) (actual time=3.623..3.623 rows=0 loops=1)
   Buffers: shared hit=3 read=4
   I/O Timings: read=3.549
   ->  Index Scan using idx_packages_packages_on_project_id_name_version_package_type on public.packages_packages  (cost=0.42..4.45 rows=1 width=8) (actual time=3.623..3.623 rows=0 loops=1)
         Index Cond: ((packages_packages.project_id = 1) AND ((packages_packages.name)::text = 'foo'::text) AND ((packages_packages.version)::text = '1.0'::text) AND (packages_packages.package_type = 3))
         Filter: (packages_packages.id <> 1)
         Rows Removed by Filter: 1
         Buffers: shared hit=3 read=4
         I/O Timings: read=3.549
   ->  Index Only Scan using index_packages_conan_metadata_on_package_id_username_channel on public.packages_conan_metadata  (cost=0.14..2.16 rows=1 width=8) (actual time=0.000..0.000 rows=0 loops=0)
         Index Cond: ((packages_conan_metadata.package_id = packages_packages.id) AND (packages_conan_metadata.package_username = 'mygroup+myorg'::text) AND (packages_conan_metadata.package_channel = 'stable'::text))
         Heap Fetches: 0

visualized explain

Summary

Time: 5.240 ms
  - planning: 1.571 ms
  - execution: 3.669 ms
    - I/O read: 3.549 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 3 (~24.00 KiB) from the buffer pool
  - reads: 4 (~32.00 KiB) from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

The project used for this query has 3 Conan packages.

Index migrations

Results as supplied from #database-lab

exec CREATE UNIQUE INDEX CONCURRENTLY "index_packages_conan_metadata_on_package_id_username_channel" ON "packages_conan_metadata" ("package_id", "package_username", "package_channel")

The query has been executed. Duration: 97.000 ms
exec drop index "index_packages_conan_metadata_on_package_id

The query has been executed. Duration: 25.000 ms

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team

Closes #37335 (closed)

Edited by Steve Abrams

Merge request reports