Skip to content

Add Nuget metadatum support

David Fernandez requested to merge 42680-10io-add-nuget-metadata into master

What does this MR do?

This MR adds nuget metadata support at the package level.

The .nuspec file has several fields that we are interested in (see #42680 (closed)). This MR adds support for the following fields:

  • projectUrl
  • licenseUrl
  • iconUrl

Checking the .nuspec XSD we can see that all these fields are optional:

Once we have those fields saved, they will be exposed in those APIs:

  • Nuget Search endpoint
  • Nuget metadata endpoint for a package name
  • Nuget metadata endpoint for a package name + package version

Approach

  • We're going to re-use the same approach used on the metadata for other package types: add a packages_nuget_metadata table and link it to Packages::Package.
  • We will have 3 text columns, project_url, license_url and icon_url. All 3 will have a constraint to keep them at a reasonable length.
  • At the Packages::NugetMetadatum, we will validate them using the public_url validator.
  • In addition, we will validate that we don't have 3 blank values.
  • Note that nuget package file can be uploaded several times. Each time, we need to check if a Packages::Package already exists and re-use it if that's the case.
  • From the previous point, on the nuget metadatum, we will need to update all the 3 columns with the latest version and keep a eye that if we have a new version where these 3 values are blank and we have a Packages::NugetMetadatum object, we should destroy it.
  • The above is handled by a dedicated service: Packages::Nuget::UpdateMetadatumService.
  • At the grape entity level, we are using a shared entity to deal with all three fields: EE::API::Entities::Nuget::Metadatum

Screenshots

Uploading a nuget package:

$ nuget push DummyProject.WithMoreMetadata.1.2.3.nupkg -source local
WARNING: No API Key was provided and no API Key could be found for 'http://gitlab.local:8000/api/v4/projects/1/packages/nuget'. To save an API Key for a source use the 'setApiKey' command.
Pushing DummyProject.WithMoreMetadata.1.2.3.nupkg to 'http://gitlab.local:8000/api/v4/projects/1/packages/nuget'...
  PUT http://gitlab.local:8000/api/v4/projects/1/packages/nuget/
  Created http://gitlab.local:8000/api/v4/projects/1/packages/nuget/ 9379ms
Your package was pushed.

Visual Studio will use these urls to present more details while searching for packages with the GitLab Nuget Repository:

Screenshot_2020-05-06_at_16.10.04

Database Review

Up migration

== 20200430130048 CreatePackagesNugetMetadata: migrating ======================
-- table_exists?(:packages_nuget_metadata)
   -> 0.0004s
-- create_table(:packages_nuget_metadata, {:id=>false})
   -> 0.0408s
-- transaction_open?()
   -> 0.0000s
-- execute("ALTER TABLE packages_nuget_metadata\nADD CONSTRAINT packages_nuget_metadata_license_url_constraint\nCHECK ( char_length(license_url) <= 255 )\nNOT VALID;\n")
   -> 0.0010s
-- execute("SET statement_timeout TO 0")
   -> 0.0002s
-- execute("ALTER TABLE packages_nuget_metadata VALIDATE CONSTRAINT packages_nuget_metadata_license_url_constraint;")
   -> 0.0004s
-- execute("RESET ALL")
   -> 0.0001s
-- transaction_open?()
   -> 0.0000s
-- execute("ALTER TABLE packages_nuget_metadata\nADD CONSTRAINT packages_nuget_metadata_project_url_constraint\nCHECK ( char_length(project_url) <= 255 )\nNOT VALID;\n")
   -> 0.0002s
-- execute("SET statement_timeout TO 0")
   -> 0.0001s
-- execute("ALTER TABLE packages_nuget_metadata VALIDATE CONSTRAINT packages_nuget_metadata_project_url_constraint;")
   -> 0.0004s
-- execute("RESET ALL")
   -> 0.0001s
-- transaction_open?()
   -> 0.0000s
-- execute("ALTER TABLE packages_nuget_metadata\nADD CONSTRAINT packages_nuget_metadata_icon_url_constraint\nCHECK ( char_length(icon_url) <= 255 )\nNOT VALID;\n")
   -> 0.0013s
-- execute("SET statement_timeout TO 0")
   -> 0.0001s
-- execute("ALTER TABLE packages_nuget_metadata VALIDATE CONSTRAINT packages_nuget_metadata_icon_url_constraint;")
   -> 0.0008s
-- execute("RESET ALL")
   -> 0.0001s
== 20200430130048 CreatePackagesNugetMetadata: migrated (0.0563s) =============

Down migration

== 20200430130048 CreatePackagesNugetMetadata: reverting ======================
-- drop_table(:packages_nuget_metadata)
   -> 0.0161s
== 20200430130048 CreatePackagesNugetMetadata: reverted (0.0161s) =============

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by David Fernandez

Merge request reports