Fix NuGet installs with symbol packages
🔎 What does this MR do?
With !63581 (merged), we recently added support for NuGet symbol packages in the GitLab package registry.
There is a bug, however, when installing NuGet packages where a symbol package is present. When installing a package, the NuGet client first requests some metadata for the given package. This metadata contains the download URLs for the package archive. Right now, we simply return the URL for the last file belonging to a given package. The problem is, now that we support symbol packages, the package could contain a regular package file, and a symbol package file. If the symbol package file was the last uploaded, it will be returned instead of the regular package file. The NuGet client will attempt to install the symbol package and
This MR updates the metadata presenter to always return the actual NuGet package files, which all are of the .nupkg format (symbol packages have .snupkg format).
📽 Screenshots (strongly suggested)
The packageContent value in the responses below contains the download URL for the package archive. Note in the before it is returning the .snupkg file, which is incorrect.
Before
→ curl --user root:$TOKEN "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/index" | jq
{
"count": 1,
"items": [
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json",
"lower": "8.1.97",
"upper": "8.1.97",
"count": 1,
"items": [
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json",
"packageContent": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/download/Prism.Core/8.1.97/prism.core.8.1.97.snupkg",
"catalogEntry": {
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json",
"authors": "",
"dependencyGroups": [
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json#dependencyGroup/.netframework4.6.1",
"@type": "PackageDependencyGroup",
"targetFramework": ".NETFramework4.6.1",
"dependencies": [
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json#dependencyGroup/.netframework4.6.1/system.valuetuple",
"@type": "PackageDependency",
"id": "System.ValueTuple",
"range": "4.5.0"
}
]
}
],
"id": "Prism.Core",
"version": "8.1.97",
"tags": "xaml mvvm xamarin wpf prism",
"packageContent": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/download/Prism.Core/8.1.97/prism.core.8.1.97.snupkg",
"summary": "",
"projectUrl": "https://github.com/PrismLibrary/Prism",
"licenseUrl": "https://aka.ms/deprecateLicenseUrl"
}
}
]
}
]
}
→ curl --user root:$TOKEN "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97" | jq
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json",
"packageContent": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/download/Prism.Core/8.1.97/prism.core.8.1.97.snupkg",
"catalogEntry": {
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json",
"authors": "",
"dependencyGroups": [
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json#dependencyGroup/.netframework4.6.1",
"@type": "PackageDependencyGroup",
"targetFramework": ".NETFramework4.6.1",
"dependencies": [
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json#dependencyGroup/.netframework4.6.1/system.valuetuple",
"@type": "PackageDependency",
"id": "System.ValueTuple",
"range": "4.5.0"
}
]
}
],
"id": "Prism.Core",
"version": "8.1.97",
"tags": "xaml mvvm xamarin wpf prism",
"packageContent": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/download/Prism.Core/8.1.97/prism.core.8.1.97.snupkg",
"summary": "",
"projectUrl": "https://github.com/PrismLibrary/Prism",
"licenseUrl": "https://aka.ms/deprecateLicenseUrl"
}
}After
→ curl --user root:$TOKEN "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/index" | jq
{
"count": 1,
"items": [
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json",
"lower": "8.1.97",
"upper": "8.1.97",
"count": 1,
"items": [
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json",
"packageContent": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/download/Prism.Core/8.1.97/prism.core.8.1.97.nupkg",
"catalogEntry": {
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json",
"authors": "",
"dependencyGroups": [
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json#dependencyGroup/.netframework4.6.1",
"@type": "PackageDependencyGroup",
"targetFramework": ".NETFramework4.6.1",
"dependencies": [
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json#dependencyGroup/.netframework4.6.1/system.valuetuple",
"@type": "PackageDependency",
"id": "System.ValueTuple",
"range": "4.5.0"
}
]
}
],
"id": "Prism.Core",
"version": "8.1.97",
"tags": "xaml mvvm xamarin wpf prism",
"packageContent": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/download/Prism.Core/8.1.97/prism.core.8.1.97.nupkg",
"summary": "",
"projectUrl": "https://github.com/PrismLibrary/Prism",
"licenseUrl": "https://aka.ms/deprecateLicenseUrl"
}
}
]
}
]
}
→ curl --user root:$TOKEN "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97" | jq
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json",
"packageContent": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/download/Prism.Core/8.1.97/prism.core.8.1.97.nupkg",
"catalogEntry": {
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json",
"authors": "",
"dependencyGroups": [
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json#dependencyGroup/.netframework4.6.1",
"@type": "PackageDependencyGroup",
"targetFramework": ".NETFramework4.6.1",
"dependencies": [
{
"@id": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/metadata/Prism.Core/8.1.97.json#dependencyGroup/.netframework4.6.1/system.valuetuple",
"@type": "PackageDependency",
"id": "System.ValueTuple",
"range": "4.5.0"
}
]
}
],
"id": "Prism.Core",
"version": "8.1.97",
"tags": "xaml mvvm xamarin wpf prism",
"packageContent": "https://gdk.test:3443/api/v4/projects/22/packages/nuget/download/Prism.Core/8.1.97/prism.core.8.1.97.nupkg",
"summary": "",
"projectUrl": "https://github.com/PrismLibrary/Prism",
"licenseUrl": "https://aka.ms/deprecateLicenseUrl"
}
}
🐘 Database
This MR adds a new scope to the Packages::PackageFile model: .with_format(format)
The query this is used in is:
package.package_files.with_format(NUGET_PACKAGE_FORMAT).last&.file_namewhich generates the SQL:
SELECT "packages_package_files".*
FROM "packages_package_files"
WHERE "packages_package_files"."package_id" = 2137184
AND "packages_package_files"."file_name" ILIKE '%.nupkg'
ORDER BY "packages_package_files"."id" DESC
LIMIT 1;Explain plan (postgres.ai link):
Limit (cost=44.22..44.23 rows=1 width=829) (actual time=10.432..10.433 rows=1 loops=1)
Buffers: shared hit=6 read=6 dirtied=1
I/O Timings: read=9.453 write=0.000
-> Sort (cost=44.22..44.23 rows=1 width=829) (actual time=10.430..10.431 rows=1 loops=1)
Sort Key: packages_package_files.id DESC
Sort Method: quicksort Memory: 25kB
Buffers: shared hit=6 read=6 dirtied=1
I/O Timings: read=9.453 write=0.000
-> Index Scan using index_packages_package_files_on_package_id_and_file_name on public.packages_package_files (cost=0.56..44.21 rows=1 width=829) (actual time=9.849..10.394 rows=1 loops=1)
Index Cond: (packages_package_files.package_id = 2137184)
Filter: ((packages_package_files.file_name)::text ~~* '%.nupkg'::text)
Rows Removed by Filter: 1
Buffers: shared hit=3 read=6 dirtied=1
I/O Timings: read=9.453 write=0.000Note that this will always be used within the scope of one package (package.package_files...). So by the time we get to matching based on file_name, the number of results will be relatively small. I think we could consider a trigram GIN index, but I think it might be pre/over-optimizing at this point until we start to see any slower queries.
☑ Does this MR meet the acceptance criteria?
Conformity
-
I have included changelog trailers, or none are needed. (Does this MR need a changelog?) -
I have added/updated documentation, or it's not needed. (Is documentation required?) -
I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?) -
I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?) -
I have self-reviewed this MR per code review guidelines. -
This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines) -
I have followed the style guides. -
This change is backwards compatible across updates, or this does not apply.
Availability and Testing
-
I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.) -
I have tested this MR in all supported browsers, or it's not needed. -
I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed.
Related to #334551 (closed)
Edited by Steve Abrams