Skip to content

Fix nuget metadata endpoint wrong "upper" or "lower" version

What does this MR do and why?

In the NuGet Repository metadata endpoint, the only param we receive is the package_name. The metadata endpoint should respond with a set of information about the requested package. Among those information, there are the "lower" & "upper" versions of the package.

The maximum number of versions the metadata endpoint is allowed to respond with is 300. If a specific package has versions more than 300, the "lower" version attribute in the response would be incorrect. Why?

Because in order to determine the "lower" & "upper" versions, we iterate on the fetched packages and extract their versions, sort them, and then return the first and last as the "lower" & "upper" versions. And since we are limited to 300 versions, the "lower" version would be the first version of the most recent 300 versions, but not the correct actual lower version.

The ultimate solution would be caching the metadata for each package and storing it as a JSON file in the Object Storage. However, I tried to figure out the cost of querying all versions of a package if it has more than 300 versions, and it turned out that it's not that expensive query with the right database index in place.

So as an alternative fix, we can rely on fetching all versions from the database when the versions count exceeds the limit of 300. Otherwise, we don't make any extra queries.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. Create some dummy Nuget packages (say 5) with the same name but different versions and publish them to a project in your local NuGet Repository
    • to create a dummy package:
      1. make sure you have nuget cli installed
      2. navigate to a new directory and run nuget spec in the terminal. A new Package.nuspec file will be created.
      3. Open the .nuspec file in your text editor and modify the version value (the default is 1.0.0) then save the file and close it.
      4. Run nuget pack in the terminal. A new Package.X.X.X.nupkg file will be created. X.X.X is the version you put in the .nuspec file.
      5. Publish the .nupkg file to the NuGet Repository
      6. Repeat the previous steps by changing the version and creating a new .nupkg package file. Publish the new .nupkg to publish a new version.
  2. In the rails console enable the feature flag:
Feature.enable(:nuget_metadata_version_bounds, <project>)
  1. Change this package finder limit to be less than the number of versions you published. So if you published 5 packages, make this limit 3 for example. That would allow you to validate that the correct lower and upper bounds of the version are returned correctly in the metadata endpoint.
  2. In the browser open this metadata endpoint URL for the package you are testing against:
http://gdk.test:3000/api/v4/projects/<project_id>/packages/nuget/metadata/<package_name>/index.json
  1. Check the "lower" & "upper" attributes in the JSON response, they shouldn't be limited to the finder's limit; the correct bounds should be returned.
  2. On master, the "lower" value will be different and limited to the finder's limit returned packages.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Database analysis

!131994 (comment 1566862504)

Related to #241376

Edited by Moaz Khalifa

Merge request reports