Skip to content

Fix N+1 when generating pypi package's versions

🍀 What does this MR do and why?

This MR fixes N+1 issue, discovered while looking at Kibana search (internal) for /api/:version/projects/:id/packages/pypi/simple/*package_name that triggers too many SQL queries.

📸 Screenshots or screen recordings

🌼 With master branch

It creates separate SQL query for every package's file and package's metadatum.

Screenshot_2022-11-23_at_20.25.44

🌸 With this MR

It creates one SQL query for all package's files and one SQL query for package's metadatum.

Screenshot_2022-11-23_at_20.28.58

🔬 How to set up and validate locally

  1. Prepare two packages with the same name but different versions in rails console:

    def fixture_file_upload(*args, **kwargs)
      Rack::Test::UploadedFile.new(*args, **kwargs)
    end
    
    package = FactoryBot.create(:npm_package, project: Project.first)
    FactoryBot.create(:pypi_package, project: Project.first, name: package.name)
    
    # Grab package's name to use later in the API request
    package.name
  2. Trigger API request:

    curl --user <user>:<token> "http://gdk.test:3000/api/v4/projects/<project_id>/packages/pypi/simple/<package_name>"
  3. Verify SQL requests from app/presenters/packages/pypi/simple_package_versions_presenter.rb in the development.log

    With master branch

    It creates separate SQL query for every package's file and package's metadatum.

    With this MR

    It creates one SQL query for all package's files and one SQL query for package's metadatum.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #381595 (closed)

Edited by Dzmitry (Dima) Meshcharakou

Merge request reports