Skip to content

Adds graphql search options for Ml::ModelVersion

Eduardo Bonet requested to merge model_registry/search_model_versions into master

What does this MR do and why?

Adds graphql search options for Ml::ModelVersion

Adds a new resolver that allows searching through the versions of a machine learning model, along with model changes necessary to support the searching and ordering.

How to set up and validate locally

  1. In rails console enable the experiment fully

    Feature.enable(:model_registry)
  2. In rails console, create a model and a few model versions

    p = Project.find_by(id: 1)
    Ml::FindOrCreateModelService.new(p, "model_1").execute
    Ml::FindOrCreateModelVersionService.new(p, { model_name: "model_1", version: "2.0.0" })
    5.times { |i| Ml::FindOrCreateModelVersionService.new(p, { model_name: "model_1", version: "1.0.#{i}" }).execute }
  3. Navigate to <gdk>/-/graphql_explorer and run the following query:

{
  mlModel(id: "gid://gitlab/Ml::Model/1") {
    versions(version: "1.0", orderBy: VERSION, sort: ASC) {
      nodes {
        version
        id
      }
      pageInfo {
        hasPreviousPage
        hasNextPage
        endCursor
        startCursor
      }
    }
  }
}

Database

Queries

SELECT
    "ml_model_versions".*
FROM
    "ml_model_versions"
WHERE
    "ml_model_versions"."project_id" = 19
    AND "ml_model_versions"."model_id" = 1
    AND (version LIKE '1.0%')
ORDER BY
    "ml_model_versions"."version"

Local execution with 5k model versions:

Index Scan using index_ml_model_versions_on_project_id_and_model_id_and_version on ml_model_versions  (cost=0.28..244.76 rows=5000 width=88) (actual time=0.030..2.075 rows=5000 loops=1)
  Index Cond: ((project_id = 19) AND (model_id = 1) AND (version >= '1.0'::text) AND (version < '1.1'::text))
  Filter: (version ~~ '1.0%'::text)
Planning Time: 0.166 ms
Execution Time: 2.325 ms

Migration

Up

main: == [advisory_lock_connection] object_id: 195780, pg_backend_pid: 52215
main: == 20240119144837 AddIndexToMlModelVersionsOnCreatedAtOnModelId: migrating ====
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.0006s
main: -- index_exists?(:ml_model_versions, [:created_at, :model_id], {:name=>"index_ml_model_versions_on_created_at_on_model_id", :algorithm=>:concurrently})
main:    -> 0.0035s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0002s
main: -- add_index(:ml_model_versions, [:created_at, :model_id], {:name=>"index_ml_model_versions_on_created_at_on_model_id", :algorithm=>:concurrently})
main:    -> 0.0027s
main: -- execute("RESET statement_timeout")
main:    -> 0.0002s
main: == 20240119144837 AddIndexToMlModelVersionsOnCreatedAtOnModelId: migrated (0.0148s)

Down

main: == [advisory_lock_connection] object_id: 183000, pg_backend_pid: 52753
main: == 20240119144837 AddIndexToMlModelVersionsOnCreatedAtOnModelId: reverting ====
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.0116s
main: -- indexes(:ml_model_versions)
main:    -> 0.0036s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0002s
main: -- remove_index(:ml_model_versions, {:algorithm=>:concurrently, :name=>"index_ml_model_versions_on_created_at_on_model_id"})
main:    -> 0.0013s
main: -- execute("RESET statement_timeout")
main:    -> 0.0002s
main: == 20240119144837 AddIndexToMlModelVersionsOnCreatedAtOnModelId: reverted (0.0293s)

main: == [advisory_lock_connection] object_id: 183000, pg_backend_pid: 52753
Edited by Alper Akgun

Merge request reports