Skip to content

ModelsController now displays Ml::Model

Eduardo Bonet requested to merge 385061-model-registry-ui-model-list into master

What does this MR do and why?

Uses ModelVersion for ModelsController

In the first iteration for Ml::ModelsController we used Packages::Package of type ml_model as the base for the model. We have since added Ml::Model and Ml::ModelVersion to split domain logic, and this change makes ModelsController handle those classes instead of Packages::Package

Follow up updates the presenter, controller and views: !127130 (merged)

How to set up and validate locally

  1. Enable the feature flag

    echo "Feature.enable(:model_registry)" | bundle exec rails c
  2. Create data on rails console

    p = Project.find_by_id(19) # or the id of your project
    models = Array.new(10) { |i| Ml::Model.create!(name: "model_#{i}", project: p, default_experiment: Ml::Experiment.create!(name: "model_#{i}", project: p)) }
    models.take(5).each { |m| 2.times {|i| m.versions.create!(project: p, version: "1.0.#{i}") } }
    mv = models[0].latest_version
    package = Packages::Package.create!(name: mv.name, version: mv.version, project: p, packaget_type: :ml_model)
    mv.update!(package_id = package.id)
  3. Navigate to <your_project>/-/ml/models. Note that some of the of the versions are not showing (they don't exist) or links don't work (packages do not exist). UI improvements are not in the scope of this MR. image

Database

by_project
SELECT
    "ml_models".*
FROM
    "ml_models"
WHERE
    "ml_models"."project_id" = 1

https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/20530/commands/67298

Production has no data, here's it running locally:

Index Scan using index_ml_models_on_project_id_and_name on ml_models  (cost=0.15..5.22 rows=4 width=64) (actual time=0.024..0.028 rows=10 loops=1)
  Index Cond: (project_id = 19)
Planning Time: 0.108 ms
Execution Time: 0.050 ms
including_latest_version
> Ml::Model.including_latest_version.take(5)
  Ml::Model Load (0.8ms)  SELECT "ml_models".* FROM "ml_models" LIMIT 5 /*application:console,db_config_name:main,console_hostname:Eduardos-MacBook-Pro-2.local,console_username:eduardobonet,line:(pry):1:in `__pry__'*/
  Ml::ModelVersion Load (0.7ms)  SELECT DISTINCT ON (model_id) * FROM "ml_model_versions" WHERE "ml_model_versions"."model_id" IN (1, 2, 3, 4, 5) ORDER BY model_id, id DESC /*application:console,db_config_name:main,console_hostname:Eduardos-MacBook-Pro-2.local,console_username:eduardobonet,line:(pry):1:in `__pry__'*/
SELECT DISTINCT ON (model_id)
    *
FROM
    "ml_model_versions"
WHERE
    "ml_model_versions"."model_id" IN (1, 2, 3, 4, 5)
ORDER BY
    model_id,
    id DESC

production has no data, local output of explain analyse (10k rows on Ml::ModelVersions):

Unique  (cost=0.29..132.06 rows=11 width=55) (actual time=0.026..0.657 rows=4 loops=1)
  ->  Index Scan using unique_ml_model_versions_on_model_id_and_id on ml_model_versions  (cost=0.29..124.54 rows=3008 width=55) (actual time=0.026..0.476 rows=3008 loops=1)
"        Index Cond: (model_id = ANY ('{1,2,3,4,5}'::bigint[]))"
Planning Time: 0.283 ms
Execution Time: 0.667 ms
Migration

Up:

main: == [advisory_lock_connection] object_id: 220080, pg_backend_pid: 11451
main: == 20230720142334 IndexMlModelVersionsOnModelIdAndId: migrating ===============
main: -- transaction_open?()
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.0098s
main: -- index_exists?(:ml_model_versions, [:model_id, :id], {:name=>"unique_ml_model_versions_on_model_id_and_id", :order=>{:id=>:desc}, :algorithm=>:concurrently})
main:    -> 0.0029s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0003s
main: -- add_index(:ml_model_versions, [:model_id, :id], {:name=>"unique_ml_model_versions_on_model_id_and_id", :order=>{:id=>:desc}, :algorithm=>:concurrently})
main:    -> 0.0017s
main: -- execute("RESET statement_timeout")
main:    -> 0.0003s
main: == 20230720142334 IndexMlModelVersionsOnModelIdAndId: migrated (0.0296s) ======

main: == [advisory_lock_connection] object_id: 220080, pg_backend_pid: 11451

main: == [advisory_lock_connection] object_id: 220620, pg_backend_pid: 18398
main: == 20230725210728 DropIndexMlModelVersionsOnModelId: migrating ================
main: -- transaction_open?()
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.0006s
main: -- indexes(:ml_model_versions)
main:    -> 0.0057s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0003s
main: -- remove_index(:ml_model_versions, {:algorithm=>:concurrently, :name=>"index_ml_model_versions_on_model_id"})
main:    -> 0.0017s
main: -- execute("RESET statement_timeout")
main:    -> 0.0006s
main: == 20230725210728 DropIndexMlModelVersionsOnModelId: migrated (0.0199s) =======

Down:


main: == [advisory_lock_connection] object_id: 223900, pg_backend_pid: 19734
main: == 20230725210728 DropIndexMlModelVersionsOnModelId: reverting ================
main: -- transaction_open?()
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.1307s
main: -- index_exists?(:ml_model_versions, :model_id, {:name=>"index_ml_model_versions_on_model_id", :algorithm=>:concurrently})
main:    -> 0.0085s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0004s
main: -- add_index(:ml_model_versions, :model_id, {:name=>"index_ml_model_versions_on_model_id", :algorithm=>:concurrently})
main:    -> 0.0110s
main: -- execute("RESET statement_timeout")
main:    -> 0.0004s
main: == 20230725210728 DropIndexMlModelVersionsOnModelId: reverted (0.1681s) =======

main: == [advisory_lock_connection] object_id: 223900, pg_backend_pid: 197

main: == [advisory_lock_connection] object_id: 223040, pg_backend_pid: 14312
main: == 20230720142334 IndexMlModelVersionsOnModelIdAndId: reverting ===============
main: -- transaction_open?()
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.1086s
main: -- indexes(:ml_model_versions)
main:    -> 0.0039s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0003s
main: -- remove_index(:ml_model_versions, {:algorithm=>:concurrently, :name=>"unique_ml_model_versions_on_model_id_and_id"})
main:    -> 0.0021s
main: -- execute("RESET statement_timeout")
main:    -> 0.0003s
main: == 20230720142334 IndexMlModelVersionsOnModelIdAndId: reverted (0.1301s) ====

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #385061 (closed)

Edited by Eduardo Bonet

Merge request reports