Resolve "Create UI to compare different Candidates"

What does this MR do and why?

Adds list of experiments and details on the candidates associated to an experiment.

Database

Scope :latest

SELECT DISTINCT ON (candidate_id, name) *
FROM "ml_candidate_metrics"
WHERE "ml_candidate_metrics"."candidate_id" IN (68, 69, 70, 71, 72)
ORDER BY candidate_id, name, id DESC;
Unique  (cost=1.01..1.02 rows=1 width=88) (actual time=0.021..0.026 rows=10 loops=1)
  ->  Sort  (cost=1.01..1.01 rows=1 width=88) (actual time=0.021..0.022 rows=10 loops=1)
"        Sort Key: candidate_id, name, id DESC"
        Sort Method: quicksort  Memory: 26kB
        ->  Seq Scan on ml_candidate_metrics  (cost=0.00..1.00 rows=1 width=88) (actual time=0.011..0.013 rows=10 loops=1)
"              Filter: (candidate_id = ANY ('{68,69,70,71,72}'::bigint[]))"
              Rows Removed by Filter: 27
Planning Time: 0.084 ms
Execution Time: 0.041 ms

Ml::Experiment.py_project_id

SELECT "ml_experiments".* 
FROM "ml_experiments" 
WHERE "ml_experiments"."project_id" = 32 
ORDER BY "ml_experiments"."id" DESC
Sort  (cost=1.01..1.01 rows=1 width=88) (actual time=0.040..0.041 rows=3 loops=1)
  Sort Key: id DESC
  Sort Method: quicksort  Memory: 25kB
  ->  Seq Scan on ml_experiments  (cost=0.00..1.00 rows=1 width=88) (actual time=0.010..0.010 rows=3 loops=1)
        Filter: (project_id = 32)
        Rows Removed by Filter: 4
Planning Time: 0.113 ms
Execution Time: 0.053 ms

Ml::Experiment.py_project_id_and_iid

SELECT "ml_experiments".* FROM "ml_experiments" WHERE "ml_experiments"."project_id" = 32 AND "ml_experiments"."iid" = 1 LIMIT 1 /*application:web,correlation_id:01GFNM5CE3CRA0P52PEDGR5CPF,endpoint_id:Projects::Ml::ExperimentsController#show,db_config_name:main,line:/app/models/ml/experiment.rb:18:in `by_project_id_and_iid'*
Limit  (cost=0.00..1.00 rows=1 width=88) (actual time=0.028..0.028 rows=1 loops=1)
  ->  Seq Scan on ml_experiments  (cost=0.00..1.00 rows=1 width=88) (actual time=0.025..0.025 rows=1 loops=1)
        Filter: ((project_id = 32) AND (iid = 1))
        Rows Removed by Filter: 4
Planning Time: 1.003 ms
Execution Time: 0.052 ms
  • if candidate.params is used:
SELECT "ml_candidate_params".* 
FROM "ml_candidate_params" 
WHERE "ml_candidate_params"."candidate_id" 
IN (68, 69, 70, 71, 72) 
Seq Scan on ml_candidate_params  (cost=0.00..1.29 rows=1 width=43) (actual time=0.020..0.022 rows=5 loops=1)
"  Filter: (candidate_id = ANY ('{68,69,70,71,72}'::bigint[]))"
  Rows Removed by Filter: 30
Planning Time: 0.534 ms
Execution Time: 0.030 ms
  • if candidate.latest_metrics is accessed:
SELECT DISTINCT ON (candidate_id, name) * 
FROM "ml_candidate_metrics" 
WHERE "ml_candidate_metrics"."candidate_id" IN (68, 69, 70, 71, 72) 
ORDER BY candidate_id, name, id DESC /*application:web,correlation_id:01GFNMRCB6TXBFJSS9C73TQ02W,endpoint_id:Projects::Ml::ExperimentsController#show,db_config_name:main,line:/app/models/ml/experiment.rb:18:in `by_project_id_and_iid'*/
Unique  (cost=2.01..2.09 rows=8 width=88)
  ->  Sort  (cost=2.01..2.04 rows=10 width=88)
"        Sort Key: candidate_id, name, id DESC"
        ->  Seq Scan on ml_candidate_metrics  (cost=0.00..1.84 rows=10 width=88)
"              Filter: (candidate_id = ANY ('{68,69,70,71,72}'::bigint[]))"

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Enable the invite modal

    echo "Feature.enable(:ml_experiment_tracking)" | bundle exec rails c
  2. Populate the database. In rails console:

    experiments = 3.times.map { |v| Ml::Experiment.create!(name: "Gitlab Experiment #{v}", user_id: user_id, project_id: project_id) }
    exp = experiments[0]
    5.times.each { |i| exp.candidates.create!(user_id: user_id, start_time: 0) }
    exp.candidates.each_with_index { |c, i| c.metrics.create!(name: "auc", value: i*0.1 , tracked_at: Time.zone.now, step: 1)} 
    exp.candidates.each_with_index { |c, i| c.metrics.create!(name: "accuracy", value: i* 0.1+0.1 , tracked_at: Time.zone.now, step: 1)} 
    exp.candidates.each { |c| c.params.create!(name: "algorithm", value: ["LogisticRegression", "DecisionTree"].sample )} 
  3. Visit http://gdk.test:3000/root/ml-test/-/ml/experiments/, it should show the follow: image

  4. Visit http://gdk.test:3000/root/ml-test/-/ml/experiments/2, or click Gitlab Experiment 1, it should show the follow: image

  5. Visit http://gdk.test:3000/root/ml-test/-/ml/experiments/1, or click Gitlab Experiment 0, it should show the follow: image

  6. Without feature flag enabled, http://gdk.test:3000/root/ml-test/-/ml/experiments/ should give 404

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #370480 (closed)

Edited by Eduardo Bonet

Merge request reports

Loading