Skip to content

Resolve "Create ML Experiment Tracking Data Model"

Eduardo Bonet requested to merge 370479-create-ml-experiment-tracking-data-model into master

What does this MR do and why?

Adds the minimal data model for ML Experiment Tracking. These will serve a simple CRUD:

POST/GET <my-project>/ml/experiments
POST/GET/PUT/DELETE <my-project>/ml/experiments/{experiment_iid}
POST/GET/PUT/DELETE <my-project>/ml/experiments/{experiment_iid}/candidates/{candidate_iid}
POST/DELETE <my-project>/ml/experiments/{experiment_iid}/candidates/{candidate_iid}/metric
POST/DELETE <my-project>/ml/experiments/{experiment_iid}/candidates/{candidate_iid}/param

Experiment:

{ 
   'project_id': int
   'experiment_iid': int
   'name': string
}

Candidates:

{
   experiment_iid: int
   candidate_iid: uuid
   metrics: [
      'metric1_name=(string)': 'metric1_value=(float)',
      'metric2_name=(string)': 'metric1_value=(float)'
   ],
   params: [
      'param1_name=(string)': 'param1_value=(float)',
      'param2_name=(string)': 'param2_value=(float)'
   ]
}

Tables Risk

  • What is the anticipated growth for the new table over the next 3 months, 6 months, 1 year? What assumptions are these based on?
  • How many reads and writes per hour would you expect this table to have in 3 months, 6 months, 1 year? Under what circumstances are rows updated? What assumptions are these based on?
  • Based on the anticipated data volume and access patterns, does the new table pose an availability risk to GitLab.com or self-managed instances? Does the proposed design scale to support the needs of GitLab.com and self-managed customers?

We expect data access on these tables to be very low in the beginning, since we are still incubating this feature.

Command outputs

###rails db:migrate

main: == 20220811092243 CreateMlExperiments: migrating ==============================
main: -- create_table(:ml_experiments, {})
main: -- quote_column_name(:name)
main:    -> 0.0000s
main:    -> 0.0355s
main: == 20220811092243 CreateMlExperiments: migrated (0.0374s) =====================

main: == 20220811092244 CreateMlCandidates: migrating ===============================
main: -- create_table(:ml_candidates, {})
main:    -> 0.0192s
main: == 20220811092244 CreateMlCandidates: migrated (0.0195s) ======================

main: == 20220811092245 CreateMlCandidateParams: migrating ==========================
main: -- create_table(:ml_candidate_params, {})
main: -- quote_column_name(:name)
main:    -> 0.0000s
main: -- quote_column_name(:value)
main:    -> 0.0000s
main:    -> 0.0087s
main: == 20220811092245 CreateMlCandidateParams: migrated (0.0088s) =================

main: == 20220811092246 CreateMlCandidateMetrics: migrating =========================
main: -- create_table(:ml_candidate_metrics, {})
main: -- quote_column_name(:name)
main:    -> 0.0000s
main:    -> 0.0075s
main: == 20220811092246 CreateMlCandidateMetrics: migrated (0.0076s) ================

main: == 20220811092251 AddMlCandidatesReferenceToExperiment: migrating =============
main: -- transaction_open?()
main:    -> 0.0000s
main: -- foreign_keys(:ml_candidates)
main:    -> 0.0079s
main: -- transaction_open?()
main:    -> 0.0000s
main: -- execute("ALTER TABLE ml_candidates\nADD CONSTRAINT fk_56d6ed4d3d\nFOREIGN KEY (experiment_id)\nREFERENCES ml_experiments (id)\nON DELETE CASCADE\nNOT VALID;\n")
main:    -> 0.0011s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0004s
main: -- execute("ALTER TABLE ml_candidates VALIDATE CONSTRAINT fk_56d6ed4d3d;")
main:    -> 0.0015s
main: -- execute("RESET statement_timeout")
main:    -> 0.0004s
main: == 20220811092251 AddMlCandidatesReferenceToExperiment: migrated (0.0228s) ====

main: == 20220811092253 AddMlExperimentsReferenceToProject: migrating ===============
main: -- transaction_open?()
main:    -> 0.0000s
main: -- foreign_keys(:ml_experiments)
main:    -> 0.0037s
main: -- transaction_open?()
main:    -> 0.0000s
main: -- execute("ALTER TABLE ml_experiments\nADD CONSTRAINT fk_ad89c59858\nFOREIGN KEY (project_id)\nREFERENCES projects (id)\nON DELETE CASCADE\nNOT VALID;\n")
main:    -> 0.0019s
main: -- execute("ALTER TABLE ml_experiments VALIDATE CONSTRAINT fk_ad89c59858;")
main:    -> 0.0142s
main: == 20220811092253 AddMlExperimentsReferenceToProject: migrated (0.0238s) ======

ci: == 20220811092243 CreateMlExperiments: migrating ==============================
ci: -- create_table(:ml_experiments, {})
ci: -- quote_column_name(:name)
ci:    -> 0.0000s
ci:    -> 0.0136s
ci: == 20220811092243 CreateMlExperiments: migrated (0.0137s) =====================

ci: == 20220811092244 CreateMlCandidates: migrating ===============================
ci: -- create_table(:ml_candidates, {})
ci:    -> 0.0075s
ci: == 20220811092244 CreateMlCandidates: migrated (0.0076s) ======================

ci: == 20220811092245 CreateMlCandidateParams: migrating ==========================
ci: -- create_table(:ml_candidate_params, {})
ci: -- quote_column_name(:name)
ci:    -> 0.0000s
ci: -- quote_column_name(:value)
ci:    -> 0.0000s
ci:    -> 0.0068s
ci: == 20220811092245 CreateMlCandidateParams: migrated (0.0069s) =================

ci: == 20220811092246 CreateMlCandidateMetrics: migrating =========================
ci: -- create_table(:ml_candidate_metrics, {})
ci: -- quote_column_name(:name)
ci:    -> 0.0000s
ci:    -> 0.0061s
ci: == 20220811092246 CreateMlCandidateMetrics: migrated (0.0062s) ================

ci: == 20220811092251 AddMlCandidatesReferenceToExperiment: migrating =============
ci: -- transaction_open?()
ci:    -> 0.0000s
ci: -- foreign_keys(:ml_candidates)
ci:    -> 0.0057s
ci: -- transaction_open?()
ci:    -> 0.0000s
ci: -- execute("ALTER TABLE ml_candidates\nADD CONSTRAINT fk_56d6ed4d3d\nFOREIGN KEY (experiment_id)\nREFERENCES ml_experiments (id)\nON DELETE CASCADE\nNOT VALID;\n")
ci:    -> 0.0010s
ci: -- execute("SET statement_timeout TO 0")
ci:    -> 0.0004s
ci: -- execute("ALTER TABLE ml_candidates VALIDATE CONSTRAINT fk_56d6ed4d3d;")
ci:    -> 0.0019s
ci: -- execute("RESET statement_timeout")
ci:    -> 0.0003s
ci: == 20220811092251 AddMlCandidatesReferenceToExperiment: migrated (0.0122s) ====

ci: == 20220811092253 AddMlExperimentsReferenceToProject: migrating ===============
ci: -- transaction_open?()
ci:    -> 0.0000s
ci: -- foreign_keys(:ml_experiments)
ci:    -> 0.0030s
ci: -- transaction_open?()
ci:    -> 0.0000s
ci: -- execute("ALTER TABLE ml_experiments\nADD CONSTRAINT fk_ad89c59858\nFOREIGN KEY (project_id)\nREFERENCES projects (id)\nON DELETE CASCADE\nNOT VALID;\n")
ci:    -> 0.0019s
ci: -- execute("ALTER TABLE ml_experiments VALIDATE CONSTRAINT fk_ad89c59858;")
ci:    -> 0.0173s
ci: == 20220811092253 AddMlExperimentsReferenceToProject: migrated (0.0254s) ======

rails db:rollback:main STEP=6 

main: == 20220811092253 AddMlExperimentsReferenceToProject: reverting ===============
main: -- transaction_open?()
main:    -> 0.0000s
main: -- remove_foreign_key(:ml_experiments, {:column=>:project_id})
main:    -> 0.0090s
main: == 20220811092253 AddMlExperimentsReferenceToProject: reverted (0.0229s) ======

main: == 20220811092251 AddMlCandidatesReferenceToExperiment: reverting =============
main: -- transaction_open?()
main:    -> 0.0000s
main: -- remove_foreign_key(:ml_candidates, {:column=>:experiment_id})
main:    -> 0.0041s
main: == 20220811092251 AddMlCandidatesReferenceToExperiment: reverted (0.0062s) ====

main: == 20220811092246 CreateMlCandidateMetrics: reverting =========================
main: -- drop_table(:ml_candidate_metrics, {})
main:    -> 0.0022s
main: == 20220811092246 CreateMlCandidateMetrics: reverted (0.0042s) ================

main: == 20220811092245 CreateMlCandidateParams: reverting ==========================
main: -- drop_table(:ml_candidate_params, {})
main:    -> 0.0017s
main: == 20220811092245 CreateMlCandidateParams: reverted (0.0019s) =================

main: == 20220811092244 CreateMlCandidates: reverting ===============================
main: -- drop_table(:ml_candidates, {})
main:    -> 0.0021s
main: == 20220811092244 CreateMlCandidates: reverted (0.0022s) ======================

main: == 20220811092243 CreateMlExperiments: reverting ==============================
main: -- drop_table(:ml_experiments, {})
main:    -> 0.0026s
main: == 20220811092243 CreateMlExperiments: reverted (0.0027s) =====================

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #370479 (closed)

Edited by Eduardo Bonet

Merge request reports