Accept the MLflow list form of order_by on the search endpoints
What does this MR do and why?
The MLflow runs/search, experiments/search and registered-models/search endpoints now accept order_by as a list of strings, the form the MLflow REST API defines and the MLflow Python client sends. The endpoints declared order_by as a String, so every ordered search from the client returned 400 order_by is invalid before the finder ran. A list orders by its first entry; the list handling is behind the mlflow_search_order_by_list feature flag (default off).
Design notes
lib/api/ml/mlflow/runs.rb,experiments.rbandregistered_models.rbdeclareorder_byastypes: [String, Array[String]]; Grape still coerces scalars (5,true) to a String as on master. TheAPI::APIbefore hookcoerce_nil_params_to_array!(lib/api/helpers/common_helpers.rb) turns anullvalue of a param whose documented type contains[...]into[](the form-encoded request specs sendnull), sofirst_order_by_entryreturns the declared default for an empty list on every flag state:nullkeeps the default it had on master, and[](400 on master) gets the default too, as the MLflow server does. A request without the key gets the default as before, and a plain""string behaves as on master. grape-swagger renders the param asoneOfstring/array indoc/api/openapi/openapi_v3.yaml; some client generators type such a param asAnyType.API::Ml::Mlflow::ApiHelpers#first_order_by_entryresolves a list to one string before the existingcandidates_order_paramsandmodel_order_paramsparse it. The finders (Projects::Ml::CandidateFinder,Projects::Ml::ExperimentFinder) order by one column or metric and break ties onid, so only the first entry can apply. A list with an empty entry or an unsupported column returns400 INVALID_PARAMETER_VALUEthroughinvalid_parameter!; a plain string keeps the current fallback to the default order, so existing clients see no change.- Supported list columns are compared case-insensitively. Runs:
CandidateFinder::VALID_ORDER_BY_COLUMNSplusmetrics.<name>(metric names keep their case), with the MLflow namesstart_timeandrun_namemapped tocreated_atandnameand an optionalattributes.prefix. Experiments and registered models:name,created_at,updated_at,last_updated_timestamp,id. The direction keeps the existing fallback to the finder default. - Flag:
mlflow_search_order_by_list(gitlab_com_derisk, default off). With the flag off a list returns400 INVALID_PARAMETER_VALUEwith the same message as today. Rollback is disabling the flag; string requests are not affected by the flag. No changelog trailer because the change is behind a default-off flag. - Docs: a "Searching experiments, runs, and models" section on the MLflow client page with the history entry and flag note, following the
mlflow_bounded_list_responsespattern on the same page. doc/api/openapi/openapi_v3.yamlis regenerated (rake gitlab:openapi:v3:generate); only the threeorder_byschemas change.- No query changes: the list path reuses the queries the string path already runs. The metric order uses the
LATERALquery from !263900 (merged) (ml_candidate_metric_order_lateralis enabled).
References
- Related to https://gitlab.com/gitlab-org/gitlab/-/work_items/638606
- Rollout issue: https://gitlab.com/gitlab-org/gitlab/-/work_items/638609
- Sibling MR !265948 (merged) adds
track_mlflow_eventcalls to the same two endpoint files after the search results load. This MR only changes theparamsblocks andapi_helpers.rb, so the two apply in either order. - MLflow REST API: https://mlflow.org/docs/latest/api_reference/rest-api.html#search-runs (
order_by: "List of columns to be ordered by")
Screenshots or screen recordings
No UI change.
How to set up and validate locally
Fixture: a project where you are a developer, with a personal access token (api scope). <project_id> is the project ID shown under Settings > General.
-
Create an experiment with two runs and a metric (requires
pip install mlflow):export MLFLOW_TRACKING_URI="http://gdk.test:3000/api/v4/projects/<project_id>/ml/mlflow" export MLFLOW_TRACKING_TOKEN="<token>" python3 - <<'PY' import mlflow mlflow.set_experiment('order-by') for value in (0.2, 0.9): with mlflow.start_run(): mlflow.log_metric('accuracy', value) PY -
Search with the MLflow client, which sends
order_byas a list:python3 -c "import mlflow; print(mlflow.search_runs(experiment_names=['order-by'], order_by=['metrics.accuracy DESC'])[['run_id', 'metrics.accuracy']])"Expect on
master:MlflowException: INVALID_PARAMETER_VALUE: order_by is invalid. -
On this branch, enable the flag in a Rails console:
Feature.enable(:mlflow_search_order_by_list). -
Repeat step 2. Expect: the two runs,
0.9first. -
http://gdk.test:3000/<group>/<project>/-/ml/experimentsstill lists the experiment and its runs.
Edge cases
Replace <project_id> and <token>; <experiment_id> is the number in the experiment page URL.
search() {
curl -s -X POST -H "Authorization: Bearer <token>" -H "Content-Type: application/json" \
"http://gdk.test:3000/api/v4/projects/<project_id>/ml/mlflow/api/2.0/mlflow/runs/search" -d "$1"; echo
}
search '{"experiment_ids":["<experiment_id>"],"order_by":["name ASC"]}' # 200, runs by name
search '{"experiment_ids":["<experiment_id>"],"order_by":"name ASC"}' # 200, unchanged string form
search '{"experiment_ids":["<experiment_id>"],"order_by":[]}' # 200, default order (created_at DESC)
search '{"experiment_ids":["<experiment_id>"],"order_by":[""]}' # 400 order_by entries must not be empty
search '{"experiment_ids":["<experiment_id>"],"order_by":["params.x"]}' # 400 Unsupported order_by column: params.x
search '{"experiment_ids":["<experiment_id>"],"max_results":1,"order_by":["metrics.accuracy DESC"]}' # pass the next_page_token back with the same order_by: second runsearch '{"experiment_ids":["<experiment_id>"],"order_by":["attributes.start_time DESC"]}' (the MLflow UI default) orders like created_at DESC; run_name orders like name. experiments/search and registered-models/search (GET) accept the same shapes with name, created_at, updated_at, id and last_updated_timestamp as columns. With Feature.disable(:mlflow_search_order_by_list) every non-empty list returns 400 order_by is invalid, as before; an empty list takes the default order.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.