Accept the MLflow list form of order_by on the search endpoints

What does this MR do and why?

The MLflow runs/search, experiments/search and registered-models/search endpoints now accept order_by as a list of strings, the form the MLflow REST API defines and the MLflow Python client sends. The endpoints declared order_by as a String, so every ordered search from the client returned 400 order_by is invalid before the finder ran. A list orders by its first entry; the list handling is behind the mlflow_search_order_by_list feature flag (default off).

Design notes
  • lib/api/ml/mlflow/runs.rb, experiments.rb and registered_models.rb declare order_by as types: [String, Array[String]]; Grape still coerces scalars (5, true) to a String as on master. The API::API before hook coerce_nil_params_to_array! (lib/api/helpers/common_helpers.rb) turns a null value of a param whose documented type contains [...] into [] (the form-encoded request specs send null), so first_order_by_entry returns the declared default for an empty list on every flag state: null keeps the default it had on master, and [] (400 on master) gets the default too, as the MLflow server does. A request without the key gets the default as before, and a plain "" string behaves as on master. grape-swagger renders the param as oneOf string/array in doc/api/openapi/openapi_v3.yaml; some client generators type such a param as AnyType.
  • API::Ml::Mlflow::ApiHelpers#first_order_by_entry resolves a list to one string before the existing candidates_order_params and model_order_params parse it. The finders (Projects::Ml::CandidateFinder, Projects::Ml::ExperimentFinder) order by one column or metric and break ties on id, so only the first entry can apply. A list with an empty entry or an unsupported column returns 400 INVALID_PARAMETER_VALUE through invalid_parameter!; a plain string keeps the current fallback to the default order, so existing clients see no change.
  • Supported list columns are compared case-insensitively. Runs: CandidateFinder::VALID_ORDER_BY_COLUMNS plus metrics.<name> (metric names keep their case), with the MLflow names start_time and run_name mapped to created_at and name and an optional attributes. prefix. Experiments and registered models: name, created_at, updated_at, last_updated_timestamp, id. The direction keeps the existing fallback to the finder default.
  • Flag: mlflow_search_order_by_list (gitlab_com_derisk, default off). With the flag off a list returns 400 INVALID_PARAMETER_VALUE with the same message as today. Rollback is disabling the flag; string requests are not affected by the flag. No changelog trailer because the change is behind a default-off flag.
  • Docs: a "Searching experiments, runs, and models" section on the MLflow client page with the history entry and flag note, following the mlflow_bounded_list_responses pattern on the same page.
  • doc/api/openapi/openapi_v3.yaml is regenerated (rake gitlab:openapi:v3:generate); only the three order_by schemas change.
  • No query changes: the list path reuses the queries the string path already runs. The metric order uses the LATERAL query from !263900 (merged) (ml_candidate_metric_order_lateral is enabled).

References

Screenshots or screen recordings

No UI change.

How to set up and validate locally

Fixture: a project where you are a developer, with a personal access token (api scope). <project_id> is the project ID shown under Settings > General.

  1. Create an experiment with two runs and a metric (requires pip install mlflow):

    export MLFLOW_TRACKING_URI="http://gdk.test:3000/api/v4/projects/<project_id>/ml/mlflow"
    export MLFLOW_TRACKING_TOKEN="<token>"
    python3 - <<'PY'
    import mlflow
    mlflow.set_experiment('order-by')
    for value in (0.2, 0.9):
        with mlflow.start_run():
            mlflow.log_metric('accuracy', value)
    PY
  2. Search with the MLflow client, which sends order_by as a list:

    python3 -c "import mlflow; print(mlflow.search_runs(experiment_names=['order-by'], order_by=['metrics.accuracy DESC'])[['run_id', 'metrics.accuracy']])"

    Expect on master: MlflowException: INVALID_PARAMETER_VALUE: order_by is invalid.

  3. On this branch, enable the flag in a Rails console: Feature.enable(:mlflow_search_order_by_list).

  4. Repeat step 2. Expect: the two runs, 0.9 first.

  5. http://gdk.test:3000/<group>/<project>/-/ml/experiments still lists the experiment and its runs.

Edge cases

Replace <project_id> and <token>; <experiment_id> is the number in the experiment page URL.

search() {
  curl -s -X POST -H "Authorization: Bearer <token>" -H "Content-Type: application/json" \
    "http://gdk.test:3000/api/v4/projects/<project_id>/ml/mlflow/api/2.0/mlflow/runs/search" -d "$1"; echo
}
search '{"experiment_ids":["<experiment_id>"],"order_by":["name ASC"]}'          # 200, runs by name
search '{"experiment_ids":["<experiment_id>"],"order_by":"name ASC"}'            # 200, unchanged string form
search '{"experiment_ids":["<experiment_id>"],"order_by":[]}'                    # 200, default order (created_at DESC)
search '{"experiment_ids":["<experiment_id>"],"order_by":[""]}'                  # 400 order_by entries must not be empty
search '{"experiment_ids":["<experiment_id>"],"order_by":["params.x"]}'          # 400 Unsupported order_by column: params.x
search '{"experiment_ids":["<experiment_id>"],"max_results":1,"order_by":["metrics.accuracy DESC"]}'  # pass the next_page_token back with the same order_by: second run

search '{"experiment_ids":["<experiment_id>"],"order_by":["attributes.start_time DESC"]}' (the MLflow UI default) orders like created_at DESC; run_name orders like name. experiments/search and registered-models/search (GET) accept the same shapes with name, created_at, updated_at, id and last_updated_timestamp as columns. With Feature.disable(:mlflow_search_order_by_list) every non-empty list returns 400 order_by is invalid, as before; an empty list takes the default order.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Eduardo Bonet

Merge request reports

Loading
Loading