Limit ML candidate metrics, params and metadata to one candidate
What does this MR do and why?
The metrics, params and metadata fields of MlCandidate now resolve for one candidate per GraphQL operation, behind the limit_ml_candidate_graphql_fields feature flag (gitlab_com_derisk, default off, project actor). Each of these connections runs its own query for every candidate, and their complexity does not grow with the page size, so one query through mlExperiment.candidates or mlModel.versions runs three queries per candidate it returns. The limit is Gitlab::Graphql::Limit::FieldCallCount with limit: 1, as on MlModel.versions; rollback is to disable the flag.
Design notes
- Why the child fields and not
candidates: a limit onMlExperiment.candidatesorMlModel.candidatesmissesmlModel { versions(first: 100) { nodes { candidate { metrics { ... } } } } }(theversionslimit is per model, so each version's candidate runs its own three queries). It also breaks API clients that list candidates without these three fields. - The flag check is in
lib/gitlab/graphql/limit/ml_candidate_field_call_count.rb, a subclass ofFieldCallCountthat skips the count when the flag is off for the candidate's project. The actor comes fromProject.actor_from_id(candidate.project_id), so the check loads no record. After the rollout, the three fields useFieldCallCountdirectly and the subclass is deleted (cleanup step of the rollout issue). - Response over the limit:
FieldCallCountraisesLimitErrorwith"metrics" field can be requested only for 1 MlCandidate(s) at a time.. The three fields are non-null, so the error sets the nearest nullable parent tonull. Throughcandidates, that is the candidate node. Throughversions,MlModelVersion.candidateis also non-null, so the whole version node becomesnull, includingversion; the same applies tomlModel.version(...)andlatestVersionwhen their candidate is over the limit. The first candidate resolves as before. graphql-ruby reports the first failing field of each node. - The field descriptions name the flag and the limit, as the GraphQL style guide asks for feature-flagged behavior and for
FieldCallCount. The GraphQL reference and the introspection files are regenerated. - Query count: with the flag on, a query reads
ml_candidate_metrics,ml_candidate_paramsandml_candidate_metadataonce each, whatever the number of candidates. The spec inspec/requests/api/graphql/ml/experiments_spec.rbcounts the queries per table. - Follow-ups, not in this MR:
- With the flag on, the other candidate fields still cost work per candidate.
creator,ciJoband_links { showPath }look up records for each candidate (several uncached queries per candidate in the test environment), so a query that lists many candidates with these fields is still not bounded by one page. Batch those presenter lookups, or bound the nestedcandidatesconnection. - Batch the
metrics,paramsandmetadataconnections across candidates, which could let the limit go later.
- With the flag on, the other candidate fields still cost work per candidate.
Breaking change
A query that selects one of these fields for more than one candidate returns an error with the flag on, and loses data for every candidate after the first. API clients that list model versions with candidate metrics, params or metadata lose whole version entries, not only the candidate fields. The GraphQL style guide counts "anything else that can result in queries hitting a limit that previously was allowed" as a breaking change. The documented exemption for performance concerns applies instead of the deprecation process: "GitLab might make immediate breaking changes to the GraphQL API to patch critical security or performance concerns if the deprecation process would pose significant risk" (Breaking change exemptions). The deprecation process would keep the per-candidate cost until the next major release. The flag stays off on GitLab.com until the change has approval through the Breaking Change Exception process; the request is https://gitlab.com/gitlab-com/Product/-/work_items/14590, and the rollout issue tracks it. The flag gives a rollback.
The limit counts calls of each field, not candidates (existing FieldCallCount behavior): one candidate with m1: metrics(first: 1) m2: metrics(first: 2) also fails on m2.
The UI is not affected. Only getModel and getModelVersion (model and model version detail pages) select these fields, for one candidate each, and the new specs run both query files. No other frontend query and no internal GraphQL caller selects them. GitLab.com GraphQL logs show a small number of API requests that select them for several candidates. API clients that select them for several candidates in one query need one query per candidate.
References
- Related to https://gitlab.com/gitlab-org/gitlab/-/work_items/635043
- Feature flag rollout: https://gitlab.com/gitlab-org/gitlab/-/work_items/635197
- Same mechanism:
MlModel.versions(app/graphql/resolvers/ml/find_model_versions_resolver.rb) - No sibling MR edits these files.
Screenshots or screen recordings
No UI change.
How to set up and validate locally
-
In a Rails console, enable the flag:
Feature.enable(:limit_ml_candidate_graphql_fields). -
In any project, go to Deploy > Model registry, select Create model, and create a model.
-
On the model page, select Create new version and create version
1.0.0. Repeat for version1.0.1. -
Open the model page, then the page of version
1.0.0. Expect: both pages load as onmaster, with no error message. -
Open
http://gdk.test:3000/-/graphql-explorerand run the query below with the model ID from the page URL. Expect:versions.nodeshas one version with itsversionand its candidate'smetrics, and onenullentry in place of the other version.errorshas"metrics" field can be requested only for 1 MlCandidate(s) at a time.query { mlModel(id: "gid://gitlab/Ml::Model/<model_id>") { versions { nodes { version candidate { id metrics { nodes { name value } } } } } } } -
Run
Feature.disable(:limit_ml_candidate_graphql_fields)and run the query again. Expect: both versions with their candidates, no error.
More scenarios
- Experiments, flag on:
mlExperiment(id: "gid://gitlab/Ml::Experiment/<id>") { candidates { nodes { id params { nodes { name } } } } }on an experiment with two or more candidates returns the params of one candidate,nullfor the others, and the"params"limit error. metadatabehaves the same.- A query that selects
metrics,paramsandmetadatafor one candidate, for examplemlModel { latestVersion { candidate { ... } } }, returns no error.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Database review
Not needed: the change adds and changes no query, index or migration. It stops queries from running for the second and later candidates.