feat(duo_workflows): add dropReason dimension and filter
What does this MR do and why?
Adds a dropReason dimension and filter to the Duo workflows data source.
gitlab-org/gitlab#631550 asks why about 33,000 Developer flow sessions on the DAP Impact dashboard show as "Failed" with no reason. gitlab!256455 (merged) records the reason as text with a fixed vocabulary, and gitlab!260703 (merged) exposes it in the aggregation engine. This MR lets GLQL compile queries against it.
Changes:
- Adds
DropReasonto theFieldenum insrc/types/field.rs. It is parsed fromdropreasoncase-insensitively and displayed asdropReason. - In
src/analyzer/sources/duo_workflows.rs:- Adds a
DROP_REASONSconstant with the seven names master stores (job_failed,no_status_reported,no_activity,workload_not_created,api_reported,preconditions_not_met,start_failed). - Adds
drop_reason_enum(), built exactly likeflow_status_enum(). - Adds the field to the valid filter fields, sort fields and dimensions.
- Uses a negatable scalar-or-list field type, the filter key
NegatableFieldNameandlowercase_string_list_filter_value. So!=andnot incompile todropReasonNot.
- Adds a
- The enum is closed on purpose. The engine silently ignores unknown reason names, so a typo would compile to a narrower filter instead of an error. This is the same reasoning as
status. src/schema/schema.jsonis regenerated withcargo run --bin generate-schemaand prettier.
Example:
query: type = DuoWorkflow and status = "failed" and dropReason != "no_activity"
dimensions: dropReason
metrics: totalCountTests: three tests were added first in tests/duo_workflows_tests.rs. I watched them fail with "dropReason is not a valid dimension for Duo workflows".
- The filter translates to the backend key (
dropReason: ["job_failed"],dropReasonNot: ["no_activity"]). dropReasonis a dimension.- Unknown values are rejected.
cargo test, cargo fmt --check and cargo clippy --all-targets are clean.
This depends on the engine side in gitlab-org/gitlab: gitlab!260703 (merged). The dashboard panel and the @gitlab/query-language-rust package bump come in a later MR.
How to set up and validate locally
-
Check out this branch.
-
Run the new tests:
cargo test --test duo_workflows_tests drop_reason -
Run the full suite:
cargo test
Pipeline
graphql-validate validates the compiled queries against the GitLab master schema, so it fails with Unknown argument dropReason until gitlab!260703 (merged) (and its base !260682) reach master. The MR stays a draft until then. archlinux:ruby and ruby-head fail on the same rb_sys / bundler errors as on main and are allowed to fail.
Related to gitlab-org/gitlab#631550
Implementation plan (from the 4-MR plan for #631550)
MR 3 — GLQL compiler field (gitlab-org/glql)
Task 7: Teach the DuoWorkflows source dropReason
Files (in /Users/css/gitlab_code/glql):
- Modify:
src/analyzer/sources/duo_workflows.rs(dimension list, filter mapping, thelowercase_string_list_filter_valuestyle used bystatus) - Test: the
#[case]tables in the same file andsrc/analyzer/analytics.rs - Modify:
CHANGELOG.md
Interfaces:
-
Produces: GLQL field
dropReasonusable inquery:(dropReason = "timeout",dropReason in (...),dropReason != ...) and indimensions:. -
Step 1: Write the failing analyzer cases
Next to the existing status cases, add cases compiling type = DuoWorkflow and dropReason = "ci_failure" to the GraphQL argument dropReason: ["ci_failure"], dropReason != "timeout" to dropReasonNot: ["timeout"], dimensions: dropReason to the dropReason dimension selection, and dropReason = "nope" to a compile error naming the allowed values.
- Step 2: Run the tests
cargo test duo_workflowsExpected: the new cases fail with "unknown field".
- Step 3: Implement
Follow the status field in duo_workflows.rs exactly: a Field entry with the display name dropReason, mapped to engine identifier dropReason, an allowed-value list ["timeout", "reconcile", "ci_failure", "start_failure", "agent_error"], negatable list filter, and dimension support. Add a CHANGELOG entry under Unreleased.
- Step 4: Run the full suite and the GraphQL validation against the monolith schema
cargo test
npm run test:graphql # needs tmp/gitlab_schema.graphql regenerated from MR 2's branchExpected: green.
- Step 5: Commit and open the glql MR
Title feat(duo_workflows): add dropReason dimension and filter. Link MR 2. After it merges and a glql release is cut, bump @gitlab/query-language-rust in the monolith's package.json as part of MR 4.