feat(duo_workflows): add dropReason dimension and filter

What does this MR do and why?

Adds a dropReason dimension and filter to the Duo workflows data source.

gitlab-org/gitlab#631550 asks why about 33,000 Developer flow sessions on the DAP Impact dashboard show as "Failed" with no reason. gitlab!256455 (merged) records the reason as text with a fixed vocabulary, and gitlab!260703 (merged) exposes it in the aggregation engine. This MR lets GLQL compile queries against it.

Changes:

  • Adds DropReason to the Field enum in src/types/field.rs. It is parsed from dropreason case-insensitively and displayed as dropReason.
  • In src/analyzer/sources/duo_workflows.rs:
    • Adds a DROP_REASONS constant with the seven names master stores (job_failed, no_status_reported, no_activity, workload_not_created, api_reported, preconditions_not_met, start_failed).
    • Adds drop_reason_enum(), built exactly like flow_status_enum().
    • Adds the field to the valid filter fields, sort fields and dimensions.
    • Uses a negatable scalar-or-list field type, the filter key NegatableFieldName and lowercase_string_list_filter_value. So != and not in compile to dropReasonNot.
  • The enum is closed on purpose. The engine silently ignores unknown reason names, so a typo would compile to a narrower filter instead of an error. This is the same reasoning as status.
  • src/schema/schema.json is regenerated with cargo run --bin generate-schema and prettier.

Example:

query: type = DuoWorkflow and status = "failed" and dropReason != "no_activity"
dimensions: dropReason
metrics: totalCount

Tests: three tests were added first in tests/duo_workflows_tests.rs. I watched them fail with "dropReason is not a valid dimension for Duo workflows".

  • The filter translates to the backend key (dropReason: ["job_failed"], dropReasonNot: ["no_activity"]).
  • dropReason is a dimension.
  • Unknown values are rejected.

cargo test, cargo fmt --check and cargo clippy --all-targets are clean.

This depends on the engine side in gitlab-org/gitlab: gitlab!260703 (merged). The dashboard panel and the @gitlab/query-language-rust package bump come in a later MR.

How to set up and validate locally

  1. Check out this branch.

  2. Run the new tests:

    cargo test --test duo_workflows_tests drop_reason
  3. Run the full suite:

    cargo test

Pipeline

graphql-validate validates the compiled queries against the GitLab master schema, so it fails with Unknown argument dropReason until gitlab!260703 (merged) (and its base !260682) reach master. The MR stays a draft until then. archlinux:ruby and ruby-head fail on the same rb_sys / bundler errors as on main and are allowed to fail.

Related to gitlab-org/gitlab#631550

Implementation plan (from the 4-MR plan for #631550)

MR 3 — GLQL compiler field (gitlab-org/glql)

Task 7: Teach the DuoWorkflows source dropReason

Files (in /Users/css/gitlab_code/glql):

  • Modify: src/analyzer/sources/duo_workflows.rs (dimension list, filter mapping, the lowercase_string_list_filter_value style used by status)
  • Test: the #[case] tables in the same file and src/analyzer/analytics.rs
  • Modify: CHANGELOG.md

Interfaces:

  • Produces: GLQL field dropReason usable in query: (dropReason = "timeout", dropReason in (...), dropReason != ...) and in dimensions:.

  • Step 1: Write the failing analyzer cases

Next to the existing status cases, add cases compiling type = DuoWorkflow and dropReason = "ci_failure" to the GraphQL argument dropReason: ["ci_failure"], dropReason != "timeout" to dropReasonNot: ["timeout"], dimensions: dropReason to the dropReason dimension selection, and dropReason = "nope" to a compile error naming the allowed values.

  • Step 2: Run the tests
cargo test duo_workflows

Expected: the new cases fail with "unknown field".

  • Step 3: Implement

Follow the status field in duo_workflows.rs exactly: a Field entry with the display name dropReason, mapped to engine identifier dropReason, an allowed-value list ["timeout", "reconcile", "ci_failure", "start_failure", "agent_error"], negatable list filter, and dimension support. Add a CHANGELOG entry under Unreleased.

  • Step 4: Run the full suite and the GraphQL validation against the monolith schema
cargo test
npm run test:graphql   # needs tmp/gitlab_schema.graphql regenerated from MR 2's branch

Expected: green.

  • Step 5: Commit and open the glql MR

Title feat(duo_workflows): add dropReason dimension and filter. Link MR 2. After it merges and a glql release is cut, bump @gitlab/query-language-rust in the monolith's package.json as part of MR 4.


Edited by Chandra Saripaka

Merge request reports

Loading
Loading