# Summary Add support for querying CI/Pipeline data in GLQL to enable the Data Analyst Agent to answer questions about pipelines and make CI data more explorable throughout the platform via GLQL embedded views. ## Context From discussion in https://gitlab.com/groups/gitlab-org/-/work_items/20337#note_2986990716: > Adding CI data would not only be a win for Data Analyst (GA) but also make it accessible on work items using GLQL embedded views, thereby making CI data more explorable throughout the entire platform. A POC has validated that pipeline queries can work end-to-end in GLQL. This issue tracks the productionization of that work. ## Implementation Work is broken down into child tasks: 1. **Add a Pipeline source type** — [#98](https://gitlab.com/gitlab-org/glql/-/work_items/98) 2. **Add a Job source type** — [#99](https://gitlab.com/gitlab-org/glql/-/work_items/99) ## Out of Scope (Future Iterations) - **CI aggregation/analytical data** - charts, single stats, and aggregate visualizations - **Additional visualization types** - may be more useful for CI data than tables/lists ## Related - Parent epic: gitlab-org&20337 - Previous performance spike: gitlab!135857 (closed) --- <details> <summary>Original planning notes (archived)</summary> ## Proposed solution Add the `project.pipelines` GraphQL API resource as a new GLQL data source. ### Proposed query Existing `Issue` type query: ``` type = Issue and project = "gitlab-org/gitlab" and state = opened and label = "bug" ``` Proposed `Pipeline` type query: ``` type = Pipeline and project = "gitlab-org/gitlab" and state = FAILED and ref = "main" ``` **Key Differences** | Aspect | Issue | Pipeline | |--------|-------|----------| | State field | state: opened | status: failed (pipeline specific) | | Required fields | id, iid, title, webUrl, reference, state | id, iid, status, path, ref | | URL field | webUrl | path | | Identifier | reference (e.g., #123) | ref (branch name) | | Status type | String (opened, closed) | Enum (failed, success, etc.) | ## POC Findings @jiaan created https://gitlab.com/gitlab-org/glql/-/merge_requests/319+ that demonstrated a basic implementation with the following approach: **Example query:** ``` type = Pipeline and project = "gitlab-org/gitlab" and state = FAILED and ref = "main" ``` **Key design decisions from POC:** - `type = Pipeline` as exclusive source (cannot combine with Issue/MR queries) - `state` in GLQL maps to `status` in GraphQL (uppercase enums: `FAILED`, `SUCCESS`, `RUNNING`, etc.) - Required fields differ from issues: `['id', 'iid', 'status', 'path', 'ref']` - Supports both project and group scope (with `includeSubgroups: true`) - Jobs excluded from initial scope due to nested structure complexity ## Implementation Plan ### Parser/Compiler Changes - Add `Pipelines` variant to `Source` enum (Rust and TypeScript) - Update parser tokens to support pipeline status values (`CREATED`, `WAITING_FOR_RESOURCE`, `PREPARING`, `PENDING`, `RUNNING`, `SUCCESS`, `FAILED`, `CANCELED`, `SKIPPED`, `MANUAL`, `SCHEDULED`) - Update parser tokens to support pipeline source values (`PUSH`, `WEB`, `SCHEDULE`, `API`, `MERGE_REQUEST_EVENT`, etc.) - Change token parsing from `alphanumeric1` to `take_while1(|c| c.is_alphanumeric() || c == '_')` to support underscored tokens ### Field Mapping - Add pipeline-specific filter fields: `state`, `ref`, `sha`, `source`, `username` - Add pipeline date fields: `created`, `updated`, `finished`, `started`, `committed` - Add pipeline display-only fields: `duration`, `queuedDuration`, `coverage`, `name`, `path`, `failureReason` - Implement source-specific required fields function - Implement source-aware field rendering (pipeline `status` is scalar enum vs issue `state` object) ### GraphQL Generation - Generate `pipelines` query with correct field mappings (`state` → `status`, etc.) ### Transformer - Add 'pipelines' to DATA_SOURCES array to enable extraction of pipeline data from GraphQL responses - npm/src/transformer/data.ts - src/transformer/data.rs ### GitLab frontend - The default fields issue (`constants.js` defaults to title) which isn't supported on Pipelines. We can use `status` + `ref` instead (`ref` alone doesn't give much value to users) ### Documentation - Update GLQL documentation with pipeline query syntax and examples - Document supported pipeline fields (filters vs display-only) - Document pipeline status and source enum values ### Tests - Update applicable specs - Ensure both GLQL embedded views and GLQL API function as expected in GitLab/GDK ## Open Questions | Question | Context | Options | |----------|---------|---------| | **Default fields handling** | `DEFAULT_DISPLAY_FIELDS = 'title'` in GitLab frontend doesn't exist on pipelines. For pipelines default to `status` + `ref` | 1) Require explicit `fields:` for pipelines 2) Make defaults source-aware in GitLab 3) Have GLQL substitute defaults per source | | **Uppercase enums** | Pipelines use uppercase enums such as `CANCELED`, `SUCCESS`, etc whereas Issues uses lowercase fields | Do make GLQL Pipelines lowercase or require uppercase enums? | | **Display formatting** | Pipeline data has different display needs | How should status badges and duration be formatted in the transformer? | </details>