Add GLQL parser support for querying CodeSuggestion analytics using `mode: analytics` syntax. **Epic:** gitlab-org&21212 Add CodeSuggestion aggregations to GLQL **Milestone:** 18.11 **Depends on:** gitlab#589608 (GraphQL mounting - MR !226274) **Related:** #592262 (UI), #592264 (Documentation) ## Example Query ```yaml mode: analytics query: type = CodeSuggestion and timestamp >= -30d dimensions: - language as "Language" - timeSegment(1w) on timestamp as "Week" metrics: - totalCount as "Total Suggestions" - acceptanceRate as "Acceptance Rate" - usersCount as "Active Users" sort: acceptanceRate desc limit: 10 ``` ## Background CodeSuggestions is the **first aggregation-based source type** in GLQL using GitLab's aggregation framework. Unlike standard sources (Issues, MergeRequests), it requires: - Nested GraphQL path: `aiUsage { codeSuggestions { aggregated { } } }` - Two-level structure: filters at outer level, sort/pagination at inner `aggregated` connection - Separation of dimensions (groupable) vs metrics (aggregatable) ### GraphQL Schema Reference Backend: `Analytics::AggregationEngines::CodeSuggestions` ([view engine](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/analytics/aggregation_engines/code_suggestions.rb)) **Filters:** `userId`, `language`, `ideName`, `timestampFrom`, `timestampTo` **Dimensions:** `user`, `language`, `ideName`, `timestamp(granularity)` **Metrics:** `totalCount`, `usersCount`, `acceptanceRate`, `suggestionSizeSum`, `acceptedCount`, `rejectedCount`, `shownCount` ## Implementation Plan ### Key Decisions | Aspect | Decision | Rationale | | -------------------- | ---------------------------------- | ----------------------------------------------------- | | **Field naming** | Exact 1:1 with GraphQL | `acceptanceRate` not `acceptance`, `totalCount` not `count` | | **Timestamp syntax** | `timeSegment(Xd/w/m/q/y)` | Maps to `timestamp(granularity: "...")` | | **Analytics mode** | Explicit `mode: analytics` required | Prevents ambiguity, clear validation | | **User dimension** | Auto-expand to all UserCore fields | `{ id, username, name, avatarUrl, webUrl }` for avatars | | **Nested paths** | Trait-based `parent_scope()` method | Extensible for future AI/analytics sources | | **Sort fields** | Allow dimensions AND metrics | Maximum flexibility | | **Field categorization** | Hardcoded per source analyzer | Follows existing Issues/MergeRequests pattern | ### Critical Gotchas **1. Nested GraphQL Path** First source requiring parent scope nesting. Add `parent_scope() -> Option<&str>` and `source_query_name() -> &str` to `SourceAnalyzer` trait. **2. Aggregated Connection** The `aggregated { }` wrapper is auto-generated by backend framework. GLQL must generate: `codeSuggestions(...filters) { aggregated(...sort) { nodes { dimensions {...} ...metrics } } }` **3. Dimensions vs Metrics Structure** - Dimensions: Go in `dimensions { }` block (for GROUP BY) - Metrics: Go at node level (calculated values) - Add `field_category()` method to trait returning `Dimension | Metric | Standard` **4. User Field Dual Nature** - As filter: `userId: [Int!]` (array) - As dimension: `user { id, username, name, avatarUrl, webUrl }` (object) **5. Filter Mapping** `user` → `userId`, `timestamp >=` → `timestampFrom`, `timestamp <=` → `timestampTo` **6. Analytics Mode Validation** Error cases: - `dimensions`/`metrics` without `mode: analytics` - `mode: analytics` without `dimensions`/`metrics` - `fields` property with `mode: analytics` (incompatible) ### Files Requiring Changes **GLQL Rust Compiler** ([gitlab-org/glql](https://gitlab.com/gitlab-org/glql)): - `src/types/source.rs` - Add `CodeSuggestions` variant - `src/types/field.rs` - Add 11 new fields (Language, IdeName, Timestamp, User, + 7 metrics) - `src/analyzer/sources/mod.rs` - Update trait: add `parent_scope()`, `source_query_name()`, `field_category()` - `src/analyzer/sources/code_suggestions.rs` - **NEW** Create analyzer implementation - `src/codegen/graphql.rs` - Handle nested paths + aggregated wrapper generation - `src/codegen/graphql_filters.rs` - Map filters to GraphQL arguments - `src/transformer/data.rs` - Flatten `dimensions { }`, extract metrics from nodes - `tests/code_suggestion_tests.rs` - **NEW** Comprehensive test coverage **GitLab Rails** ([gitlab-org/gitlab](https://gitlab.com/gitlab-org/gitlab)): - `config/feature_flags/gitlab_com_derisk/glql_code_suggestion_analytics_aggregation.yml` - **NEW** Feature flag - `Gemfile` - Update `gitlab_query_language` gem version - `ee/spec/requests/api/graphql/glql/code_suggestions_spec.rb` - **NEW** Integration tests ## Available Fields Reference ### Filters (Outer Level) | GLQL Field | GraphQL Argument | Type | Notes | | ------------ | ---------------- | --------- | ----------------- | | `user` | `userId` | `[Int!]` | Plural in GraphQL | | `language` | `language` | `[String!]` | Array | | `ideName` | `ideName` | `[String!]` | Array | | `timestamp >=` | `timestampFrom` | `Time` | Range start | | `timestamp <=` | `timestampTo` | `Time` | Range end | ### Dimensions (Groupable) | GLQL Field | GraphQL Field | Type | Parameters | Notes | | ---------- | ------------- | -------- | ----------- | --------------------------------------------------------- | | `user` | `user` | UserCore | - | Auto-expands to `{ id, username, name, avatarUrl, webUrl }` | | `language` | `language` | String | - | Programming language | | `ideName` | `ideName` | String | - | IDE name | | `timestamp` | `timestamp` | Date | `granularity` | Use `timeSegment(Xd/w/m/q/y)` syntax | ### Metrics (Aggregatable) | GLQL Field | GraphQL Field | Type | Description | | ----------------- | ----------------- | ----- | ----------------- | | `totalCount` | `totalCount` | Int | Total suggestions | | `usersCount` | `usersCount` | Int | Unique users | | `acceptanceRate` | `acceptanceRate` | Float | Accepted / shown | | `suggestionSizeSum` | `suggestionSizeSum` | Int | Total volume | | `acceptedCount` | `acceptedCount` | Int | Accepted count | | `rejectedCount` | `rejectedCount` | Int | Rejected count | | `shownCount` | `shownCount` | Int | Shown count | ## Feature Flag **Name:** `glql_code_suggestion_analytics_aggregation` **Type:** `gitlab_com_derisk` **Purpose:** Gates CodeSuggestion analytics functionality **Rollout:** Remove flag in same milestone per epic guidance ## Acceptance Criteria **GLQL Repository** ([gitlab-org/glql](https://gitlab.com/gitlab-org/glql)): - [ ] `CodeSuggestionsSourceAnalyzer` implements trait with `parent_scope()`, `source_query_name()`, `field_category()` - [ ] Code generator creates correct nested GraphQL: `aiUsage { codeSuggestions { aggregated { } } }` - [ ] Dimensions generate `dimensions { }` block, metrics at node level - [ ] Sort validation works for both dimensions and metrics - [ ] Filter mapping handles `user` → `userId`, timestamp ranges - [ ] `timeSegment()` syntax parses and maps to GraphQL granularity - [ ] Analytics mode validation catches all error cases - [ ] Parser tests cover all fields, filters, dimensions, metrics - [ ] Transformer flattens aggregated responses correctly - [ ] No regressions in existing sources **GitLab Repository** ([gitlab-org/gitlab](https://gitlab.com/gitlab-org/gitlab)): - [ ] Feature flag `glql_code_suggestion_analytics_aggregation` created (type: `gitlab_com_derisk`) - [ ] Update `gitlab_query_language` gem dependency - [ ] Integration tests verify end-to-end query execution - [ ] Feature flag gates functionality correctly ## Notes for Related Issues **For #592262 (UI Integration):** - User dimension auto-expands to `{ id, username, name, avatarUrl, webUrl }` for avatar rendering - Exact field names: `acceptanceRate`, `usersCount`, `totalCount`, etc. - Available dimensions: `user`, `language`, `ideName`, `timestamp` - Available metrics: `totalCount`, `usersCount`, `acceptanceRate`, `suggestionSizeSum`, `acceptedCount`, `rejectedCount`, `shownCount` **For #592264 (Documentation):** - Document `timeSegment(Xd/w/m/q/y)` syntax for timestamp granularity - Document analytics mode validation rules - Provide example queries for common use cases - Document feature flag requirement - Document exact field naming conventions ## Example Queries for Testing **Basic Analytics:** ```yaml type: CodeSuggestion mode: analytics query: timestamp >= -30d dimensions: - language as "Language" metrics: - totalCount as "Total" - acceptanceRate as "Rate" sort: totalCount desc limit: 10 ``` **Time Bucketing:** ```yaml type: CodeSuggestion mode: analytics query: timestamp >= -90d dimensions: - timeSegment(1w) on timestamp as "Week" - language as "Language" metrics: - totalCount as "Total" - usersCount as "Users" sort: timestamp asc ``` **Filter + Group:** ```yaml type: CodeSuggestion mode: analytics query: timestamp >= -30d and ideName = "vscode" and language = "ruby" dimensions: - user as "Developer" metrics: - totalCount as "Total" - acceptedCount as "Accepted" - acceptanceRate as "Rate" sort: acceptanceRate desc limit: 20 ```