Add author_human filter to merge request analytics

What does this MR do and why?

The DAP impact dashboard compares MR cycle time for Duo-created merge requests with everything else. The work item asks for that baseline to hold only human-authored merge requests. Bots, service accounts and other non-human users, such as dependency-update bots, sit in the non-Duo bucket and pull its baseline down. ClickHouse merge_requests only stores author_id, so there was no way to filter them out.

This MR adds:

  1. non_human_users_dict: a ClickHouse dictionary of user IDs whose latest siphon_users row has a user_type other than human (0). It follows banned_users_dict and refreshes every 5 to 60 minutes. Because it uses user_type != 0, new user types are covered without a migration.
  2. author_human filter on Analytics::AggregationEngines::MergeRequests, exposed in GraphQL as authorHuman. It is true when the merge request has an author who is not in the dictionary. Merge requests with no author count as not human.
  3. Docs: the authorHuman GLQL filter, and how a merge request is classified as created by Duo. That is the case when an Agent Platform session records that it opened the merge request (link_type = created), whoever the author is.

The filter is meant only for the non-Duo side (createdByDuo = false). Flows that run remotely open merge requests as a service account, which is not human, so filtering the Duo side would drop real Duo merge requests.

Nothing uses the filter yet. Follow-ups:

  • glql!569 (merged) adds authorHuman to the GLQL compiler.
  • After a GLQL release, bump @gitlab/query-language-rust and gitlab_query_language, and add authorHuman = true to the "Cycle time: all other" and "Merged: all other" panels in dap_impact.yaml.
Why a dictionary of non-human users

A siphon_users subquery in the filter expression, measured on gitlab.com for gitlab-org:

Query Duration Data read
Current baseline ~200 ms 17 MB
Baseline + siphon_users subquery ~2 s warm, >10 s cold 1.4 GB

The dictionary runs that scan once per refresh instead of on every dashboard load. A stored column on merge_requests would need a backfill and a new join in the enrichment view, and it would go stale when a user's type changes.

gitlab.com has about 3 million non-human users and 23 million humans. A dictionary of non-human users is therefore about an eighth of the size a dictionary of humans would be: an estimated 60–120 MB per replica, against 0.5–1 GB.

For database review:

  • The dictionary source query is the same siphon_users scan, run on each refresh and on each replica. After deploy, check element_count, bytes_allocated and loading_duration in system.dictionaries. If the refresh is too heavy, raise LIFETIME; it doesn't change the result.
  • Placeholder users from imports are non-human, so imported merge requests not yet reassigned to a real user count as not human. So do merge requests by deleted users, because GitLab moves them to the ghost user.
  • On an instance that replicates merge_requests but not users, the dictionary is empty and authorHuman = true keeps every merge request that has an author.

References

Screenshots or screen recordings

Not applicable: backend only, no UI change.

How to set up and validate locally

  1. Run bundle exec rake gitlab:clickhouse:migrate.
  2. In ClickHouse, insert a project bot user and reload the dictionary:
    INSERT INTO siphon_users (id, user_type) VALUES (424242, 6);
    SYSTEM RELOAD DICTIONARY non_human_users_dict;
    SELECT dictHas('non_human_users_dict', toUInt64(424242)); -- 1
  3. In GraphiQL, query analytics { mergeRequests(authorHuman: [true]) { ... } } for a group with merge requests synced to ClickHouse. Merge requests by user 424242 are excluded.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

🤖 Generated with Claude Code

Edited by Serhii Yarynovskyi

Merge request reports

Loading
Loading