Skip to content

Fix Notes API performance issue with server-side filtering

What does this MR do and why?

As noted in gitlab-org/quality/quality-engineering/team-tasks#3846 (comment 2778551818) there is an undocumented param activity_filter which can be applied to notes requests.
By using activity_filter=only_comments params we can avoid expensive calls which retrieve all system notes when looking for relevant notes to update.

Screenshots or screen recordings

Using a similar approach as in !343 (merged)

I have some test issues with 3000 system notes. With the currently approach this scales poorly as we iterate over all system notes. Instead, by filtering using the activity_filter=only_comments we can drastically reduce the overhead on the pagination of notes.

➜  gitlab_quality-test_tooling git:(jmd/add_activity_filter_issue_notes) bundle exec ./exe/slow-test-issues --project group3846/project3846 --token $GITLAB_QA_ACCESS_TOKEN --input-files slow_standard.json
[Sep 25 2025 17:14:28 IST (Gitlab QA)] INFO  -- push to GCS is disabled.
[Sep 25 2025 17:14:28 IST (Gitlab QA)] INFO  -- Reporting tests in `slow_standard.json`.
[Sep 25 2025 17:14:28 IST (Gitlab QA)] INFO  -- => Processing 2 tests in slow_standard.json
[Sep 25 2025 17:14:28 IST (Gitlab QA)] INFO  --  => Processing issues for slow test 'Deploy Git clone using a deploy key user sets up a deploy key with QA::Runtime::Key::RSA(8192) to clone code using pipelines'...
Ignoring the following error: Server responded with code 404, message: {"message"=>"404 Commit Not Found"}. Request URI: http://gdk.test:3000/api/v4/projects/group3846%2Fproject3846/repository/files/qa%2Fqa%2Fspecs%2Ffeatures%2Fbrowser_ui%2F6_deploy%2Fdeploy_key%2Fclone_using_deploy_key_spec.rb/raw
[Sep 25 2025 17:14:34 IST (Gitlab QA)] INFO  --    => Reporting slow test to existing issue: http://gdk.test:3000/group3846/project3846/-/issues/2
[Sep 25 2025 17:15:23 IST (Gitlab QA)] INFO  --  => Processing issues for slow test 'Deploy Auto DevOps with a Kubernetes Agent runs auto devops'...
Ignoring the following error: Server responded with code 404, message: {"message"=>"404 Commit Not Found"}. Request URI: http://gdk.test:3000/api/v4/projects/group3846%2Fproject3846/repository/files/qa%2Fqa%2Fspecs%2Ffeatures%2Fbrowser_ui%2F6_deploy%2Fauto_devops%2Fcreate_project_with_auto_devops_spec.rb/raw
[Sep 25 2025 17:15:29 IST (Gitlab QA)] INFO  --    => Reporting slow test to existing issue: http://gdk.test:3000/group3846/project3846/-/issues/3
[Sep 25 2025 17:16:16 IST (Gitlab QA)] INFO  --  => Reported 2 slow tests.

# duration 1m 48sec


➜  gitlab_quality-test_tooling git:(jmd/add_activity_filter_issue_notes) bundle exec ./exe/slow-test-issues --project group3846/project3846 --token $GITLAB_QA_ACCESS_TOKEN --input-files slow_standard.json
[Sep 25 2025 17:16:19 IST (Gitlab QA)] INFO  -- push to GCS is disabled.
[Sep 25 2025 17:16:19 IST (Gitlab QA)] INFO  -- Reporting tests in `slow_standard.json`.
[Sep 25 2025 17:16:19 IST (Gitlab QA)] INFO  -- => Processing 2 tests in slow_standard.json
[Sep 25 2025 17:16:19 IST (Gitlab QA)] INFO  --  => Processing issues for slow test 'Deploy Git clone using a deploy key user sets up a deploy key with QA::Runtime::Key::RSA(8192) to clone code using pipelines'...
Ignoring the following error: Server responded with code 404, message: {"message"=>"404 Commit Not Found"}. Request URI: http://gdk.test:3000/api/v4/projects/group3846%2Fproject3846/repository/files/qa%2Fqa%2Fspecs%2Ffeatures%2Fbrowser_ui%2F6_deploy%2Fdeploy_key%2Fclone_using_deploy_key_spec.rb/raw
[Sep 25 2025 17:16:25 IST (Gitlab QA)] INFO  --    => Reporting slow test to existing issue: http://gdk.test:3000/group3846/project3846/-/issues/2
[Sep 25 2025 17:16:25 IST (Gitlab QA)] INFO  --  => Processing issues for slow test 'Deploy Auto DevOps with a Kubernetes Agent runs auto devops'...
Ignoring the following error: Server responded with code 404, message: {"message"=>"404 Commit Not Found"}. Request URI: http://gdk.test:3000/api/v4/projects/group3846%2Fproject3846/repository/files/qa%2Fqa%2Fspecs%2Ffeatures%2Fbrowser_ui%2F6_deploy%2Fauto_devops%2Fcreate_project_with_auto_devops_spec.rb/raw
[Sep 25 2025 17:16:31 IST (Gitlab QA)] INFO  --    => Reporting slow test to existing issue: http://gdk.test:3000/group3846/project3846/-/issues/3
[Sep 25 2025 17:16:31 IST (Gitlab QA)] INFO  --  => Reported 2 slow tests.

# duration 12 sec

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by John McDonnell

Merge request reports

Loading