Increase the page size for the contributions GraphQL query
What does this MR do and why?
This change increases the max page size for the contributions
query. Clients may request up to 500 records per page. The main user of this query is the Contribution Analytics feature, where we noticed that large volume of batching (pagination) queries affect the page load significantly: https://gitlab.com/gitlab-com/gitlab-OKRs/-/work_items/4084#note_1644920882
By default, we preserve the default page size (100) for backward compability reason. The frontend controls the page size which is currently behind a feature flag: use_500_page_size_for_contribution_analytics
How to set up and validate locally
- Ensure that you're on premium.
- Go to a group and visit
Analyze > Contribution Analytics
- Inspect the GraphQL request and notice that it uses
first: null
(means: fall back to the default page size. - Enable the
use_500_page_size_for_contribution_analytics
feature flag. - Not the same request should sent the
first: 500
argument.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #431546 (closed)
Merge request reports
Activity
changed milestone to %16.7
added devopsplan groupoptimize sectiondev workflowready for development labels
assigned to @ahegyi
- A deleted user
added backend feature flag frontend labels
- Resolved by Adam Hegyi
Multiversion compatibility
This merge request updates GraphQL backend and frontend code.
To prevent an incident, ensure the updated frontend code is backwards compatible.
For more information, see the multiversion compatibility documentation.
Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer backend @ibaum
(UTC-6, 7 hours behind author)
@engwan
(UTC+8, 7 hours ahead of author)
frontend @robyrne
(UTC+0, 1 hour behind author)
@ntepluhina
(UTC+1, same timezone as author)
Please check reviewer's status!
Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
Danger- Resolved by 🤖 GitLab Bot 🤖
Proper labels assigned to this merge request. Please ignore me.
@ahegyi - please see the following guidance and update this merge request.1 Error Please add typebug typefeature, or typemaintenance label to this merge request. Edited by 🤖 GitLab Bot 🤖
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@f6dca8b3
added 188 commits
-
493e604d...ccb15d3f - 187 commits from branch
master
- 49b34bc8 - Allow larger page size for contributions query
-
493e604d...ccb15d3f - 187 commits from branch
added featureenhancement typefeature labels
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@2335bf18
Bundle size analysis [beta]
This compares changes in bundle size for entry points between the commits f7f4979f and 44b589b8
Special assetsEntrypoint / Name Size before Size after Diff Diff in percent average 4.08 MB 4.08 MB - 0.0 % mainChunk 3.08 MB 3.08 MB - 0.0 %
Note: We do not have exact data for f7f4979f. So we have used data from: 094a3c9f.
The intended commit has no webpack pipeline, so we chose the last commit with one before it.Please look at the full report for more details
Read more about how this report works.
Generated by
Danger- Resolved by Adam Hegyi
@apennells, could you assist here? What would be the best way to test the GraphQL query page size param on the FE?
requested review from @apennells
Allure report
allure-report-publisher
generated test report!e2e-test-on-gdk:
test report for 44b589b8expand test summary
+-----------------------------------------------------------------------+ | suites summary | +------------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +------------------+--------+--------+---------+-------+-------+--------+ | Create | 40 | 0 | 7 | 0 | 47 | ✅ | | Verify | 32 | 0 | 0 | 0 | 32 | ✅ | | Govern | 48 | 0 | 0 | 0 | 48 | ✅ | | Data Stores | 22 | 0 | 0 | 0 | 22 | ✅ | | Plan | 55 | 0 | 0 | 0 | 55 | ✅ | | Monitor | 4 | 0 | 0 | 0 | 4 | ✅ | | Framework sanity | 0 | 0 | 1 | 0 | 1 | ➖ | | Package | 0 | 0 | 1 | 0 | 1 | ➖ | | Manage | 0 | 0 | 1 | 0 | 1 | ➖ | +------------------+--------+--------+---------+-------+-------+--------+ | Total | 201 | 0 | 10 | 0 | 211 | ✅ | +------------------+--------+--------+---------+-------+-------+--------+
e2e-package-and-test:
test report for 44b589b8expand test summary
+------------------------------------------------------------+ | suites summary | +-------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +-------+--------+--------+---------+-------+-------+--------+ | Plan | 158 | 0 | 6 | 0 | 164 | ✅ | +-------+--------+--------+---------+-------+-------+--------+ | Total | 158 | 0 | 6 | 0 | 164 | ✅ | +-------+--------+--------+---------+-------+-------+--------+
e2e-review-qa:
test report for 44b589b8expand test summary
+-----------------------------------------------------------------------+ | suites summary | +------------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +------------------+--------+--------+---------+-------+-------+--------+ | Plan | 54 | 0 | 1 | 0 | 55 | ✅ | | Data Stores | 2 | 0 | 0 | 0 | 2 | ✅ | | Create | 8 | 0 | 2 | 0 | 10 | ✅ | | Monitor | 4 | 0 | 0 | 0 | 4 | ✅ | | Package | 0 | 0 | 1 | 0 | 1 | ➖ | | Framework sanity | 0 | 0 | 1 | 0 | 1 | ➖ | | Govern | 3 | 0 | 0 | 0 | 3 | ✅ | +------------------+--------+--------+---------+-------+-------+--------+ | Total | 71 | 0 | 5 | 0 | 76 | ✅ | +------------------+--------+--------+---------+-------+-------+--------+
added 1 commit
- abc14b70 - Allow larger page size for contributions query
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@741f9b67
added 1 commit
- fd095dbe - Allow larger page size for contributions query
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@f9dd6786
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@1ae5ebe1
added 1 commit
- 80bf927a - Allow larger page size for contributions query
- A deleted user
added Solutions development guidelines docsimprovement documentation maintenancerefactor typemaintenance labels and removed typefeature label
removed featureenhancement label
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@d7c5dbe2
- Resolved by Adam Hegyi
Hi, @cablett! Could you do the BE review?
requested review from @cablett
- Resolved by Adam Hegyi
Hi, @ekigbo! Could you do the FE review?
requested review from @ekigbo
- Resolved by Adam Hegyi
added 1 commit
- 44b589b8 - Allow larger page size for contributions query
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@c2312c34
- Resolved by Adam Hegyi
Hi, @felipe_artur! Could you do the BE maintainer review?
requested review from @felipe_artur
added workflowin review label and removed workflowready for development label
- Resolved by Adam Hegyi
@ekigbo
, thanks for approving this merge request.This is the first time the merge request has been approved. To ensure we don't only run predictive pipelines, and we don't break
master
, a new pipeline will be started shortly.Please wait for the pipeline to start before resolving this discussion and set auto-merge for the new pipeline. See merging a merge request for more details.
added pipeline:mr-approved label
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@93fdc4e9
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@cdf8b7c5
enabled an automatic merge when the pipeline for cdf8b7c5 succeeds
mentioned in commit gitlab-org-sandbox/gitlab-jh-validation@20797b28
mentioned in commit 75f63d6e
added workflowstaging-canary label and removed workflowin review label
@ahegyi This merge request was deployed to the workflowstaging-canary environment. You may want to enable the associated feature flag on this environment with/chatops run feature set use_500_page_size_for_contribution_analytics true --staging
.This message was generated automatically. You're welcome to improve it.
@ahegyi This merge request was deployed to the workflowcanary environment. You may want to enable the associated feature flag on this environment with/chatops run feature set use_500_page_size_for_contribution_analytics true --production
.This message was generated automatically. You're welcome to improve it.
@ahegyi This merge request was deployed to the workflowstaging environment. You may want to enable the associated feature flag on this environment with/chatops run feature set use_500_page_size_for_contribution_analytics true --staging
.This message was generated automatically. You're welcome to improve it.
@ahegyi This merge request was deployed to the workflowproduction environment. You may want to enable the associated feature flag on this environment with/chatops run feature set use_500_page_size_for_contribution_analytics true --production
.This message was generated automatically. You're welcome to improve it.
17 17 "The end date must be within #{NUMBER_OF_DAYS} days after the start date." 18 18 # rubocop:enable Layout/LineLength 19 19 20 max_page_size 500 21 default_page_size GitlabSchema.default_max_page_size # ensure backwards compability This line seems to be causing specs not to run anymore locally:
An error occurred while loading ./spec/models/environment_spec.rb. Failure/Error: type.fields[field_name.to_s.camelize(:lower)].description NoMethodError: undefined method `description' for nil:NilClass type.fields[field_name.to_s.camelize(:lower)].description ^^^^^^^^^^^^ # ./lib/gitlab/graphql/copy_field_description.rb:17:in `copy_field_description' # ./ee/app/graphql/mutations/boards/scoped_board_mutation.rb:13:in `block in <module:ScopedBoardMutation>' # ./lib/gitlab/patch/prependable.rb:39:in `class_eval' # ./lib/gitlab/patch/prependable.rb:39:in `prepend_features' # ./ee/app/graphql/ee/mutations/boards/create.rb:11:in `prepend' # ./ee/app/graphql/ee/mutations/boards/create.rb:11:in `block in <module:Create>' # ./lib/gitlab/patch/prependable.rb:39:in `class_eval' # ./lib/gitlab/patch/prependable.rb:39:in `prepend_features' # ./config/initializers/0_inject_enterprise_edition_module.rb:45:in `prepend' # ./config/initializers/0_inject_enterprise_edition_module.rb:45:in `prepend_module' # ./config/initializers/0_inject_enterprise_edition_module.rb:8:in `block in prepend_mod_with' # ./config/initializers/0_inject_enterprise_edition_module.rb:60:in `block in each_extension_for' # ./config/initializers/0_inject_enterprise_edition_module.rb:53:in `each' # ./config/initializers/0_inject_enterprise_edition_module.rb:53:in `each_extension_for' # ./config/initializers/0_inject_enterprise_edition_module.rb:7:in `prepend_mod_with' # ./app/graphql/mutations/boards/create.rb:32:in `<main>' # /Users/tkuah/.rbenv/versions/3.1.4/lib/ruby/gems/3.1.0/gems/bootsnap-1.17.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require' # /Users/tkuah/.rbenv/versions/3.1.4/lib/ruby/gems/3.1.0/gems/zeitwerk-2.6.7/lib/zeitwerk/kernel.rb:30:in `require' # ./app/graphql/types/mutation_type.rb:34:in `<class:MutationType>' # ./app/graphql/types/mutation_type.rb:4:in `<module:Types>' # ./app/graphql/types/mutation_type.rb:3:in `<main>' # /Users/tkuah/.rbenv/versions/3.1.4/lib/ruby/gems/3.1.0/gems/bootsnap-1.17.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require' # /Users/tkuah/.rbenv/versions/3.1.4/lib/ruby/gems/3.1.0/gems/zeitwerk-2.6.7/lib/zeitwerk/kernel.rb:30:in `require' # ./app/graphql/gitlab_schema.rb:29:in `<class:GitlabSchema>' # ./app/graphql/gitlab_schema.rb:5:in `<main>' # /Users/tkuah/.rbenv/versions/3.1.4/lib/ruby/gems/3.1.0/gems/bootsnap-1.17.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require' # /Users/tkuah/.rbenv/versions/3.1.4/lib/ruby/gems/3.1.0/gems/zeitwerk-2.6.7/lib/zeitwerk/kernel.rb:30:in `require' # ./ee/app/graphql/resolvers/analytics/contribution_analytics/contributions_resolver.rb:23:in `<class:ContributionsResolver>' # ./ee/app/graphql/resolvers/analytics/contribution_analytics/contributions_resolver.rb:6:in `<module:ContributionAnalytics>' # ./ee/app/graphql/resolvers/analytics/contribution_analytics/contributions_resolver.rb:5:in `<module:Analytics>' # ./ee/app/graphql/resolvers/analytics/contribution_analytics/contributions_resolver.rb:4:in `<module:Resolvers>' # ./ee/app/graphql/resolvers/analytics/contribution_analytics/contributions_resolver.rb:3:in `<main>'
Maybe swap out
GitlabSchema.default_max_page_size
for100
to avoid this load issue ?Opened !137239 (merged)
Edited by Thong Kuah@tkuah Thanks for the fix!
mentioned in commit bcb7603b
mentioned in merge request !137239 (merged)
added workflowcanary label and removed workflowstaging-canary label
added workflowstaging label and removed workflowcanary label
added workflowproduction label and removed workflowstaging label
added workflowpost-deploy-db-staging label and removed workflowproduction label
added workflowpost-deploy-db-production label and removed workflowpost-deploy-db-staging label
removed docsimprovement label
removed development guidelines label
removed documentation label
removed Solutions label
added releasedcandidate label
mentioned in merge request kubitus-project/kubitus-installer!2678 (merged)