Explore removing Node Selector and switching to unified node pools in Cloud Native Hybrid Reference Architectures
- Originated from the discussion gitlab-org/gitlab!62316 (comment 596456387)
The proposal is to remove Node Selector in Cloud Native charts for Reference Architecture to remove complexity.
Initial test results with 50k environment showed that 50k Hybrid with node selector is slightly more performant than without, but the results are comparable:
GPT results comparison
| NAME | TARGET | no node selector | with node selector |
|---|---|---|---|
| api_v4_groups | 500ms | 152.33ms (-) | 137.97ms (-2.87%) |
| api_v4_groups_group | 7500ms | 2069.28ms (-) | 1904.24ms (-33.01%) |
| api_v4_groups_group_subgroups | 500ms | 220.65ms (-) | 160.58ms (-12.01%) |
| api_v4_groups_issues | 3500ms | 3145.41ms (-) | 3223.31ms (+15.58%) |
| api_v4_groups_merge_requests | 3500ms | 3003.68ms (-) | 3167.19ms (+32.7%) |
| api_v4_groups_projects | 3500ms | 2070.42ms (-) | 2069.22ms (-0.24%) |
| api_v4_projects | 7000ms | 4506.66ms (-) | 4154.46ms (-70.44%) |
| api_v4_projects_deploy_keys | 500ms | 99.58ms (-) | 86.54ms (-2.61%) |
| api_v4_projects_issues | 500ms | 722.40ms (-) | 481.37ms (-48.21%) |
| api_v4_projects_issues_issue | 1500ms | 704.19ms (-) | 578.46ms (-25.15%) |
| api_v4_projects_issues_search | 12000ms | 7855.40ms (-) | 7870.82ms (+3.08%) |
| api_v4_projects_languages | 500ms | 85.98ms (-) | 81.56ms (-0.88%) |
| api_v4_projects_merge_requests | 1000ms | 507.74ms (-) | 385.57ms (-24.43%) |
| api_v4_projects_merge_requests_merge_request | 2000ms | 4997.89ms (-) | 4192.57ms (-161.06%) |
| api_v4_projects_merge_requests_merge_request_changes | 3500ms | 6939.63ms (-) | 5860.28ms (-215.87%) |
| api_v4_projects_merge_requests_merge_request_commits | 500ms | 120.84ms (-) | 117.24ms (-0.72%) |
| api_v4_projects_merge_requests_merge_request_discussions | 500ms | 294.55ms (-) | 284.56ms (-2.0%) |
| api_v4_projects_project | 500ms | 179.62ms (-) | 176.31ms (-0.66%) |
| api_v4_projects_project_pipelines | 500ms | 103.98ms (-) | 103.65ms (-0.07%) |
| api_v4_projects_project_pipelines_pipeline | 500ms | 121.16ms (-) | 118.75ms (-0.48%) |
| api_v4_projects_project_services | 500ms | 86.68ms (-) | 84.08ms (-0.52%) |
| api_v4_projects_releases | 500ms | 136.69ms (-) | 135.88ms (-0.16%) |
| api_v4_projects_repository_branches | 500ms | 86.13ms (-) | 82.91ms (-0.64%) |
| api_v4_projects_repository_branches_branch | 500ms | 130.51ms (-) | 116.59ms (-2.78%) |
| api_v4_projects_repository_branches_search | 6000ms | 85.44ms (-) | 83.06ms (-0.48%) |
| api_v4_projects_repository_commits | 500ms | 111.87ms (-) | 111.10ms (-0.15%) |
| api_v4_projects_repository_commits_commit | 500ms | 166.64ms (-) | 163.18ms (-0.69%) |
| api_v4_projects_repository_commits_commit_diff | 500ms | 175.60ms (-) | 169.47ms (-1.23%) |
| api_v4_projects_repository_compare | 8000ms | 6443.07ms (-) | 5818.58ms (-124.9%) |
| api_v4_projects_repository_files_file | 500ms | 183.24ms (-) | 170.69ms (-2.51%) |
| api_v4_projects_repository_files_file_blame | 35000ms | 13481.83ms (-) | 11482.64ms (-399.84%) |
| api_v4_projects_repository_files_file_raw | 500ms | 134.71ms (-) | 126.26ms (-1.69%) |
| api_v4_projects_repository_tags | 10000ms | 7715.89ms (-) | 5982.40ms (-346.7%) |
| api_v4_projects_repository_tree | 500ms | 138.00ms (-) | 129.74ms (-1.65%) |
| api_v4_user | 500ms | 77.94ms (-) | 73.99ms (-0.79%) |
| api_v4_users | 500ms | 88.11ms (-) | 83.52ms (-0.92%) |
| git_ls_remote | 500ms | 93.54ms (-) | 89.87ms (-0.73%) |
| git_pull | 500ms | 115.95ms (-) | 112.47ms (-0.7%) |
| git_push | 1000ms | 464.26ms (-) | 462.10ms (-0.43%) |
| scenario_api_list_group_variables | 500ms | 124.00ms (-) | 119.17ms (-0.97%) |
| scenario_api_list_project_variables | 500ms | 139.24ms (-) | 133.30ms (-1.19%) |
| scenario_api_new_branches | 1500ms | 359.86ms (-) | 344.37ms (-3.1%) |
| scenario_api_new_commits | 700ms | 457.17ms (-) | 448.84ms (-1.67%) |
| scenario_api_new_group_variables | 500ms | 102.88ms (-) | 95.45ms (-1.49%) |
| scenario_api_new_issues | 2500ms | 256.86ms (-) | 247.65ms (-1.84%) |
| scenario_api_new_project_variables | 500ms | 122.90ms (-) | 107.78ms (-3.02%) |
| web_group | 500ms | 217.90ms (-) | 202.68ms (-3.04%) |
| web_group_issues | 500ms | 374.95ms (-) | 361.78ms (-2.63%) |
| web_group_merge_requests | 500ms | 345.91ms (-) | 329.42ms (-3.3%) |
| web_project | 500ms | 352.79ms (-) | 329.43ms (-4.67%) |
| web_project_branches | 1000ms | 844.15ms (-) | 742.62ms (-20.31%) |
| web_project_branches_search | 1300ms | 948.19ms (-) | 853.02ms (-19.03%) |
| web_project_commit | 10000ms | 2253.68ms (-) | 2204.30ms (-9.88%) |
| web_project_commits | 750ms | 563.48ms (-) | 554.33ms (-1.83%) |
| web_project_file_blame | 7000ms | 3985.56ms (-) | 3895.73ms (-17.97%) |
| web_project_file_rendered | 3000ms | 790.03ms (-) | 760.68ms (-5.87%) |
| web_project_file_source | 1700ms | 1058.36ms (-) | 1079.07ms (+4.14%) |
| web_project_files | 800ms | 263.38ms (-) | 250.88ms (-2.5%) |
| web_project_issue | 2000ms | 943.34ms (-) | 924.42ms (-3.78%) |
| web_project_issues | 500ms | 381.27ms (-) | 369.04ms (-2.45%) |
| web_project_issues_search | 500ms | 374.80ms (-) | 361.25ms (-2.71%) |
| web_project_merge_request | 7500ms | 4921.24ms (-) | 4865.32ms (-11.18%) |
| web_project_merge_request_changes | 1500ms | 807.50ms (-) | 797.55ms (-1.99%) |
| web_project_merge_request_commits | 1750ms | 1432.89ms (-) | 1213.27ms (-43.92%) |
| web_project_merge_requests | 500ms | 379.00ms (-) | 374.56ms (-0.89%) |
| web_project_pipelines | 1000ms | 537.77ms (-) | 536.72ms (-0.21%) |
| web_project_pipelines_pipeline | 2500ms | 1386.47ms (-) | 1387.44ms (+0.19%) |
| web_project_repository_compare | 7500ms | 5836.24ms (-) | 5927.46ms (+18.24%) |
| web_project_tags | 1500ms | 853.22ms (-) | 842.33ms (-2.18%) |
| web_search_global | 2000ms | 138.71ms (-) | 129.67ms (-1.81%) |
| web_search_groups | 3000ms | 119.57ms (-) | 112.91ms (-1.33%) |
| web_search_projects | 3000ms | 147.09ms (-) | 142.56ms (-0.91%) |
| web_user | 4000ms | 338.53ms (-) | 339.55ms (+0.2%) |
So it looks like from performance perspective this change should not be impactful, however there could have been other considerations for separating the workloads. The issue is to explore this area and consult with Distribution team on best practices.
Edited by Grant Young