timeout and error 502 - seem to be related to one user
Summary
We have one user who gets timeouts when creating an issue or a comment. This looks like this:
However, when reloading the page, you see that the issue or the comment was instantly created.
Log:
2020-11-17T14:50:03.660853107Z source=rack-timeout id=Riv9FU0uu58 timeout=60000ms service=60000ms state=timed_out at=error
The GUI is still reloading and running into 502 after about 5min.
Also, when trying to set Todos triggered by that user to "Done", the GUI is running in timeout. This seems to be the case on different users - it just depends to the user from whom the todo was triggered.
Log:
lib/gitlab/error_tracking.rb:52:in `with_context'
lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
source=rack-timeout id=bxBj8DDsWZ1 timeout=60000ms service=60000ms state=timed_out at=error
Results of GitLab environment info
Expand for output related to GitLab environment info
``` root@gitlab-bundle-5bc6dd6957-2mdnd:/# gitlab-rake gitlab:env:info System information System: Proxy: no Current User: git Using RVM: no Ruby Version: 2.6.6p146 Gem Version: 2.7.10 Bundler Version:1.17.3 Rake Version: 12.3.3 Redis Version: 5.0.9 Git Version: 2.28.0 Sidekiq Version:5.2.9 Go Version: unknown GitLab information Version: 13.4.0-ee Revision: e70802d39ca Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 11.9 URL: https://git.example.org HTTP Clone URL: https://git.example.org/some-group/some-project.git SSH Clone URL: ssh://git@git.example.org:32222/some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: no Using Omniauth: yes Omniauth Providers: GitLab Shell Version: 13.7.0 Repository storage paths: - default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git ```
Results of GitLab application Check
Expand for output related to the GitLab application check
root@gitlab-bundle-5bc6dd6957-2mdnd:/# gitlab-rake gitlab:env:info System information System: Proxy: no Current User: git Using RVM: no Ruby Version: 2.6.6p146 Gem Version: 2.7.10 Bundler Version:1.17.3 Rake Version: 12.3.3 Redis Version: 5.0.9 Git Version: 2.28.0 Sidekiq Version:5.2.9 Go Version: unknown GitLab information Version: 13.4.0-ee Revision: e70802d39ca Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 11.9 URL: https://git.reddoxx.com HTTP Clone URL: https://git.reddoxx.com/some-group/some-project.git SSH Clone URL: ssh://git@git.reddoxx.com:32222/some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: no Using Omniauth: yes Omniauth Providers: GitLab Shell Version: 13.7.0 Repository storage paths: - default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git root@gitlab-bundle-5bc6dd6957-2mdnd:/# gitlab-rake gitlab:check SANITIZE=true Checking GitLab subtasks ... Checking GitLab Shell ... GitLab Shell: ... GitLab Shell version >= 13.7.0 ? ... OK (13.7.0) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Internal API available: OK Redis available via internal API: OK gitlab-shell self-check successful Checking GitLab Shell ... Finished Checking Gitaly ... Gitaly: ... default ... OK Checking Gitaly ... Finished Checking Sidekiq ... Sidekiq: ... Running? ... yes Number of Sidekiq processes ... 1 Checking Sidekiq ... Finished Checking Incoming Email ... Incoming Email: ... Reply by email is disabled in config/gitlab.yml Checking Incoming Email ... Finished Checking LDAP ... LDAP: ... LDAP is disabled in config/gitlab.yml Checking LDAP ... Finished Checking GitLab App ... Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 4/2 ... yes 51/3 ... yes 5/7 ... yes 50/9 ... yes 4/10 ... yes 2/11 ... yes 4/12 ... yes 4/13 ... yes 4/14 ... yes 8/15 ... yes 8/16 ... yes 8/17 ... yes 8/18 ... yes 51/19 ... yes 51/20 ... yes 51/21 ... yes 51/22 ... yes 94/23 ... yes 8/24 ... yes 8/25 ... yes 8/26 ... yes 51/27 ... yes 8/28 ... yes 8/29 ... yes 8/30 ... yes 12/31 ... yes 10/32 ... yes 10/33 ... yes 10/34 ... yes 8/36 ... yes 51/37 ... yes 50/38 ... yes 11/39 ... yes 11/40 ... yes 11/41 ... yes 11/42 ... yes 11/43 ... yes 12/46 ... yes 12/48 ... yes 5/49 ... yes 13/50 ... yes 51/51 ... yes 14/52 ... yes 14/54 ... yes 14/55 ... yes 14/56 ... yes 14/57 ... yes 14/58 ... yes 4/59 ... yes 15/61 ... yes 12/62 ... yes 4/63 ... yes 5/64 ... yes 5/65 ... yes 12/66 ... yes 5/67 ... yes 17/68 ... yes 5/69 ... yes 8/70 ... yes 5/71 ... yes 5/72 ... yes 33/73 ... yes 5/74 ... yes 37/75 ... yes 8/76 ... yes 8/77 ... yes 37/78 ... yes 37/79 ... yes 19/80 ... yes 19/81 ... yes 19/82 ... yes 19/83 ... yes 19/84 ... yes 19/85 ... yes 19/86 ... yes 19/87 ... yes 20/88 ... yes 33/89 ... yes 37/92 ... yes 32/97 ... yes 25/99 ... yes 4/101 ... yes 25/105 ... yes 25/106 ... yes 25/107 ... yes 2/111 ... yes 2/113 ... yes 2/114 ... yes 32/115 ... yes 2/116 ... yes 25/118 ... yes 25/119 ... yes 17/124 ... yes 15/126 ... yes 33/132 ... yes 33/134 ... yes 4/135 ... yes 32/138 ... yes 33/139 ... yes 33/140 ... yes 33/141 ... yes 33/143 ... yes 35/144 ... yes 22/145 ... yes 35/146 ... yes 35/147 ... yes 35/148 ... yes 4/149 ... yes 5/150 ... yes 5/151 ... yes 22/153 ... yes 2/154 ... yes 14/155 ... yes 35/156 ... yes 22/158 ... yes 33/159 ... yes 5/162 ... yes 22/163 ... yes 36/164 ... yes 36/165 ... yes 36/166 ... yes 36/170 ... yes 36/171 ... yes 36/172 ... yes 36/173 ... yes 36/174 ... yes 36/175 ... yes 36/176 ... yes 36/177 ... yes 36/178 ... yes 36/179 ... yes 36/180 ... yes 36/181 ... yes 36/182 ... yes 36/183 ... yes 36/184 ... yes 36/185 ... yes 36/186 ... yes 4/187 ... yes 33/191 ... yes 32/193 ... yes 5/194 ... yes 22/198 ... yes 33/199 ... yes 14/200 ... yes 4/204 ... yes 33/205 ... yes 5/206 ... yes 33/207 ... yes 9/208 ... yes 4/209 ... yes 14/210 ... yes 4/211 ... yes 33/214 ... yes 36/215 ... yes 35/216 ... yes 35/218 ... yes 36/219 ... yes 40/222 ... yes 32/224 ... yes 33/231 ... yes 33/233 ... yes 42/234 ... yes 33/235 ... yes 32/237 ... yes 36/238 ... yes 36/239 ... yes 17/240 ... yes 36/241 ... yes 32/242 ... yes 40/243 ... yes 40/244 ... yes 51/246 ... yes 31/247 ... yes 14/249 ... yes 22/255 ... yes 33/257 ... yes 37/258 ... yes 33/259 ... yes 66/260 ... yes 36/261 ... yes 40/262 ... yes 33/265 ... yes 36/268 ... yes 43/269 ... yes 43/270 ... yes 43/271 ... yes 35/272 ... yes 31/273 ... yes 33/274 ... yes 35/278 ... yes 42/279 ... yes 42/280 ... yes 4/281 ... yes 43/282 ... yes 33/284 ... yes 33/285 ... yes 45/288 ... yes 4/289 ... yes 43/290 ... yes 43/291 ... yes 35/292 ... yes 43/293 ... yes 43/295 ... yes 33/296 ... yes 43/297 ... yes 43/298 ... yes 35/299 ... yes 22/300 ... yes 31/301 ... yes 43/302 ... yes 33/303 ... yes 43/304 ... yes 51/305 ... yes 43/306 ... yes 54/309 ... yes 36/310 ... yes 22/311 ... yes 43/313 ... yes 2/314 ... yes 45/316 ... yes 42/317 ... yes 43/318 ... yes 25/319 ... yes 43/320 ... yes 43/321 ... yes 43/322 ... yes 43/323 ... yes 45/324 ... yes 45/326 ... yes 43/327 ... yes 43/328 ... yes 14/329 ... yes 43/331 ... yes 43/333 ... yes 43/334 ... yes 5/335 ... yes 46/336 ... yes 43/337 ... yes 33/342 ... yes 43/343 ... yes 36/344 ... yes 2/345 ... yes 43/347 ... yes 14/348 ... yes 49/349 ... yes 45/350 ... yes 45/351 ... yes 46/352 ... yes 22/353 ... yes 43/354 ... yes 35/355 ... yes 43/356 ... yes 46/357 ... yes 46/359 ... yes 33/360 ... yes 4/361 ... yes 43/362 ... yes 9/363 ... yes 43/364 ... yes 43/365 ... yes 36/366 ... yes 45/367 ... yes 45/368 ... yes 43/369 ... yes 43/370 ... yes 45/372 ... yes 31/373 ... yes 33/374 ... yes 54/376 ... yes 33/377 ... yes 53/378 ... yes 33/379 ... yes 53/380 ... yes 53/381 ... yes 45/382 ... yes 15/384 ... yes 43/385 ... yes 45/386 ... yes 22/387 ... yes 32/388 ... yes 32/389 ... yes 43/390 ... yes 53/391 ... yes 22/392 ... yes 53/393 ... yes 53/394 ... yes 43/395 ... yes 53/396 ... yes 53/397 ... yes 22/398 ... yes 33/399 ... yes 14/401 ... yes 57/402 ... yes 57/403 ... yes 14/404 ... yes 2/405 ... yes 22/406 ... yes 57/408 ... yes 22/410 ... yes 51/411 ... yes 51/412 ... yes 53/413 ... yes 43/414 ... yes 43/415 ... yes 22/417 ... yes 22/418 ... yes 58/419 ... yes 45/420 ... yes 68/421 ... yes 58/423 ... yes 43/424 ... yes 43/425 ... yes 57/427 ... yes 58/428 ... yes 63/432 ... yes 63/433 ... yes 63/434 ... yes 63/436 ... yes 64/437 ... yes 63/438 ... yes 43/439 ... yes 64/440 ... yes 68/442 ... yes 32/443 ... yes 2/444 ... yes 43/445 ... yes 43/446 ... yes 43/447 ... yes 43/448 ... yes 25/450 ... yes 15/451 ... yes 33/452 ... yes 58/453 ... yes 43/454 ... yes 45/455 ... yes 43/456 ... yes 43/457 ... yes 43/458 ... yes 68/460 ... yes 51/461 ... yes 66/463 ... yes 43/464 ... yes 33/465 ... yes 33/466 ... yes 68/468 ... yes 43/469 ... yes 33/470 ... yes 68/471 ... yes 43/472 ... yes 43/473 ... yes 43/474 ... yes 33/475 ... yes 9/476 ... yes 43/477 ... yes 68/478 ... yes 68/479 ... yes 32/480 ... yes 22/481 ... yes 2/482 ... yes 43/483 ... yes 43/484 ... yes 66/485 ... yes 68/486 ... yes 43/487 ... yes 43/488 ... yes 35/489 ... yes 9/490 ... yes 43/491 ... yes 43/492 ... yes 43/493 ... yes 43/494 ... yes 33/495 ... yes 33/496 ... yes 43/497 ... yes 2/498 ... yes 4/499 ... yes 31/500 ... yes 22/501 ... yes 66/502 ... yes 49/503 ... yes 43/504 ... yes 35/505 ... yes 36/508 ... yes 36/510 ... yes 68/511 ... yes 68/512 ... yes 43/514 ... yes 14/517 ... yes 35/518 ... yes 43/519 ... yes 14/520 ... yes 2/521 ... yes 45/522 ... yes 72/523 ... yes 68/524 ... yes 72/525 ... yes 72/526 ... yes 72/527 ... yes 72/528 ... yes 72/529 ... yes 72/530 ... yes 72/531 ... yes 72/532 ... yes 45/533 ... yes 22/534 ... yes 80/535 ... yes 80/536 ... yes 80/537 ... yes 81/538 ... yes 81/539 ... yes 43/540 ... yes 22/541 ... yes 81/543 ... yes 81/544 ... yes 33/545 ... yes 14/546 ... yes 36/547 ... yes 45/549 ... yes 56/550 ... yes 2/551 ... yes 43/552 ... yes 36/553 ... yes 9/554 ... yes 36/555 ... yes Redis version >= 4.0.0? ... yes Ruby version >= 2.5.3 ? ... yes (2.6.6) Git version >= 2.24.0 ? ... yes (2.28.0) Git user has default SSH configuration? ... yes Active users: ... 19 Is authorized keys file accessible? ... yes GitLab configured to store new projects in hashed storage? ... yes All projects are in hashed storage? ... yes Elasticsearch version 6.x - 7.x? ... skipped (elasticsearch is disabled) Checking GitLab App ... Finished Checking GitLab subtasks ... Finished
Possible fixes
No idea.
Edited by Marc Bihlmaier

