intermittent 502 error in gitlab while accessing project page
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
I could see intermittent 502 error while accessing the gitlab projects
- The gitlab.rb settings is like below for unicorn `## Tweak unicorn settings. unicorn['worker_timeout'] = 300
unicorn['worker_processes'] = 2
unicorn['listen'] = '127.0.0.1'`
The unicorn worker process is still in default as 2 but i could see the unicorn process in server is more. PFB
git 2461 57974 1 11:21 ? 00:02:58 unicorn worker[51] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 2952 57974 2 12:00 ? 00:02:44 unicorn worker[17] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 3052 57974 6 14:10 ? 00:00:02 unicorn worker[53] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 3393 57974 1 09:22 ? 00:04:40 unicorn worker[19] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
root 4770 4753 0 Jan24 ? 00:00:00 runsv unicorn
root 4786 4770 0 Jan24 ? 00:00:00 svlogd -tt /var/log/gitlab/unicorn
git 5331 57974 1 11:43 ? 00:02:51 unicorn worker[13] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 5426 57974 2 11:21 ? 00:03:28 unicorn worker[40] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 5508 57974 1 11:43 ? 00:02:34 unicorn worker[46] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
root 7482 64262 0 14:11 pts/0 00:00:00 grep unicorn
git 7807 57974 1 12:09 ? 00:02:08 unicorn worker[28] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 8071 57974 1 12:44 ? 00:01:32 unicorn worker[41] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 8302 57974 2 11:33 ? 00:03:25 unicorn worker[3] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 9082 57974 1 09:32 ? 00:05:04 unicorn worker[14] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 9521 57974 1 09:42 ? 00:04:23 unicorn worker[7] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 9563 57974 1 10:55 ? 00:03:15 unicorn worker[44] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 10025 57974 1 08:43 ? 00:05:43 unicorn worker[25] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 10293 57974 1 08:20 ? 00:05:45 unicorn worker[60] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 11996 57974 1 05:23 ? 00:07:01 unicorn worker[63] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 12658 57974 1 10:03 ? 00:04:51 unicorn worker[27] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 12799 57974 1 10:18 ? 00:03:46 unicorn worker[11] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 14594 57974 2 12:29 ? 00:02:02 unicorn worker[57] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 16520 57974 1 12:46 ? 00:01:30 unicorn worker[0] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 17654 57974 1 10:40 ? 00:03:55 unicorn worker[48] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 20043 57974 1 08:04 ? 00:06:25 unicorn worker[45] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 21460 57974 1 10:12 ? 00:03:58 unicorn worker[26] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 21537 57974 2 13:44 ? 00:00:33 unicorn worker[61] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 22456 57974 0 Jun09 ? 00:11:52 unicorn worker[10] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 23467 57974 1 11:15 ? 00:02:59 unicorn worker[8] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 23785 57974 1 12:11 ? 00:02:17 unicorn worker[62] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 24462 57974 1 12:11 ? 00:02:18 unicorn worker[55] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 27218 57974 1 13:45 ? 00:00:27 unicorn worker[39] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 27401 57974 1 05:31 ? 00:07:08 unicorn worker[59] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 30191 57974 2 12:57 ? 00:01:29 unicorn worker[36] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 30406 57974 2 14:04 ? 00:00:08 unicorn worker[1] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 30648 57974 1 13:07 ? 00:01:10 unicorn worker[6] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 31128 57974 2 14:04 ? 00:00:08 unicorn worker[31] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 33321 57974 1 13:46 ? 00:00:29 unicorn worker[23] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 38453 57974 1 11:57 ? 00:02:13 unicorn worker[54] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 39039 57974 2 12:33 ? 00:02:08 unicorn worker[2] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 40203 57974 0 Jun08 ? 00:19:21 unicorn worker[42] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 40334 57974 1 13:18 ? 00:00:48 unicorn worker[24] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 40438 57974 2 13:47 ? 00:00:30 unicorn worker[52] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 41214 57974 1 08:34 ? 00:05:52 unicorn worker[37] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 41818 57974 1 09:37 ? 00:04:54 unicorn worker[58] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 42092 57974 1 06:25 ? 00:07:55 unicorn worker[49] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 42930 57974 1 08:51 ? 00:05:57 unicorn worker[16] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 43680 57974 0 Jun07 ? 00:24:53 unicorn worker[34] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 43692 57974 2 14:06 ? 00:00:08 unicorn worker[33] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 43925 57974 1 09:47 ? 00:04:30 unicorn worker[9] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 44086 57974 1 10:24 ? 00:03:49 unicorn worker[5] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 46931 57974 1 13:10 ? 00:01:07 unicorn worker[35] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 49912 57974 1 06:29 ? 00:07:10 unicorn worker[64] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 50853 57974 1 10:44 ? 00:03:37 unicorn worker[20] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 51474 57974 1 12:51 ? 00:01:23 unicorn worker[47] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 51833 57974 1 10:51 ? 00:03:27 unicorn worker[29] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 53962 57974 2 12:35 ? 00:01:59 unicorn worker[12] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 54969 57974 1 10:08 ? 00:04:51 unicorn worker[18] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 57436 57974 2 12:16 ? 00:02:20 unicorn worker[21] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 57517 57974 1 09:30 ? 00:05:17 unicorn worker[4] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 57974 1 0 May09 ? 00:04:36 unicorn master -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 58657 4770 0 May09 ? 00:25:17 /bin/bash /opt/gitlab/embedded/bin/gitlab-unicorn-wrapper
git 58709 57974 1 10:16 ? 00:04:09 unicorn worker[22] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 59173 57974 1 11:20 ? 00:03:15 unicorn worker[30] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 59913 57974 1 09:50 ? 00:04:44 unicorn worker[38] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 62314 57974 1 11:20 ? 00:03:14 unicorn worker[43] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 64252 57974 1 12:08 ? 00:02:12 unicorn worker[15] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 64374 57974 1 13:41 ? 00:00:32 unicorn worker[32] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 64387 57974 1 12:08 ? 00:02:14 unicorn worker[50] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
git 64922 57974 1 05:17 ? 00:07:33 unicorn worker[56] -D -E production -c /var/opt/gitlab/gitlab-rails/etc/unicorn.rb /opt/gitlab/embedded/service/gitlab-rails/config.ru
what is the difference in unicorn worker default value and the number of process currently running?
Currently we have enough CPU as like below,
Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 56 On-line CPU(s) list: 0-55 Thread(s) per core: 2 Core(s) per socket: 14 Socket(s): 2 NUMA node(s): 2 Vendor ID: GenuineIntel CPU family: 6 Model: 63 Model name: Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz Stepping: 2 CPU MHz: 2600.000 BogoMIPS: 5193.72 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 35840K NUMA node0 CPU(s): 0-13,28-41 NUMA node1 CPU(s): 14-27,42-55