Gitlab 14.2.1, issue with external postgres
Hello, forgive me for my bad english.
I’ve upgraded my GitLab instance from 14.1.0-ee to 14.2.1-ee, and since this upgrade randomly the users have some errors (500, 404, etc). After investigation, it seems that all connections to my separate postgres database are full:
FATAL: remaining connection slots are reserved for non-replication superuser connectionsIt seems that the 14.2.1 version makes a lot more connections to postgres or does not release unused connections ?
Actually I have updated postgres max_connections to 200 (100 to 200). But I really don’t know if it is a real solution.
- Gtlab 14.2.1-ee on docker
- Postgres 12.3 on docker
Some outputs from postgres:
root@0b4b4644e1b3:/var/lib/postgresql# psql -U gitlab -d gitlab
psql (12.3 (Ubuntu 12.3-1.pgdg18.04+1))
Type "help" for help.
gitlab=> SELECT * FROM pg_stat_activity where usename='gitlab';
datid | datname | pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_st
art | xact_start | query_start | state_change | wait_event_type | wait_event | state | backend_xid | backend_xmin |
query
| backend_type
-------+---------+-------+----------+---------+-----------------------------------------------------------------+-------------+-----------------+-------------+-------------------
------------+------------------------------+-------------------------------+-------------------------------+-----------------+------------+--------+-------------+--------------+-
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------
16385 | gitlab | 22571 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 54454 | 2021-08-31 08:42:0
0.317996+00 | | 2021-08-31 08:42:18.360894+00 | 2021-08-31 08:42:18.360942+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22429 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 51436 | 2021-08-31 08:05:5
9.448245+00 | | 2021-08-31 08:42:42.167675+00 | 2021-08-31 08:42:42.167755+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22472 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 52634 | 2021-08-31 08:16:1
0.116239+00 | | 2021-08-31 08:38:43.10234+00 | 2021-08-31 08:38:43.102423+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22553 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 54200 | 2021-08-31 08:37:1
3.72556+00 | | 2021-08-31 08:43:21.039844+00 | 2021-08-31 08:43:21.039922+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22530 | 16384 | gitlab | puma: cluster worker 1: 387 [gitlab-puma-worker] | 10.0.12.10 | | 53876 | 2021-08-31 08:31:4
3.607718+00 | | 2021-08-31 08:43:33.755613+00 | 2021-08-31 08:43:33.755666+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP1FF1HPPFVK89SMJA011C*/ SELECT "users".* FROM "users" WHERE "users"."id" = 20 ORDER BY "users"."id" ASC LIMIT 1
| client backend
16385 | gitlab | 22512 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 53598 | 2021-08-31 08:26:1
8.962853+00 | | 2021-08-31 08:43:36.193993+00 | 2021-08-31 08:43:36.19413+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22452 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 52160 | 2021-08-31 08:10:1
2.385983+00 | | 2021-08-31 08:37:14.024923+00 | 2021-08-31 08:37:14.024985+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22395 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 50856 | 2021-08-31 07:56:1
3.481689+00 | | 2021-08-31 08:22:31.532157+00 | 2021-08-31 08:22:31.532297+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 21953 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 41138 | 2021-08-31 05:59:3
2.886124+00 | | 2021-08-31 06:54:08.285028+00 | 2021-08-31 06:54:08.285119+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22555 | 16384 | gitlab | puma: cluster worker 3: 387 [gitlab-puma-worker] | 10.0.12.10 | | 54216 | 2021-08-31 08:37:2
4.841446+00 | | 2021-08-31 08:42:56.175025+00 | 2021-08-31 08:42:56.175136+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP09MQN9EF8XFSV17PHB1K,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 83 AN
D "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 22541 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [1 of 50 busy] | 10.0.12.10 | | 54070 | 2021-08-31 08:35:1
1.273369+00 | | 2021-08-31 08:43:36.299296+00 | 2021-08-31 08:43:36.299392+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22108 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [5 of 50 busy] | 10.0.12.10 | | 44390 | 2021-08-31 06:38:4
6.273605+00 | | 2021-08-31 07:23:17.594836+00 | 2021-08-31 07:23:17.594907+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22531 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [1 of 50 busy] | 10.0.12.10 | | 53888 | 2021-08-31 08:31:4
6.594758+00 | | 2021-08-31 08:31:46.611075+00 | 2021-08-31 08:31:46.611188+00 | Client | ClientRead | idle | | |
/*application:sidekiq*/ SELECT t.oid, t.typname
+| client backend
| | | | | | | | |
| | | | | | | | |
FROM pg_type as t
+|
| | | | | | | | |
| | | | | | | | |
WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')
+|
| | | | | | | | |
| | | | | | | | |
|
16385 | gitlab | 22358 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 50040 | 2021-08-31 07:45:1
2.728221+00 | | 2021-08-31 08:24:07.265607+00 | 2021-08-31 08:24:07.265727+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22478 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [2 of 50 busy] | 10.0.12.10 | | 52732 | 2021-08-31 08:18:2
0.047683+00 | | 2021-08-31 08:42:59.590561+00 | 2021-08-31 08:42:59.590738+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22532 | 16384 | gitlab | puma: cluster worker 1: 387 [gitlab-puma-worker] | 10.0.12.10 | | 53904 | 2021-08-31 08:31:5
5.062387+00 | | 2021-08-31 08:42:55.653491+00 | 2021-08-31 08:42:55.653611+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP09MV8NYZ1GWW22QS0XCK,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 207 A
ND "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 22373 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 50258 | 2021-08-31 07:48:2
8.264866+00 | | 2021-08-31 08:43:03.731836+00 | 2021-08-31 08:43:03.731975+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22205 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 47096 | 2021-08-31 07:04:1
2.074096+00 | | 2021-08-31 07:04:14.541187+00 | 2021-08-31 07:04:14.541383+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22083 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 43910 | 2021-08-31 06:32:1
1.619388+00 | | 2021-08-31 07:02:33.446094+00 | 2021-08-31 07:02:33.446183+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22533 | 16384 | gitlab | puma: cluster worker 1: 387 [gitlab-puma-worker] | 10.0.12.10 | | 53906 | 2021-08-31 08:31:5
5.063824+00 | | 2021-08-31 08:42:55.56317+00 | 2021-08-31 08:42:55.563223+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP09MX5BTB3GYKBCMMD6AB,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 211 A
ND "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 22525 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 53814 | 2021-08-31 08:30:2
7.103938+00 | | 2021-08-31 08:43:36.193471+00 | 2021-08-31 08:43:36.193604+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22230 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 47548 | 2021-08-31 07:10:3
0.464335+00 | | 2021-08-31 07:28:50.51636+00 | 2021-08-31 07:28:50.516418+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 21973 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 41634 | 2021-08-31 06:05:0
6.961761+00 | | 2021-08-31 07:03:08.214214+00 | 2021-08-31 07:03:08.214273+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22458 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 52302 | 2021-08-31 08:12:3
0.959291+00 | | 2021-08-31 08:35:17.750668+00 | 2021-08-31 08:35:17.750827+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22562 | 16384 | gitlab | puma: cluster worker 4: 387 [gitlab-puma-worker] | 10.0.12.10 | | 54302 | 2021-08-31 08:38:5
5.101809+00 | | 2021-08-31 08:43:08.889148+00 | 2021-08-31 08:43:08.889294+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP0Q33Y80JZCBY78V3GGR9,endpoint_id:UploadsController#show*/ SELECT "uploads".* FROM "uploads" WHERE "uploads"."uploader" = 'AvatarUploader' A
ND "uploads"."path" IN ('uploads/-/system/user/avatar/14/avatar.png', 'user/avatar/14/avatar.png') ORDER BY "uploads"."id" ASC LIMIT 1000 | client backend
16385 | gitlab | 22368 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 50194 | 2021-08-31 07:47:1
2.71085+00 | | 2021-08-31 08:37:14.125146+00 | 2021-08-31 08:37:14.125232+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22488 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 52976 | 2021-08-31 08:21:1
3.5762+00 | | 2021-08-31 08:38:13.540753+00 | 2021-08-31 08:38:13.540922+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 16401 | 16384 | gitlab | psql | | | -1 | 2021-08-30 06:12:4
2.910748+00 | | 2021-08-30 06:22:44.416112+00 | 2021-08-30 06:22:44.417547+00 | Client | ClientRead | idle | | |
SELECT sum(numbackends) FROM pg_stat_database;
| client backend
16385 | gitlab | 22543 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [1 of 50 busy] | 10.0.12.10 | | 54072 | 2021-08-31 08:35:1
2.989308+00 | | 2021-08-31 08:42:29.306116+00 | 2021-08-31 08:42:29.306273+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22359 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 50042 | 2021-08-31 07:45:1
3.02714+00 | | 2021-08-31 08:42:18.158878+00 | 2021-08-31 08:42:18.158954+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22337 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 49814 | 2021-08-31 07:40:2
3.136129+00 | | 2021-08-31 08:35:53.990553+00 | 2021-08-31 08:35:53.990672+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22482 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [3 of 50 busy] | 10.0.12.10 | | 52828 | 2021-08-31 08:20:1
1.350642+00 | | 2021-08-31 08:43:24.172706+00 | 2021-08-31 08:43:24.172882+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22255 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 48124 | 2021-08-31 07:17:5
4.483681+00 | | 2021-08-31 07:22:39.823541+00 | 2021-08-31 07:22:39.823621+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22518 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [1 of 50 busy] | 10.0.12.10 | | 53714 | 2021-08-31 08:28:2
5.438983+00 | | 2021-08-31 08:41:13.499223+00 | 2021-08-31 08:41:13.499449+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22557 | 16384 | gitlab | puma: cluster worker 3: 387 [gitlab-puma-worker] | 10.0.12.10 | | 54262 | 2021-08-31 08:37:5
5.039293+00 | | 2021-08-31 08:42:55.748146+00 | 2021-08-31 08:42:55.748201+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP09MRMWCZ0Z3F8V4Z6QQ1,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 68 AN
D "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 22546 | 16384 | gitlab | puma: cluster worker 0: 387 [gitlab-puma-worker] | 10.0.12.10 | | 54118 | 2021-08-31 08:36:0
3.18236+00 | | 2021-08-31 08:43:34.003638+00 | 2021-08-31 08:43:34.003693+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP1FDN4B26CK0BZ7YMB2M2,endpoint_id:Projects::MergeRequests::ContentController#widget*/ SELECT "project_settings".* FROM "project_settings" WH
ERE "project_settings"."project_id" = 148 LIMIT 1 | client backend
16385 | gitlab | 22534 | 16384 | gitlab | puma: cluster worker 4: 387 [gitlab-puma-worker] | 10.0.12.10 | | 53908 | 2021-08-31 08:31:5
5.076503+00 | | 2021-08-31 08:43:28.4416+00 | 2021-08-31 08:43:28.441649+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP1A1P0745JJNX5GA0T2GW,endpoint_id:Projects::MergeRequests::DiffsController#diffs_metadata*/ SELECT "protected_branch_push_access_levels".* F
ROM "protected_branch_push_access_levels" WHERE "protected_branch_push_access_levels"."protected_branch_id" = 24 | client backend
16385 | gitlab | 22544 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [2 of 50 busy] | 10.0.12.10 | | 54084 | 2021-08-31 08:35:2
5.247198+00 | | 2021-08-31 08:42:17.958702+00 | 2021-08-31 08:42:17.958749+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22499 | 16384 | gitlab | puma: cluster worker 4: 387 [gitlab-puma-worker] | 10.0.12.10 | | 53316 | 2021-08-31 08:23:5
5.046534+00 | | 2021-08-31 08:41:55.414248+00 | 2021-08-31 08:41:55.414292+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDNYF1W0C50SW1DDFPQAHPH,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 193 A
ND "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 22439 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 51798 | 2021-08-31 08:08:0
6.414952+00 | | 2021-08-31 08:43:36.193625+00 | 2021-08-31 08:43:36.193703+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22519 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [1 of 50 busy] | 10.0.12.10 | | 53716 | 2021-08-31 08:28:2
6.146246+00 | | 2021-08-31 08:40:04.991102+00 | 2021-08-31 08:40:04.991366+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22020 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 42654 | 2021-08-31 06:16:3
7.5742+00 | | 2021-08-31 07:06:23.358335+00 | 2021-08-31 07:06:23.358368+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22441 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 51810 | 2021-08-31 08:08:1
5.924226+00 | | 2021-08-31 08:43:36.302636+00 | 2021-08-31 08:43:36.302734+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22574 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [1 of 50 busy] | 10.0.12.10 | | 54508 | 2021-08-31 08:42:1
7.959916+00 | | 2021-08-31 08:42:17.975897+00 | 2021-08-31 08:42:17.976014+00 | Client | ClientRead | idle | | |
/*application:sidekiq*/ SELECT t.oid, t.typname
+| client backend
| | | | | | | | |
| | | | | | | | |
FROM pg_type as t
+|
| | | | | | | | |
| | | | | | | | |
WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')
+|
| | | | | | | | |
| | | | | | | | |
|
16385 | gitlab | 22548 | 16384 | gitlab | puma: cluster worker 0: 387 [gitlab-puma-worker] | 10.0.12.10 | | 54152 | 2021-08-31 08:36:2
5.404988+00 | | 2021-08-31 08:42:56.257368+00 | 2021-08-31 08:42:56.257516+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP09MW2ASV2QSEE2NRGVEN,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 221 A
ND "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 22369 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 50196 | 2021-08-31 07:47:1
2.82107+00 | | 2021-08-31 08:36:06.745965+00 | 2021-08-31 08:36:06.746067+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22427 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 51406 | 2021-08-31 08:05:3
6.160126+00 | | 2021-08-31 08:43:36.302811+00 | 2021-08-31 08:43:36.30287+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22467 | 16384 | gitlab | puma: cluster worker 2: 387 [gitlab-puma-worker] | 10.0.12.10 | | 52584 | 2021-08-31 08:15:3
0.722252+00 | | 2021-08-31 08:43:13.12264+00 | 2021-08-31 08:43:13.122673+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP0V4GSJEYDPMQYX21XFSW,endpoint_id:GET /api/:version/projects/:id/merge_requests/:merge_request_iid/award_emoji*/ SELECT "routes".* FROM "rou
tes" WHERE "routes"."source_id" = 148 AND "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 22432 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 51488 | 2021-08-31 08:06:5
3.441326+00 | | 2021-08-31 08:43:03.840026+00 | 2021-08-31 08:43:03.84024+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22057 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 43350 | 2021-08-31 06:25:2
3.531674+00 | | 2021-08-31 07:23:16.587334+00 | 2021-08-31 07:23:16.587425+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22465 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [2 of 50 busy] | 10.0.12.10 | | 52564 | 2021-08-31 08:15:1
3.85284+00 | | 2021-08-31 08:40:35.163299+00 | 2021-08-31 08:40:35.163377+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22554 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 54202 | 2021-08-31 08:37:1
4.026492+00 | | 2021-08-31 08:37:14.040674+00 | 2021-08-31 08:37:14.040813+00 | Client | ClientRead | idle | | |
/*application:sidekiq*/ SELECT t.oid, t.typname
+| client backend
| | | | | | | | |
| | | | | | | | |
FROM pg_type as t
+|
| | | | | | | | |
| | | | | | | | |
WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')
+|
| | | | | | | | |
| | | | | | | | |
|
16385 | gitlab | 16531 | 16384 | gitlab | psql | | | -1 | 2021-08-30 06:39:5
6.115696+00 | | 2021-08-30 06:40:12.434596+00 | 2021-08-30 06:40:12.439607+00 | Client | ClientRead | idle | | |
SELECT sum(numbackends) FROM pg_stat_database;
| client backend
16385 | gitlab | 22484 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 52850 | 2021-08-31 08:20:3
1.038852+00 | | 2021-08-31 08:43:00.101221+00 | 2021-08-31 08:43:00.101394+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22572 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 54458 | 2021-08-31 08:42:0
3.177044+00 | | 2021-08-31 08:43:00.001355+00 | 2021-08-31 08:43:00.001588+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22378 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 50352 | 2021-08-31 07:49:5
7.037664+00 | | 2021-08-31 08:43:03.528938+00 | 2021-08-31 08:43:03.529055+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 313 | 16384 | gitlab | psql | | | -1 | 2021-08-27 06:48:0
6.752972+00 | | 2021-08-27 07:15:36.721722+00 | 2021-08-27 07:15:36.722198+00 | Client | ClientRead | idle | | |
SELECT sum(numbackends) FROM pg_stat_database;
| client backend
16385 | gitlab | 22558 | 16384 | gitlab | puma: cluster worker 3: 387 [gitlab-puma-worker] | 10.0.12.10 | | 54264 | 2021-08-31 08:37:5
5.104458+00 | | 2021-08-31 08:42:55.615065+00 | 2021-08-31 08:42:55.615165+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP09MW2ASV2QSEDYWY72AB,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 208 A
ND "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 22405 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 51062 | 2021-08-31 07:59:2
0.262743+00 | | 2021-08-31 08:43:36.400438+00 | 2021-08-31 08:43:36.400514+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22550 | 16384 | gitlab | puma: cluster worker 0: 387 [gitlab-puma-worker] | 10.0.12.10 | | 54182 | 2021-08-31 08:36:5
5.062701+00 | | 2021-08-31 08:42:55.949358+00 | 2021-08-31 08:42:55.949393+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP09MNT0YCC1WM2EVHM0AK,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 25 AN
D "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 22514 | 16384 | gitlab | puma: cluster worker 2: 387 [gitlab-puma-worker] | 10.0.12.10 | | 53648 | 2021-08-31 08:26:5
5.122925+00 | | 2021-08-31 08:43:33.880164+00 | 2021-08-31 08:43:33.880218+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP1FDXN71M6C975TWE315R,endpoint_id:GraphqlController#execute*/ SELECT "protected_branch_merge_access_levels".* FROM "protected_branch_merge_a
ccess_levels" WHERE "protected_branch_merge_access_levels"."protected_branch_id" = 152 | client backend
16385 | gitlab | 22364 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [2 of 50 busy] | 10.0.12.10 | | 50102 | 2021-08-31 07:46:0
5.016417+00 | | 2021-08-31 08:43:25.281387+00 | 2021-08-31 08:43:25.281498+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22152 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 45872 | 2021-08-31 06:50:0
7.540355+00 | | 2021-08-31 07:00:37.276257+00 | 2021-08-31 07:00:37.276366+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22470 | 16384 | gitlab | puma: cluster worker 2: 387 [gitlab-puma-worker] | 10.0.12.10 | | 52622 | 2021-08-31 08:15:5
5.033818+00 | | 2021-08-31 08:43:13.206746+00 | 2021-08-31 08:43:13.206778+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP0V3HKHM8KQJHVHXTYS9W,endpoint_id:GraphqlController#execute*/ SELECT SUM("timelogs"."time_spent") FROM "timelogs" WHERE "timelogs"."merge_re
quest_id" = 7033 | client backend
16385 | gitlab | 1129 | 16384 | gitlab | psql | | | -1 | 2021-08-27 09:50:2
8.643537+00 | | 2021-08-27 09:53:00.417732+00 | 2021-08-27 09:53:00.419578+00 | Client | ClientRead | idle | | |
SELECT sum(numbackends) FROM pg_stat_database;
| client backend
16385 | gitlab | 22595 | 16384 | gitlab | psql | | | -1 | 2021-08-31 08:43:3
3.639302+00 | 2021-08-31 08:43:36.95514+00 | 2021-08-31 08:43:36.95514+00 | 2021-08-31 08:43:36.955143+00 | | | active | | 917478 |
SELECT * FROM pg_stat_activity where usename='gitlab';
| client backend
16385 | gitlab | 16089 | 16384 | gitlab | psql | | | -1 | 2021-08-30 05:07:4
5.341506+00 | | 2021-08-30 05:35:11.516553+00 | 2021-08-30 05:35:11.517902+00 | Client | ClientRead | idle | | |
SELECT sum(numbackends) FROM pg_stat_database;
| client backend
16385 | gitlab | 22520 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [1 of 50 busy] | 10.0.12.10 | | 53718 | 2021-08-31 08:28:2
6.650532+00 | | 2021-08-31 08:42:18.058329+00 | 2021-08-31 08:42:18.058411+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22453 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 52166 | 2021-08-31 08:10:1
9.264759+00 | | 2021-08-31 08:43:36.193752+00 | 2021-08-31 08:43:36.193923+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22400 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [1 of 50 busy] | 10.0.12.10 | | 50950 | 2021-08-31 07:58:0
5.606893+00 | | 2021-08-31 08:43:03.627973+00 | 2021-08-31 08:43:03.628255+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22551 | 16384 | gitlab | puma: cluster worker 0: 387 [gitlab-puma-worker] | 10.0.12.10 | | 54184 | 2021-08-31 08:36:5
5.071917+00 | | 2021-08-31 08:43:08.915304+00 | 2021-08-31 08:43:08.915476+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP0Q2X0BMT7PG9AZJ18J94,endpoint_id:UploadsController#show*/ SELECT "uploads".* FROM "uploads" WHERE "uploads"."uploader" = 'AvatarUploader' A
ND "uploads"."path" IN ('uploads/-/system/project/avatar/148/spring-cloud.png', 'project/avatar/148/spring-cloud.png') ORDER BY "uploads"."id" ASC LIMIT 1000 | client backend
16385 | gitlab | 22183 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 46716 | 2021-08-31 06:58:1
8.85198+00 | | 2021-08-31 07:25:08.953478+00 | 2021-08-31 07:25:08.953517+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22501 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 53336 | 2021-08-31 08:24:0
9.493892+00 | | 2021-08-31 08:43:00.000665+00 | 2021-08-31 08:43:00.000852+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 21990 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 42080 | 2021-08-31 06:09:3
5.407217+00 | | 2021-08-31 07:06:25.480755+00 | 2021-08-31 07:06:25.480887+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22508 | 16384 | gitlab | puma: cluster worker 5: 387 [gitlab-puma-worker] | 10.0.12.10 | | 53568 | 2021-08-31 08:25:5
5.046937+00 | | 2021-08-31 08:41:56.052274+00 | 2021-08-31 08:41:56.052329+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDNYF1YEHJV5T5Z36H5MYPR,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 221 A
ND "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 1407 | 16384 | gitlab | psql | | | -1 | 2021-08-27 10:58:3
8.592971+00 | | 2021-08-27 10:58:51.786396+00 | 2021-08-27 10:58:51.792101+00 | Client | ClientRead | idle | | |
SELECT sum(numbackends) FROM pg_stat_database;
| client backend
16385 | gitlab | 22196 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 46928 | 2021-08-31 07:01:2
3.897252+00 | | 2021-08-31 07:01:23.911661+00 | 2021-08-31 07:01:23.911804+00 | Client | ClientRead | idle | | |
/*application:sidekiq*/ SELECT t.oid, t.typname
+| client backend
| | | | | | | | |
| | | | | | | | |
FROM pg_type as t
+|
| | | | | | | | |
| | | | | | | | |
WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')
+|
| | | | | | | | |
| | | | | | | | |
|
16385 | gitlab | 22466 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [2 of 50 busy] | 10.0.12.10 | | 52568 | 2021-08-31 08:15:1
5.178753+00 | | 2021-08-31 08:17:15.852215+00 | 2021-08-31 08:17:15.852295+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22170 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 46084 | 2021-08-31 06:54:1
7.980657+00 | | 2021-08-31 07:30:14.077983+00 | 2021-08-31 07:30:14.078048+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22149 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 45832 | 2021-08-31 06:49:1
2.77545+00 | | 2021-08-31 07:30:14.178612+00 | 2021-08-31 07:30:14.178665+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22151 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 45864 | 2021-08-31 06:49:5
6.807888+00 | | 2021-08-31 07:22:35.077211+00 | 2021-08-31 07:22:35.077327+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22058 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 43352 | 2021-08-31 06:25:2
4.241826+00 | | 2021-08-31 07:06:20.780621+00 | 2021-08-31 07:06:20.780666+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22061 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [3 of 50 busy] | 10.0.12.10 | | 43636 | 2021-08-31 06:26:2
6.916416+00 | | 2021-08-31 07:21:18.756641+00 | 2021-08-31 07:21:18.756741+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22387 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 50732 | 2021-08-31 07:53:2
8.492562+00 | | 2021-08-31 08:30:27.304686+00 | 2021-08-31 08:30:27.304761+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22471 | 16384 | gitlab | puma: cluster worker 2: 387 [gitlab-puma-worker] | 10.0.12.10 | | 52624 | 2021-08-31 08:15:5
5.036042+00 | | 2021-08-31 08:42:55.464076+00 | 2021-08-31 08:42:55.464115+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP09MS9987VC2EF1GT9466,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 193 A
ND "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 22376 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 50314 | 2021-08-31 07:49:2
3.771292+00 | | 2021-08-31 08:43:16.586487+00 | 2021-08-31 08:43:16.586565+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22559 | 16384 | gitlab | puma: cluster worker 3: 387 [gitlab-puma-worker] | 10.0.12.10 | | 54266 | 2021-08-31 08:37:5
5.118183+00 | | 2021-08-31 08:43:35.783004+00 | 2021-08-31 08:43:35.783066+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP1HEWAKMYH0Z2R49S24AB*/ SELECT "users".* FROM "users" WHERE "users"."id" = 20 ORDER BY "users"."id" ASC LIMIT 1
| client backend
16385 | gitlab | 22527 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 53850 | 2021-08-31 08:31:0
6.603976+00 | | 2021-08-31 08:40:04.99424+00 | 2021-08-31 08:40:04.994355+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22340 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 49854 | 2021-08-31 07:41:1
2.409292+00 | | 2021-08-31 08:43:03.936887+00 | 2021-08-31 08:43:03.937001+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 497 | 16384 | gitlab | psql | | | -1 | 2021-08-27 07:33:0
2.351552+00 | | 2021-08-27 07:49:06.729846+00 | 2021-08-27 07:49:06.730433+00 | Client | ClientRead | idle | | |
SELECT sum(numbackends) FROM pg_stat_database;
| client backend
16385 | gitlab | 22354 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 50006 | 2021-08-31 07:44:0
9.334558+00 | | 2021-08-31 08:43:00.100725+00 | 2021-08-31 08:43:00.101004+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22500 | 16384 | gitlab | puma: cluster worker 4: 387 [gitlab-puma-worker] | 10.0.12.10 | | 53320 | 2021-08-31 08:23:5
5.257771+00 | | 2021-08-31 08:42:55.777453+00 | 2021-08-31 08:42:55.777491+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP09MNT0YCC1WM2DAZRSZ9,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 20 AN
D "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 22566 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 54366 | 2021-08-31 08:40:0
5.902056+00 | | 2021-08-31 08:43:16.591752+00 | 2021-08-31 08:43:16.591818+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22505 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 53382 | 2021-08-31 08:25:1
4.75986+00 | | 2021-08-31 08:35:53.994055+00 | 2021-08-31 08:35:53.994226+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22509 | 16384 | gitlab | puma: cluster worker 5: 387 [gitlab-puma-worker] | 10.0.12.10 | | 53570 | 2021-08-31 08:25:5
5.063181+00 | | 2021-08-31 08:41:55.86919+00 | 2021-08-31 08:41:55.869228+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDNYF1W0C50SW1DDNAVTV9H,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 212 A
ND "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 22510 | 16384 | gitlab | puma: cluster worker 5: 387 [gitlab-puma-worker] | 10.0.12.10 | | 53572 | 2021-08-31 08:25:5
5.078978+00 | | 2021-08-31 08:43:29.737479+00 | 2021-08-31 08:43:29.737544+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP1BHX6QT4MGF8KNCXPJX0*/ SELECT "users".* FROM "users" WHERE "users"."id" = 20 ORDER BY "users"."id" ASC LIMIT 1
| client backend
16385 | gitlab | 841 | 16384 | gitlab | psql | | | -1 | 2021-08-27 08:44:3
9.959169+00 | | 2021-08-27 08:52:29.424179+00 | 2021-08-27 08:52:29.424846+00 | Client | ClientRead | idle | | |
SELECT sum(numbackends) FROM pg_stat_database;
| client backend
16385 | gitlab | 22401 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [1 of 50 busy] | 10.0.12.10 | | 50958 | 2021-08-31 07:58:0
6.01902+00 | | 2021-08-31 08:42:59.488+00 | 2021-08-31 08:42:59.488166+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22360 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 50044 | 2021-08-31 07:45:1
3.028526+00 | | 2021-08-31 08:33:06.114673+00 | 2021-08-31 08:33:06.11477+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22361 | 16384 | gitlab | sidekiq 5.2.9 queues:authorize...export_deletion [0 of 50 busy] | 10.0.12.10 | | 50046 | 2021-08-31 07:45:1
3.530242+00 | | 2021-08-31 08:43:36.194162+00 | 2021-08-31 08:43:36.194233+00 | Client | ClientRead | idle | | |
;
| client backend
16385 | gitlab | 22565 | 16384 | gitlab | puma: cluster worker 1: 387 [gitlab-puma-worker] | 10.0.12.10 | | 54354 | 2021-08-31 08:39:5
5.067874+00 | | 2021-08-31 08:42:55.646341+00 | 2021-08-31 08:42:55.646378+00 | Client | ClientRead | idle | | |
/*application:web,correlation_id:01FEDP09MNT0YCC1WM2E1TX2K5,endpoint_id:GET /api/:version/projects/:id/events*/ SELECT "routes".* FROM "routes" WHERE "routes"."source_id" = 21 AN
D "routes"."source_type" = 'Project' LIMIT 1 | client backend
16385 | gitlab | 699 | 16384 | gitlab | psql | | | -1 | 2021-08-27 08:16:4
1.034404+00 | | 2021-08-27 08:28:03.205321+00 | 2021-08-27 08:28:03.207139+00 | Client | ClientRead | idle | | |
SELECT sum(numbackends) FROM pg_stat_database;
| client backend
(102 rows)
gitlab=> SELECT sum(numbackends) FROM pg_stat_database;
sum
-----
103
(1 row)Thanks for your help.
Edited by Achilleas Pipinellis