postgres-checkup report for gitlab_production: 2019-09-30 (auto-generated)

PostgreSQL Checkup. Project: 'gitlab_production'. Database: 'gitlabhq_production'

Epoch number: '20190930001'

NOTICE: while most reports describe the “current database”, some of them may contain cluster-wide information describing all databases in the cluster.

Last modified at: 2019-09-30 13:59:26 +0000

 

Table of contents

A002 Version Information
A003 Postgres Settings
A004 Cluster Information
A005 Extensions
A006 Postgres Setting Deviations
A007 Altered Settings
D004 pg_stat_statements and pg_stat_kcache Settings
F001 Autovacuum: Current Settings
F002 Autovacuum: Transaction ID Wraparound Check
F003 Autovacuum: Dead Tuples
F004 Autovacuum: Heap Bloat (Estimated)
F005 Autovacuum: Btree Index Bloat (Estimated)
F008 Autovacuum: Resource Usage
G001 Memory-related Settings
G002 Connections and Current Activity
G003 Timeouts, Locks, Deadlocks
H001 Invalid Indexes
H002 Unused and Redundant Indexes
H003 Non-indexed Foreign Keys
K001 Globally Aggregated Query Metrics
K002 Workload Type ("The First Word" Analysis)
K003 Top-20 Queries by total_time
L001 Table Sizes
L003 Integer (int2, int4) Out-of-range Risks in PKs

---

  Table of contents

A002 Version Information

Observations

Data collected: 2019-09-30 13:54:43 +0000 UTC

Master (10.220.16.101)

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu (Ubuntu 9.6.15-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica servers:

Replica (10.220.16.102)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.103)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.104)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.105)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.106)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.107)

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu (Ubuntu 9.6.15-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.108)

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu (Ubuntu 9.6.15-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.109)

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu (Ubuntu 9.6.15-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Conclusions

• [P2] Postgres minor version on the master and replica(s) differ. Nodes 10.220.16.103, 10.220.16.106, 10.220.16.104, 10.220.16.105, 10.220.16.107, 10.220.16.108, 10.220.16.109, 10.220.16.101, 10.220.16.102 use Postgres 9.6.14, 9.6.15.

• Postgres major version being used is 9.6 and it is currently supported by PostgreSQL Global Development Group. End of life is scheduled 2021-11-11. It means that in case of bugs and security issues, updates (new minor versions) with fixes will be released and available for use. Read more: Versioning Policy.

• 9.6.15 is the most up-to-date Postgres minor version in the branch 9.6.

• [P2] The minor version being used (9.6.14) is not up-to-date (the newest version: 9.6.15). See the full list of changes between 9.6.14 and 9.6.15.

Recommendations

• [P2] Please upgrade Postgres so its versions on all nodes match.

• [P3] Consider upgrading to the newest major version: 11. It has a lot of new features and improvements.

• For more information about minor and major upgrades see:

  • Official documentation: https://www.postgresql.org/docs/current/upgrading.html
  • Major-version upgrading with minimal downtime (depesz.com)
  • Upgrading PostgreSQL on AWS RDS with minimum or zero downtime
  • Near-Zero Downtime Automated Upgrades of PostgreSQL Clusters in Cloud (2ndQuadrant.com)
  • Updating a 50 terabyte PostgreSQL database

• [P2] Consider performing upgrade to the newest minor version: 9.6.15.

---

  Table of contents

A003 Postgres Settings

Observations

Data collected: 2019-09-30 13:54:44 +0000 UTC

Master (10.220.16.101)

▼ Category	Setting	Value	Unit	Pretty value
Autovacuum	autovacuum	on
Autovacuum	autovacuum_analyze_scale_factor	0.005
Autovacuum	autovacuum_analyze_threshold	50
Autovacuum	autovacuum_freeze_max_age	200000000
Autovacuum	autovacuum_max_workers	6
Autovacuum	autovacuum_multixact_freeze_max_age	400000000
Autovacuum	autovacuum_naptime	30	s
Autovacuum	autovacuum_vacuum_cost_delay	5	ms
Autovacuum	autovacuum_vacuum_cost_limit	6000
Autovacuum	autovacuum_vacuum_scale_factor	0.005
Autovacuum	autovacuum_vacuum_threshold	50
Client Connection Defaults / Locale and Formatting	DateStyle	ISO, MDY
Client Connection Defaults / Locale and Formatting	IntervalStyle	postgres
Client Connection Defaults / Locale and Formatting	TimeZone	GMT
Client Connection Defaults / Locale and Formatting	client_encoding	UTF8
Client Connection Defaults / Locale and Formatting	default_text_search_config	pg_catalog.english
Client Connection Defaults / Locale and Formatting	extra_float_digits	0
Client Connection Defaults / Locale and Formatting	lc_collate	C.UTF-8
Client Connection Defaults / Locale and Formatting	lc_ctype	C.UTF-8
Client Connection Defaults / Locale and Formatting	lc_messages	C
Client Connection Defaults / Locale and Formatting	lc_monetary	C
Client Connection Defaults / Locale and Formatting	lc_numeric	C
Client Connection Defaults / Locale and Formatting	lc_time	C
Client Connection Defaults / Locale and Formatting	server_encoding	UTF8
Client Connection Defaults / Locale and Formatting	timezone_abbreviations	Default
Client Connection Defaults / Other Defaults	dynamic_library_path	$libdir
Client Connection Defaults / Other Defaults	gin_fuzzy_search_limit	0
Client Connection Defaults / Other Defaults	tcp_keepalives_count	9
Client Connection Defaults / Other Defaults	tcp_keepalives_idle	300	s
Client Connection Defaults / Other Defaults	tcp_keepalives_interval	75	s
Client Connection Defaults / Shared Library Preloading	local_preload_libraries
Client Connection Defaults / Shared Library Preloading	session_preload_libraries
Client Connection Defaults / Shared Library Preloading	shared_preload_libraries	pg_stat_statements
Client Connection Defaults / Statement Behavior	bytea_output	hex
Client Connection Defaults / Statement Behavior	check_function_bodies	on
Client Connection Defaults / Statement Behavior	client_min_messages	notice
Client Connection Defaults / Statement Behavior	default_tablespace
Client Connection Defaults / Statement Behavior	default_transaction_deferrable	off
Client Connection Defaults / Statement Behavior	default_transaction_isolation	read committed
Client Connection Defaults / Statement Behavior	default_transaction_read_only	off
Client Connection Defaults / Statement Behavior	gin_pending_list_limit	4096	kB	4.00 MiB
Client Connection Defaults / Statement Behavior	idle_in_transaction_session_timeout	30000	ms
Client Connection Defaults / Statement Behavior	lock_timeout	0	ms
Client Connection Defaults / Statement Behavior	search_path	"$user", public
Client Connection Defaults / Statement Behavior	session_replication_role	origin
Client Connection Defaults / Statement Behavior	statement_timeout	15000	ms
Client Connection Defaults / Statement Behavior	temp_tablespaces
Client Connection Defaults / Statement Behavior	transaction_deferrable	off
Client Connection Defaults / Statement Behavior	transaction_isolation	read committed
Client Connection Defaults / Statement Behavior	transaction_read_only	off
Client Connection Defaults / Statement Behavior	vacuum_freeze_min_age	50000000
Client Connection Defaults / Statement Behavior	vacuum_freeze_table_age	150000000
Client Connection Defaults / Statement Behavior	vacuum_multixact_freeze_min_age	5000000
Client Connection Defaults / Statement Behavior	vacuum_multixact_freeze_table_age	150000000
Client Connection Defaults / Statement Behavior	xmlbinary	base64
Client Connection Defaults / Statement Behavior	xmloption	content
Connections and Authentication / Connection Settings	bonjour	off
Connections and Authentication / Connection Settings	bonjour_name
Connections and Authentication / Connection Settings	listen_addresses	0.0.0.0
Connections and Authentication / Connection Settings	max_connections	300
Connections and Authentication / Connection Settings	port	5432
Connections and Authentication / Connection Settings	superuser_reserved_connections	3
Connections and Authentication / Connection Settings	unix_socket_directories	/var/opt/gitlab/postgresql
Connections and Authentication / Connection Settings	unix_socket_group
Connections and Authentication / Connection Settings	unix_socket_permissions	0777
Connections and Authentication / Security and Authentication	authentication_timeout	60	s
Connections and Authentication / Security and Authentication	db_user_namespace	off
Connections and Authentication / Security and Authentication	krb_caseins_users	off
Connections and Authentication / Security and Authentication	krb_server_keyfile	FILE:/etc/postgresql-common/krb5.keytab
Connections and Authentication / Security and Authentication	password_encryption	on
Connections and Authentication / Security and Authentication	row_security	on
Connections and Authentication / Security and Authentication	ssl	on
Connections and Authentication / Security and Authentication	ssl_ca_file
Connections and Authentication / Security and Authentication	ssl_cert_file	/var/opt/gitlab/postgresql/server.crt
Connections and Authentication / Security and Authentication	ssl_ciphers	HIGH:MEDIUM:+3DES:!aNULL:!SSLv3:!TLSv1
Connections and Authentication / Security and Authentication	ssl_crl_file
Connections and Authentication / Security and Authentication	ssl_ecdh_curve	prime256v1
Connections and Authentication / Security and Authentication	ssl_key_file	/var/opt/gitlab/postgresql/server.key
Connections and Authentication / Security and Authentication	ssl_prefer_server_ciphers	on
Customized Options	pg_stat_statements.max	5000
Customized Options	pg_stat_statements.save	on
Customized Options	pg_stat_statements.track	top
Customized Options	pg_stat_statements.track_utility	on
Developer Options	allow_system_table_mods	off
Developer Options	ignore_checksum_failure	off
Developer Options	ignore_system_indexes	off
Developer Options	post_auth_delay	0	s
Developer Options	pre_auth_delay	0	s
Developer Options	trace_notify	off
Developer Options	trace_recovery_messages	log
Developer Options	trace_sort	off
Developer Options	zero_damaged_pages	off
Error Handling	data_sync_retry	off
Error Handling	exit_on_error	off
Error Handling	restart_after_crash	on
File Locations	config_file	/var/opt/gitlab/postgresql/postgresql.conf
File Locations	data_directory	/var/opt/gitlab/postgresql/data
File Locations	external_pid_file
File Locations	hba_file	/var/opt/gitlab/postgresql/data/pg_hba.conf
File Locations	ident_file	/var/opt/gitlab/postgresql/pg_ident.conf
Lock Management	deadlock_timeout	5000	ms
Lock Management	max_locks_per_transaction	128
Lock Management	max_pred_locks_per_transaction	64
Preset Options	block_size	8192
Preset Options	data_checksums	off
Preset Options	debug_assertions	off
Preset Options	integer_datetimes	on
Preset Options	max_function_args	100
Preset Options	max_identifier_length	63
Preset Options	max_index_keys	32
Preset Options	segment_size	131072	8kB	1.00 GiB
Preset Options	server_version	9.6.15
Preset Options	server_version_num	90615
Preset Options	wal_block_size	8192
Preset Options	wal_segment_size	2048	8kB	16.00 MiB
Process Title	cluster_name	pg-ha-cluster
Process Title	update_process_title	on
Query Tuning / Genetic Query Optimizer	geqo	on
Query Tuning / Genetic Query Optimizer	geqo_effort	5
Query Tuning / Genetic Query Optimizer	geqo_generations	0
Query Tuning / Genetic Query Optimizer	geqo_pool_size	0
Query Tuning / Genetic Query Optimizer	geqo_seed	0
Query Tuning / Genetic Query Optimizer	geqo_selection_bias	2
Query Tuning / Genetic Query Optimizer	geqo_threshold	12
Query Tuning / Other Planner Options	constraint_exclusion	partition
Query Tuning / Other Planner Options	cursor_tuple_fraction	0.1
Query Tuning / Other Planner Options	default_statistics_target	1000
Query Tuning / Other Planner Options	force_parallel_mode	off
Query Tuning / Other Planner Options	from_collapse_limit	8
Query Tuning / Other Planner Options	join_collapse_limit	8
Query Tuning / Planner Cost Constants	cpu_index_tuple_cost	0.005
Query Tuning / Planner Cost Constants	cpu_operator_cost	0.0025
Query Tuning / Planner Cost Constants	cpu_tuple_cost	0.01
Query Tuning / Planner Cost Constants	effective_cache_size	43352064	8kB	330.75 GiB
Query Tuning / Planner Cost Constants	min_parallel_relation_size	1024	8kB	8.00 MiB
Query Tuning / Planner Cost Constants	parallel_setup_cost	1000
Query Tuning / Planner Cost Constants	parallel_tuple_cost	0.1
Query Tuning / Planner Cost Constants	random_page_cost	1.5
Query Tuning / Planner Cost Constants	seq_page_cost	4
Query Tuning / Planner Method Configuration	enable_bitmapscan	on
Query Tuning / Planner Method Configuration	enable_hashagg	on
Query Tuning / Planner Method Configuration	enable_hashjoin	on
Query Tuning / Planner Method Configuration	enable_indexonlyscan	on
Query Tuning / Planner Method Configuration	enable_indexscan	on
Query Tuning / Planner Method Configuration	enable_material	on
Query Tuning / Planner Method Configuration	enable_mergejoin	on
Query Tuning / Planner Method Configuration	enable_nestloop	on
Query Tuning / Planner Method Configuration	enable_seqscan	on
Query Tuning / Planner Method Configuration	enable_sort	on
Query Tuning / Planner Method Configuration	enable_tidscan	on
Replication	track_commit_timestamp	off
Replication / Master Server	synchronous_standby_names
Replication / Master Server	vacuum_defer_cleanup_age	0
Replication / Sending Servers	max_replication_slots	32
Replication / Sending Servers	max_wal_senders	32
Replication / Sending Servers	wal_keep_segments	2048
Replication / Sending Servers	wal_sender_timeout	60000	ms
Replication / Standby Servers	hot_standby	on
Replication / Standby Servers	hot_standby_feedback	on
Replication / Standby Servers	max_standby_archive_delay	30000	ms
Replication / Standby Servers	max_standby_streaming_delay	30000	ms
Replication / Standby Servers	wal_receiver_status_interval	10	s
Replication / Standby Servers	wal_receiver_timeout	60000	ms
Replication / Standby Servers	wal_retrieve_retry_interval	5000	ms
Reporting and Logging / What to Log	application_name	checkup
Reporting and Logging / What to Log	debug_pretty_print	on
Reporting and Logging / What to Log	debug_print_parse	off
Reporting and Logging / What to Log	debug_print_plan	off
Reporting and Logging / What to Log	debug_print_rewritten	off
Reporting and Logging / What to Log	log_autovacuum_min_duration	0	ms
Reporting and Logging / What to Log	log_checkpoints	on
Reporting and Logging / What to Log	log_connections	off
Reporting and Logging / What to Log	log_disconnections	off
Reporting and Logging / What to Log	log_duration	off
Reporting and Logging / What to Log	log_error_verbosity	default
Reporting and Logging / What to Log	log_hostname	off
Reporting and Logging / What to Log	log_line_prefix	%m [%p, %x]: [%l-1] user=%u, db=%d, app=%a, client=%h
Reporting and Logging / What to Log	log_lock_waits	on
Reporting and Logging / What to Log	log_replication_commands	off
Reporting and Logging / What to Log	log_statement	ddl
Reporting and Logging / What to Log	log_temp_files	0	kB	0.00 bytes
Reporting and Logging / What to Log	log_timezone	GMT
Reporting and Logging / When to Log	log_min_duration_statement	1000	ms
Reporting and Logging / When to Log	log_min_error_statement	error
Reporting and Logging / When to Log	log_min_messages	warning
Reporting and Logging / Where to Log	event_source	PostgreSQL
Reporting and Logging / Where to Log	log_destination	stderr
Reporting and Logging / Where to Log	log_directory	/var/log/gitlab/postgresql
Reporting and Logging / Where to Log	log_file_mode	0640
Reporting and Logging / Where to Log	log_filename	postgresql.log
Reporting and Logging / Where to Log	log_rotation_age	0	min
Reporting and Logging / Where to Log	log_rotation_size	0	kB	0.00 bytes
Reporting and Logging / Where to Log	log_truncate_on_rotation	off
Reporting and Logging / Where to Log	logging_collector	on
Reporting and Logging / Where to Log	syslog_facility	local0
Reporting and Logging / Where to Log	syslog_ident	postgres
Reporting and Logging / Where to Log	syslog_sequence_numbers	on
Reporting and Logging / Where to Log	syslog_split_messages	on
Resource Usage / Asynchronous Behavior	backend_flush_after	0	8kB	0.00 bytes
Resource Usage / Asynchronous Behavior	effective_io_concurrency	100
Resource Usage / Asynchronous Behavior	max_parallel_workers_per_gather	0
Resource Usage / Asynchronous Behavior	max_worker_processes	8
Resource Usage / Asynchronous Behavior	old_snapshot_threshold	-1	min
Resource Usage / Background Writer	bgwriter_delay	200	ms
Resource Usage / Background Writer	bgwriter_flush_after	64	8kB	512.00 KiB
Resource Usage / Background Writer	bgwriter_lru_maxpages	100
Resource Usage / Background Writer	bgwriter_lru_multiplier	2
Resource Usage / Cost-Based Vacuum Delay	vacuum_cost_delay	0	ms
Resource Usage / Cost-Based Vacuum Delay	vacuum_cost_limit	200
Resource Usage / Cost-Based Vacuum Delay	vacuum_cost_page_dirty	20
Resource Usage / Cost-Based Vacuum Delay	vacuum_cost_page_hit	1
Resource Usage / Cost-Based Vacuum Delay	vacuum_cost_page_miss	10
Resource Usage / Disk	temp_file_limit	-1	kB
Resource Usage / Kernel Resources	max_files_per_process	1000
Resource Usage / Memory	autovacuum_work_mem	-1	kB
Resource Usage / Memory	dynamic_shared_memory_type	posix
Resource Usage / Memory	huge_pages	try
Resource Usage / Memory	maintenance_work_mem	2097152	kB	2.00 GiB
Resource Usage / Memory	max_prepared_transactions	0
Resource Usage / Memory	max_stack_depth	2048	kB	2.00 MiB
Resource Usage / Memory	replacement_sort_tuples	150000
Resource Usage / Memory	shared_buffers	14450688	8kB	110.25 GiB
Resource Usage / Memory	temp_buffers	1024	8kB	8.00 MiB
Resource Usage / Memory	track_activity_query_size	1024
Resource Usage / Memory	work_mem	102400	kB	100.00 MiB
Statistics / Monitoring	log_executor_stats	off
Statistics / Monitoring	log_parser_stats	off
Statistics / Monitoring	log_planner_stats	off
Statistics / Monitoring	log_statement_stats	off
Statistics / Query and Index Statistics Collector	stats_temp_directory	pg_stat_tmp
Statistics / Query and Index Statistics Collector	track_activities	on
Statistics / Query and Index Statistics Collector	track_counts	on
Statistics / Query and Index Statistics Collector	track_functions	none
Statistics / Query and Index Statistics Collector	track_io_timing	on
Version and Platform Compatibility / Other Platforms and Clients	transform_null_equals	off
Version and Platform Compatibility / Previous PostgreSQL Versions	array_nulls	on
Version and Platform Compatibility / Previous PostgreSQL Versions	backslash_quote	safe_encoding
Version and Platform Compatibility / Previous PostgreSQL Versions	default_with_oids	off
Version and Platform Compatibility / Previous PostgreSQL Versions	escape_string_warning	on
Version and Platform Compatibility / Previous PostgreSQL Versions	lo_compat_privileges	off
Version and Platform Compatibility / Previous PostgreSQL Versions	operator_precedence_warning	off
Version and Platform Compatibility / Previous PostgreSQL Versions	quote_all_identifiers	off
Version and Platform Compatibility / Previous PostgreSQL Versions	sql_inheritance	on
Version and Platform Compatibility / Previous PostgreSQL Versions	standard_conforming_strings	on
Version and Platform Compatibility / Previous PostgreSQL Versions	synchronize_seqscans	on
Write-Ahead Log / Archiving	archive_command	/usr/bin/envdir /etc/wal-e.d/env /opt/wal-e/bin/wal-e wal-push %p
Write-Ahead Log / Archiving	archive_mode	on
Write-Ahead Log / Archiving	archive_timeout	60	s
Write-Ahead Log / Checkpoints	checkpoint_completion_target	0.7
Write-Ahead Log / Checkpoints	checkpoint_flush_after	32	8kB	256.00 KiB
Write-Ahead Log / Checkpoints	checkpoint_timeout	300	s
Write-Ahead Log / Checkpoints	checkpoint_warning	30	s
Write-Ahead Log / Checkpoints	max_wal_size	320	16MB	5.00 GiB
Write-Ahead Log / Checkpoints	min_wal_size	5	16MB	80.00 MiB
Write-Ahead Log / Settings	commit_delay	0
Write-Ahead Log / Settings	commit_siblings	5
Write-Ahead Log / Settings	fsync	on
Write-Ahead Log / Settings	full_page_writes	on
Write-Ahead Log / Settings	synchronous_commit	on
Write-Ahead Log / Settings	wal_buffers	2048	8kB	16.00 MiB
Write-Ahead Log / Settings	wal_compression	off
Write-Ahead Log / Settings	wal_level	replica
Write-Ahead Log / Settings	wal_log_hints	on
Write-Ahead Log / Settings	wal_sync_method	fdatasync
Write-Ahead Log / Settings	wal_writer_delay	200	ms
Write-Ahead Log / Settings	wal_writer_flush_after	128	8kB	1.00 MiB

---

  Table of contents

A004 Cluster Information

Observations

Data collected: 2019-09-30 13:55:25 +0000 UTC

▼ Indicator	10.220.16.101	10.220.16.102	10.220.16.103	10.220.16.104	10.220.16.105	10.220.16.106	10.220.16.107	10.220.16.108	10.220.16.109
Config file	/var/opt/gitlab/postgresql/postgresql.conf	/var/opt/gitlab/postgresql/postgresql.conf	/var/opt/gitlab/postgresql/postgresql.conf	/var/opt/gitlab/postgresql/postgresql.conf	/var/opt/gitlab/postgresql/postgresql.conf	/var/opt/gitlab/postgresql/postgresql.conf	/var/opt/gitlab/postgresql/postgresql.conf	/var/opt/gitlab/postgresql/postgresql.conf	/var/opt/gitlab/postgresql/postgresql.conf
Role	Master	Replica (delay: 00:00:00; paused: false)	Replica (delay: 00:00:00; paused: false)	Replica (delay: 00:00:00; paused: false)	Replica (delay: 00:00:00; paused: false)	Replica (delay: 00:00:00; paused: false)	Replica (delay: 00:00:00; paused: false)	Replica (delay: 00:00:00; paused: false)	Replica (delay: 00:00:00; paused: false)
Replicas	async/streaming: 10.220.16.111, 10.220.16.106, 10.220.16.110, 10.220.16.103, 10.220.16.108, 10.220.16.105, 10.220.16.102, 10.220.16.107, 10.220.16.109, 10.220.16.112, 10.220.16.104
Started At	2019-09-09 18:31:16+00	2019-09-09 20:34:44+00	2019-09-09 20:34:37+00	2019-09-09 20:34:35+00	2019-09-09 20:34:42+00	2019-09-09 20:34:06+00	2019-09-09 20:34:44+00	2019-09-09 23:09:19+00	2019-09-09 20:34:25+00
Uptime	20 days 19:05:26	20 days 17:04:07	20 days 17:05:25	20 days 17:06:52	20 days 17:08:33	20 days 17:11:20	20 days 17:13:31	20 days 14:42:06	20 days 17:21:01
Checkpoints	12025	311118	310047	311082	311010	308270	308535	307732	307875
Forced Checkpoints	77.5%	99.6%	99.6%	99.6%	99.6%	99.6%	99.6%	99.6%	99.6%
Checkpoint MB/sec	13.047043	14.911349	14.913182	14.914270	14.915614	14.914721	14.915471	14.957235	14.919222
Database Name	gitlabhq_production	gitlabhq_production	gitlabhq_production	gitlabhq_production	gitlabhq_production	gitlabhq_production	gitlabhq_production	gitlabhq_production	gitlabhq_production
Database Size	4079 GB	4079 GB	4079 GB	4079 GB	4079 GB	4079 GB	4080 GB	4080 GB	4080 GB
Stats Since	2019-09-09 18:31:36+00	2019-09-09 20:34:45+00	2019-09-09 20:34:37+00	2019-09-09 20:34:35+00	2019-09-09 20:34:43+00	2019-09-09 20:34:09+00	2019-09-09 20:34:46+00	2019-09-09 23:15:42+00	2019-09-09 20:34:25+00
Stats Age	20 days 19:05:06	20 days 17:04:05	20 days 17:05:25	20 days 17:06:52	20 days 17:08:32	20 days 17:11:17	20 days 17:13:28	20 days 14:35:43	20 days 17:21:01
Cache Effectiveness	99.39%	99.82%	99.81%	99.82%	99.82%	99.82%	99.81%	99.81%	99.81%
Successful Commits	99.71%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%
Conflicts	0	0	0	1	0	3	0	0	0
Temp Files: total size	138 GB	1202 GB	1195 GB	1191 GB	1202 GB	1181 GB	1201 GB	1181 GB	1232 GB
Temp Files: total number of files	847	6825	6823	6772	6756	6665	6919	6663	7113
Temp Files: total number of files per day	40	325	324	322	321	317	329	317	338
Temp Files: avg file size	167 MB	180 MB	179 MB	180 MB	182 MB	182 MB	178 MB	182 MB	177 MB
Deadlocks	20	0	0	0	0	0	0	0	0
Deadlocks per day	0	0	0	0	0	0	0	0	0

Databases sizes

Database	▼ Size
gitlabhq_production	3.99 TiB
postgres	6.99 MiB
template0	6.91 MiB
template1	6.91 MiB

Conclusions

Recommendations

---

  Table of contents

A005 Extensions

Observations

Data collected: 2019-09-30 13:55:27 +0000 UTC

Master (10.220.16.101)

▼ Database	Extension name	Installed version	Default version	Is old
gitlabhq_production	btree_gin	1.0	1.0
gitlabhq_production	dblink	1.2	1.2
gitlabhq_production	pageinspect	1.5	1.5
gitlabhq_production	pg_buffercache	1.2	1.2
gitlabhq_production	pg_repack	1.4.4	1.4.4
gitlabhq_production	pg_stat_statements	1.4	1.4
gitlabhq_production	pg_trgm	1.3	1.3
gitlabhq_production	pgstattuple	1.4	1.4
gitlabhq_production	plpgsql	1.0	1.0

Conclusions

Recommendations

---

  Table of contents

A006 Postgres Setting Deviations

Observations

Data collected: 2019-09-30 13:55:29 +0000 UTC

Settings (pg_settings) that Differ

▼ Setting	10.220.16.101	10.220.16.102	10.220.16.103	10.220.16.104	10.220.16.105	10.220.16.106	10.220.16.107	10.220.16.108	10.220.16.109
server_version	9.6.15	9.6.14	9.6.14	9.6.14	9.6.14	9.6.14	9.6.15	9.6.15	9.6.15
server_version_num	90615	90614	90614	90614	90614	90614	90615	90615	90615

Configs(pg_config) that differ

▼ Config	10.220.16.101	10.220.16.102	10.220.16.103	10.220.16.104	10.220.16.105	10.220.16.106	10.220.16.107	10.220.16.108	10.220.16.109
CONFIGURE	'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.15-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'	'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'	'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'	'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'	'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'	'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'	'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.15-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'	'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.15-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'	'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.15-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
VERSION	PostgreSQL 9.6.15	PostgreSQL 9.6.14	PostgreSQL 9.6.14	PostgreSQL 9.6.14	PostgreSQL 9.6.14	PostgreSQL 9.6.14	PostgreSQL 9.6.15	PostgreSQL 9.6.15	PostgreSQL 9.6.15

Conclusions

Recommendations

---

  Table of contents

A007 Altered Settings

Observations

Data collected: 2019-09-30 13:56:15 +0000 UTC

Master (10.220.16.101)

Source	Settings count	Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf	4	autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf	2	autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf	47	archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default	213

Replica servers:

Replica (10.220.16.102)

Source	Settings count	Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf	4	autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf	2	autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf	47	archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default	213

Replica (10.220.16.103)

Source	Settings count	Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf	4	autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf	2	autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf	47	archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default	213

Replica (10.220.16.104)

Source	Settings count	Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf	4	autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf	2	autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf	47	archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default	213

Replica (10.220.16.105)

Source	Settings count	Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf	4	autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf	2	autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf	47	archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default	213

Replica (10.220.16.106)

Source	Settings count	Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf	4	autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf	2	autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf	47	archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default	213

Replica (10.220.16.107)

Source	Settings count	Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf	4	autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf	2	autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf	47	archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default	213

Replica (10.220.16.108)

Source	Settings count	Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf	4	autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf	2	autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf	47	archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default	213

Replica (10.220.16.109)

Source	Settings count	Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf	4	autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf	2	autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf	47	archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default	213

Conclusions

Recommendations

---

  Table of contents

D004 pg_stat_statements and pg_stat_kcache Settings

Observations

Data collected: 2019-09-30 13:56:18 +0000 UTC

Master (10.220.16.101)

pg_stat_statements extension settings

Setting	Value	Unit	Type	Min value	Max value
pg_stat_statements.max	5000		integer	100	2147483647
pg_stat_statements.save	on		bool
pg_stat_statements.track	top		enum
pg_stat_statements.track_utility	on		bool

Conclusions

Recommendations

---

  Table of contents

F001 Autovacuum: Current Settings

Observations

Data collected: 2019-09-30 13:56:20 +0000 UTC

Master (10.220.16.101)

▼ Setting name	Value	Unit	Pretty value
autovacuum	on
autovacuum_analyze_scale_factor	0.005
autovacuum_analyze_threshold	50
autovacuum_freeze_max_age	200000000
autovacuum_max_workers	6
autovacuum_multixact_freeze_max_age	400000000
autovacuum_naptime	30	s
autovacuum_vacuum_cost_delay	5	ms
autovacuum_vacuum_cost_limit	6000
autovacuum_vacuum_scale_factor	0.005
autovacuum_vacuum_threshold	50
autovacuum_work_mem	-1	kB
maintenance_work_mem	2097152	kB	2.00 GiB
vacuum_cost_delay	0	ms
vacuum_cost_limit	200
vacuum_cost_page_dirty	20
vacuum_cost_page_hit	1
vacuum_cost_page_miss	10
vacuum_defer_cleanup_age	0
vacuum_freeze_min_age	50000000
vacuum_freeze_table_age	150000000
vacuum_multixact_freeze_min_age	5000000
vacuum_multixact_freeze_table_age	150000000

Tuned tables

▼ Namespace	Relation	Options
pg_toast	pg_toast_33145	autovacuum_enabled=true
pg_toast	pg_toast_33153	autovacuum_enabled=true
pg_toast	pg_toast_33161	autovacuum_enabled=true
pg_toast	pg_toast_33208	autovacuum_enabled=true
pg_toast	pg_toast_33221	autovacuum_enabled=true
pg_toast	pg_toast_33229	autovacuum_enabled=true
pg_toast	pg_toast_33237	autovacuum_enabled=true
pg_toast	pg_toast_33248	autovacuum_enabled=true
pg_toast	pg_toast_33262	autovacuum_enabled=true
pg_toast	pg_toast_33272	autovacuum_enabled=true
pg_toast	pg_toast_33322	autovacuum_enabled=true
pg_toast	pg_toast_33356	autovacuum_enabled=true
pg_toast	pg_toast_33369	autovacuum_enabled=true
pg_toast	pg_toast_33383	autovacuum_enabled=true
pg_toast	pg_toast_33393	autovacuum_enabled=true
pg_toast	pg_toast_33400	autovacuum_enabled=true
pg_toast	pg_toast_33426	autovacuum_enabled=true
pg_toast	pg_toast_33439	autovacuum_enabled=true
pg_toast	pg_toast_33447	autovacuum_enabled=true
pg_toast	pg_toast_33462	autovacuum_enabled=true
pg_toast	pg_toast_33476	autovacuum_enabled=true
pg_toast	pg_toast_33497	autovacuum_enabled=true
pg_toast	pg_toast_33506	autovacuum_enabled=true
pg_toast	pg_toast_33514	autovacuum_enabled=true
pg_toast	pg_toast_33527	autovacuum_enabled=true
pg_toast	pg_toast_33541	autovacuum_enabled=true
pg_toast	pg_toast_33553	autovacuum_enabled=true
pg_toast	pg_toast_33568	autovacuum_enabled=true
pg_toast	pg_toast_33587	autovacuum_enabled=true
pg_toast	pg_toast_33597	autovacuum_enabled=true
pg_toast	pg_toast_33614	autovacuum_enabled=true
pg_toast	pg_toast_33625	autovacuum_enabled=true
pg_toast	pg_toast_33634	autovacuum_enabled=true
pg_toast	pg_toast_33642	autovacuum_enabled=true
pg_toast	pg_toast_33650	autovacuum_enabled=true
pg_toast	pg_toast_33659	autovacuum_enabled=true
pg_toast	pg_toast_33667	autovacuum_enabled=true
pg_toast	pg_toast_33675	autovacuum_enabled=true
pg_toast	pg_toast_33696	autovacuum_enabled=true
pg_toast	pg_toast_33706	autovacuum_enabled=true
pg_toast	pg_toast_33749	autovacuum_enabled=true
pg_toast	pg_toast_33756	autovacuum_enabled=true
pg_toast	pg_toast_33768	autovacuum_enabled=true
pg_toast	pg_toast_33776	autovacuum_enabled=true
pg_toast	pg_toast_33791	autovacuum_enabled=true
pg_toast	pg_toast_33801	autovacuum_enabled=true
pg_toast	pg_toast_33824	autovacuum_enabled=true
pg_toast	pg_toast_33835	autovacuum_enabled=true
pg_toast	pg_toast_33851	autovacuum_enabled=true
pg_toast	pg_toast_33863	autovacuum_enabled=true
pg_toast	pg_toast_33872	autovacuum_enabled=true
pg_toast	pg_toast_33885	autovacuum_enabled=true
pg_toast	pg_toast_33898	autovacuum_enabled=true
pg_toast	pg_toast_33909	autovacuum_enabled=true
pg_toast	pg_toast_33959	autovacuum_enabled=true
public	abuse_reports	autovacuum_enabled=true
public	appearances	autovacuum_enabled=true
public	application_settings	autovacuum_enabled=true
public	approval_merge_request_rules_users	autovacuum_enabled=on
public	approvals	autovacuum_enabled=true
public	approver_groups	autovacuum_enabled=true
public	approvers	autovacuum_enabled=on
public	audit_events	autovacuum_enabled=true
public	award_emoji	autovacuum_enabled=on
public	boards	autovacuum_enabled=true
public	broadcast_messages	autovacuum_enabled=true
public	ci_builds	autovacuum_enabled=true
public	ci_pipelines	autovacuum_enabled=true
public	ci_runner_projects	autovacuum_enabled=true
public	ci_runners	autovacuum_enabled=on
public	ci_trigger_requests	autovacuum_enabled=true
public	ci_triggers	autovacuum_enabled=true
public	ci_variables	autovacuum_enabled=true
public	deploy_keys_projects	autovacuum_enabled=true
public	deployments	autovacuum_enabled=on
public	emails	autovacuum_enabled=true
public	environments	autovacuum_enabled=true
public	forked_project_links	autovacuum_enabled=true
public	geo_nodes	autovacuum_enabled=true
public	historical_data	autovacuum_enabled=true
public	identities	autovacuum_enabled=true
public	index_statuses	autovacuum_enabled=true
public	issue_metrics	autovacuum_enabled=true
public	issues	autovacuum_enabled=true
public	keys	autovacuum_enabled=true
public	label_links	autovacuum_enabled=true
public	label_priorities	autovacuum_enabled=true
public	labels	autovacuum_enabled=on
public	ldap_group_links	autovacuum_enabled=true
public	lfs_objects	autovacuum_enabled=true
public	lfs_objects_projects	autovacuum_enabled=true
public	licenses	autovacuum_enabled=true
public	lists	autovacuum_enabled=true
public	members	autovacuum_enabled=true
public	merge_request_diffs	autovacuum_enabled=true
public	merge_request_metrics	autovacuum_enabled=true
public	merge_requests	autovacuum_enabled=true
public	merge_requests_closing_issues	autovacuum_enabled=true
public	milestones	autovacuum_enabled=on
public	namespaces	autovacuum_enabled=on
public	notes	autovacuum_enabled=true
public	notification_settings	autovacuum_enabled=on
public	oauth_access_grants	autovacuum_enabled=true
public	oauth_access_tokens	autovacuum_enabled=true
public	oauth_applications	autovacuum_enabled=true
public	pages_domains	autovacuum_enabled=true
public	path_locks	autovacuum_enabled=true
public	personal_access_tokens	autovacuum_enabled=true
public	project_authorizations	autovacuum_enabled=true
public	project_features	autovacuum_enabled=true
public	project_group_links	autovacuum_enabled=true
public	project_import_data	autovacuum_enabled=on
public	project_mirror_data	autovacuum_vacuum_cost_limit=10000 autovacuum_enabled=on
public	project_repository_states	autovacuum_enabled=on
public	projects	autovacuum_vacuum_cost_limit=7000 autovacuum_enabled=on
public	protected_branch_merge_access_levels	autovacuum_enabled=true
public	protected_branch_push_access_levels	autovacuum_enabled=true
public	protected_branches	autovacuum_enabled=true
public	push_rules	autovacuum_enabled=true
public	releases	autovacuum_enabled=on
public	remote_mirrors	autovacuum_enabled=true
public	sent_notifications	autovacuum_enabled=true
public	services	autovacuum_enabled=on
public	snippets	autovacuum_enabled=on
public	spam_logs	autovacuum_enabled=true
public	subscriptions	autovacuum_enabled=true
public	taggings	autovacuum_enabled=true
public	tags	autovacuum_enabled=true
public	todos	autovacuum_enabled=true
public	trending_projects	autovacuum_enabled=true
public	u2f_registrations	autovacuum_enabled=true
public	untracked_files_for_uploads	autovacuum_enabled=on
public	user_agent_details	autovacuum_enabled=true
public	users	autovacuum_enabled=on
public	users_star_projects	autovacuum_enabled=true
public	web_hooks	autovacuum_enabled=true

Replicas settings

Setting	10.220.16.102	10.220.16.103	10.220.16.104	10.220.16.105	10.220.16.106	10.220.16.107	10.220.16.108	10.220.16.109
hot_standby_feedback	on	on	on	on	on	on	on	on

Conclusions

Recommendations

All good, no recommendations here.

---

  Table of contents

F002 Autovacuum: Transaction ID Wraparound Check

Observations

• Data collected: 2019-09-30 13:36:50 +0000 UTC
• Current database: gitlabhq_production

Master (10.220.16.101)

Databases

#	Database	▼ Age	Capacity used, %	Warning	datfrozenxid
1	gitlabhq_production	199995868	10.26		160538665
2	postgres	195569077	10.03		164965456
3	template0	47497946	2.44		313036587
4	template1	47295719	2.43		313238814

Tables in the observed database

The list is limited to 10 items. Total: 268.

#	Relation	Age	▼ Capacity used, %	Warning	rel_relfrozenxid	toast_relfrozenxid
1	notification_settings*	199995868	10.26		160538665	163739001
2	slack_integrations	199995868	10.26		160538665	163739001
3	gpg_key_subkeys	199878040	10.25		160656493	163739001
4	ci_pipeline_chat_data	199878040	10.25		160656493	163779468
5	container_repositories	199932034	10.25		160602499	163739001
6	operations_feature_flags_clients	199675759	10.24		160858774	164003145
7	ci_runner_projects*	199586477	10.24		160948056	0
8	issue_assignees	199678708	10.24		160855825	0
9	schema_migrations	199108026	10.21		161426507	161441143
10	insights	199065876	10.21		161468657	0

* This table has specific autovacuum settings. See 'F001 Autovacuum: Current settings'

Conclusions

Recommendations

All good, no recommendations here.

---

  Table of contents

F003 Autovacuum: Dead Tuples

Observations

Data collected: 2019-09-30 13:36:52 +0000 UTC
Current database: gitlabhq_production
Stats reset: 20 days 19:05:00 ago (2019-09-09 18:31:36 +0000 UTC)

Master (10.220.16.101)

The list is limited to 20 items. Total: 184.

#	Relation	reltype	Since last autovacuum	Since last vacuum	Autovacuum Count	Vacuum Count	n_tup_ins	n_tup_upd	n_tup_del	pg_class.reltuples	n_live_tup	n_dead_tup	▼Dead Tuples Ratio, %
1	project_mirror_data*	r	00:03:36.655815		5790	0	60052	335213555	15861	2157317	2157323	100263	4.44
2	namespace_aggregation_schedules	r	00:00:21.549297		61912	0	8141635	0	7992341	17748	17722	299	1.66
3	pg_toast.pg_toast_2619	t	00:00:58.960839		25505	0	4224019	0	4223631	15237	15192	174	1.13
4	pg_catalog.pg_statistic	r	00:04:59.138629		6705	0	47	942835	1	14654	14654	123	0.83
5	u2f_registrations*	r	04:40:29.220464		50	0	835	6293	320	15282	15283	118	0.77
6	protected_tag_create_access_levels	r	2 days 14:38:59.570444		8	0	2051	51	1191	18419	18421	138	0.74
7	draft_notes	r	00:07:29.114059		730	0	47639	5932	46799	11678	11678	78	0.66
8	cluster_platforms_kubernetes	r	1 day 13:31:59.386141		14	0	2406	1562	1490	24177	24193	153	0.63
9	cluster_providers_gcp	r	18:17:52.980122		15	0	1104	1811	739	10041	10045	62	0.61
10	clusters_kubernetes_namespaces	r	3 days 00:19:17.261235		8	0	2710	583	1063	30921	30991	182	0.58
11	pg_toast.pg_toast_54153184	t	1 day 15:01:46.4009		14	0	5070	0	3234	34129	34261	186	0.54
12	pg_toast.pg_toast_33659*	t	1 day 10:18:18.825141		15	0	12476	0	9282	108336	108492	560	0.51
13	resource_label_events	r	05:02:01.537767		33	0	2196074	3579123	318368	22591948	22623125	112879	0.5
14	environments*	r	2 days 19:31:56.355287		15	0	57572	74430	10311	810728	811268	4094	0.5
15	lfs_objects*	r	3 days 08:19:18.500374		11	0	957167	956027	0	17733184	17737868	87701	0.49
16	users*	r	00:40:32.833542		419	0	118720	11904407	2184	4504403	4504453	21893	0.48
17	pg_toast.pg_toast_54153124	t	1 day 04:19:14.297056		15	0	3319	0	2422	21561	21618	102	0.47
18	gpg_key_subkeys	r	4 days 12:08:56.991402		4	0	1847	0	705	38665	38702	182	0.47
19	badges	r	2 days 23:46:21.008523		7	0	2568	1323	707	26499	26534	125	0.47
20	protected_branch_unprotect_access_levels	r	00:16:06.698542		576	0	64142	0	62643	21923	21923	101	0.46

* This table has specific autovacuum settings. See 'F001 Autovacuum: Current settings'

Conclusions

Recommendations

---

  Table of contents

F004 Autovacuum: Heap Bloat (Estimated)

This report is based on estimations. The errors in bloat estimates may be significant (in some cases, up to 15% and even more). Use it only as an indicator of potential issues.

Observations

Data collected: 2019-09-30 13:36:54 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.101)

The list is limited to 20 items. Total: 274.

#	Table	Real Size	▼ Estimated bloat	Est. bloat, bytes	Est. bloat factor	Est. bloat level, %	Live Data Size	Last vacuum	Fillfactor
	===== TOTAL =====	2.77 TiB	63.57 GiB	68,251,901,952	1.02	2.25	~2.71 TiB
1	merge_request_diff_files	1.25 TiB	25.40 GiB	27,268,497,408	1.02	2.00	~1.22 TiB	2019-09-27 06:46:37 (auto)	100
2	ci_builds*	401.32 GiB	11.85 GiB	12,716,433,408	1.03	2.95	~389.48 GiB	2019-09-30 09:33:19 (auto)	100
3	merge_request_diff_commits	500.65 GiB	10.03 GiB	10,760,568,832	1.02	2.00	~490.62 GiB	2019-09-26 04:35:43 (auto)	100
4	sent_notifications*	30.77 GiB	2.08 GiB	2,227,716,096	1.07	6.74	~28.69 GiB	2019-09-26 05:45:43 (auto)	100
5	notes*	130.03 GiB	1.46 GiB	1,561,468,928	1.01	1.12	~128.57 GiB	2019-09-30 03:12:35 (auto)	100
6	audit_events*	42.46 GiB	1.44 GiB	1,543,290,880	1.04	3.39	~41.02 GiB	2019-09-26 04:02:37 (auto)	100
7	ci_pipelines*	16.48 GiB	1.29 GiB	1,377,558,528	1.08	7.79	~15.20 GiB	2019-09-30 03:25:07 (auto)	100
8	ci_job_artifacts	36.69 GiB	1.14 GiB	1,220,296,704	1.03	3.10	~35.55 GiB	2019-09-30 12:14:32 (auto)	100
9	push_event_payloads	39.41 GiB	1.09 GiB	1,170,145,280	1.03	2.77	~38.32 GiB	2019-09-27 10:11:06 (auto)	100
10	merge_request_diffs*	10.47 GiB	0.96 GiB	1,020,944,384	1.10	9.09	~9.52 GiB	2019-09-27 10:17:04 (auto)	100
11	merge_requests*	14.45 GiB	856.08 MiB	897,654,784	1.06	5.79	~13.61 GiB	2019-09-30 12:31:38 (auto)	100
12	ci_stages	11.69 GiB	583.77 MiB	612,122,624	1.05	4.88	~11.12 GiB	2019-09-30 10:19:23 (auto)	100
13	ci_build_trace_sections	93.73 GiB	537.68 MiB	563,789,824	1.01	0.56	~93.21 GiB	2019-09-26 04:07:43 (auto)	100
14	deployments*	4.91 GiB	435.74 MiB	456,900,608	1.09	8.67	~4.48 GiB	2019-09-28 21:54:36 (auto)	100
15	projects*	3.06 GiB	380.86 MiB	399,351,808	1.14	12.19	~2.68 GiB	2019-09-30 13:29:09 (auto)	100
16	resource_label_events	9.69 GiB	332.11 MiB	348,233,728	1.03	3.35	~9.37 GiB	2019-09-30 08:34:52 (auto)	100
17	project_mirror_data*	564.65 MiB	312.67 MiB	327,852,032	2.24	55.37	~251.99 MiB	2019-09-30 13:33:17 (auto)	100
18	todos*	2.94 GiB	249.47 MiB	261,586,944	1.09	8.29	~2.70 GiB	2019-09-30 08:58:08 (auto)	100
19	lfs_objects*	3.35 GiB	239.42 MiB	251,043,840	1.08	6.98	~3.12 GiB	2019-09-27 05:17:35 (auto)	100
20	users*	1.95 GiB	225.14 MiB	236,068,864	1.13	11.28	~1.73 GiB	2019-09-30 12:56:20 (auto)	100

* This table has specific autovacuum settings. See 'F001 Autovacuum: Current settings'

Conclusions

• The estimated table (heap) bloat in this DB is low, just ~2.25% (~63.57 GiB). No action is needed now. Keep watching it though.

• [P1] The following 2 tables have significant size (>1 MiB) and bloat estimate > 90.00%:

  • ci_build_trace_chunks: size 4.34 MiB, can be reduced 15.00 times, by ~4.05 MiB (~93.33%)
  • appearances: size 1.37 MiB, can be reduced 175.00 times, by ~1.36 MiB (~99.43%)

• [P2] There are 6 tables with size > 1 MiB and table bloat estimate >= 40.00% and < 90.00%:

  • project_mirror_data: size 564.65 MiB, can be reduced 2.24 times, by ~312.67 MiB (~55.37%)
  • user_preferences: size 296.29 MiB, can be reduced 2.00 times, by ~148.35 MiB (~50.07%)
  • user_callouts: size 27.81 MiB, can be reduced 1.81 times, by ~12.48 MiB (~44.87%)
  • clusters: size 4.36 MiB, can be reduced 1.78 times, by ~1.91 MiB (~43.73%)
  • lfs_file_locks: size 1.82 MiB, can be reduced 6.44 times, by ~1.54 MiB (~84.48%)
  • etc.

Recommendations

• [P1] Reduce and prevent the high level of table bloat:

  • to prevent a high level of bloat in the future, tune autovacuum: consider more aggressive autovacuum settings (see F001);
  • eliminate or reduce the current table bloat using one of the approaches listed below.

• If you want to get exact bloat numbers, clone the database, get table sizes, then apply database-wide VACUUM FULL (it eliminate all the bloat), and get new table sizes. Then compare old and new numbers.

• To reduce the table bloat, consider one of the following approaches:

  • VACUUM FULL ( requires downtime / maintenance window),
  • one of the tools reducing the bloat online, without interrupting the operations:
    • pg_repack,
    • pg_squeeze,
    • pgcompacttable.

• Read more on this topic:

  • Bloat estimation for tables (2014, ioguix)
  • Show database bloat (PostgreSQL wiki)
  • PostgreSQL Bloat: origins, monitoring and managing (2016, Compose)
  • Dealing with significant Postgres database bloat — what are your options? (2018, Compass)
  • Postgres database bloat analysis (2019, GitLab)

---

  Table of contents

F005 Autovacuum: Btree Index Bloat (Estimated)

This report is based on estimations. The errors in bloat estimates may be significant (in some cases, up to 15% and even more). Use it only as an indicator of potential issues.

Observations

Data collected: 2019-09-30 13:36:59 +0000 UTC
Current database: gitlabhq_production
This report considers only btree indexes.

Master (10.220.16.101)

The list is limited to 20 items. Total: 947.

#	Index (Table)	Index Size	Table Size	▼ Estimated bloat	Est. bloat, bytes	Est. bloat factor	Est. bloat level, %	Live Data Size	Fillfactor
	===== TOTAL =====	1.14 TiB	5.00 TiB	291.33 GiB	312,812,339,200	1.34	25.17	866.11 GiB
1	index_merge_request_diff_commits_on_sha (merge_request_diff_commits)	86.88 GiB	488.89 GiB	13.02 GiB	13,975,683,072	1.18	14.98	~73.87 GiB	90
2	index_ci_builds_on_auto_canceled_by_id (ci_builds*)	15.39 GiB	291.64 GiB	11.73 GiB	12,588,113,920	4.20	76.19	~3.67 GiB	90
3	index_ci_builds_on_protected (ci_builds*)	15.22 GiB	291.64 GiB	9.19 GiB	9,861,808,128	2.52	60.38	~6.03 GiB	90
4	index_ci_builds_on_token_encrypted (ci_builds*)	16.15 GiB	291.64 GiB	8.87 GiB	9,518,219,264	2.22	54.89	~7.29 GiB	90
5	index_ci_builds_on_token (ci_builds*)	17.01 GiB	291.64 GiB	8.65 GiB	9,279,373,312	2.03	50.83	~8.36 GiB	90
6	index_ci_builds_on_runner_id (ci_builds*)	14.32 GiB	291.64 GiB	8.35 GiB	8,957,657,088	2.40	58.26	~5.98 GiB	90
7	index_ci_job_artifacts_on_expire_at_and_job_id (ci_job_artifacts)	12.06 GiB	36.70 GiB	6.88 GiB	7,377,879,040	2.33	56.99	~5.19 GiB	90
8	tmp_build_stage_position_index (ci_builds*)	15.17 GiB	291.64 GiB	6.71 GiB	7,203,569,664	1.79	44.24	~8.46 GiB	90
9	index_ci_builds_on_queued_at (ci_builds*)	12.70 GiB	291.64 GiB	6.66 GiB	7,144,341,504	2.10	52.41	~6.05 GiB	90
10	index_ci_builds_on_updated_at (ci_builds*)	12.62 GiB	291.64 GiB	6.55 GiB	7,025,917,952	2.08	51.85	~6.08 GiB	90
12	index_ci_builds_on_project_id_and_id (ci_builds*)	12.14 GiB	291.64 GiB	6.06 GiB	6,506,594,304	2.00	49.94	~6.08 GiB	90
13	ci_builds_pkey (ci_builds*)	11.61 GiB	291.64 GiB	5.54 GiB	5,937,864,704	1.91	47.65	~6.08 GiB	90
14	index_notes_on_discussion_id (notes*)	15.56 GiB	122.37 GiB	5.33 GiB	5,713,068,032	1.52	34.21	~10.24 GiB	90
15	index_ci_build_trace_sections_on_section_name_id (ci_build_trace_sections)	32.30 GiB	94.19 GiB	4.94 GiB	5,294,194,688	1.18	15.27	~27.37 GiB	90
16	index_ci_builds_on_stage_id (ci_builds*)	10.44 GiB	291.64 GiB	4.37 GiB	4,685,144,064	1.72	41.80	~6.08 GiB	90
17	index_projects_on_repository_storage (projects*)	4.62 GiB	2.79 GiB	4.31 GiB	4,624,613,376	15.00	93.33	~315.00 MiB	90
18	index_ci_builds_on_commit_id_and_type_and_ref (ci_builds*)	34.74 GiB	291.64 GiB	4.28 GiB	4,592,795,648	1.14	12.31	~30.46 GiB	90
19	index_ci_build_trace_sections_on_project_id (ci_build_trace_sections)	31.55 GiB	94.19 GiB	4.18 GiB	4,479,926,272	1.15	13.23	~27.37 GiB	90
20	index_ci_builds_on_user_id (ci_builds*)	9.96 GiB	291.64 GiB	3.90 GiB	4,183,965,696	1.64	39.14	~6.06 GiB	90
21	index_projects_on_last_activity_at (projects*)	3.91 GiB	2.79 GiB	3.69 GiB	3,955,097,600	17.73	94.36	~225.45 MiB	90

* This table has specific autovacuum settings. See 'F001 Autovacuum: Current settings'

Conclusions

• [P1] Total index (btree only) bloat estimation is ~291.33 GiB, it is 25.17% of the overall size of all indexes and 7.14% of the DB size. Removing the index bloat will reduce the total DB size down to ~3.70 TiB. Free disk space will be increased by ~291.33 GiB. Total size of indexes is 1.34 times bigger than it could be. Notice that this is only an estimation, sometimes it may be significantly off.

• [P1] The following 37 indexes have significant size (>1 MiB) and bloat estimate > 90.00%:

  • index_projects_on_repository_storage: size 4.62 GiB, can be reduced 15.00 times, by ~4.34 GiB (~93.33%)
  • index_projects_on_last_activity_at: size 3.91 GiB, can be reduced 17.73 times, by ~3.71 GiB (~94.36%)
  • index_projects_on_runners_token: size 3.80 GiB, can be reduced 27.83 times, by ~3.68 GiB (~96.41%)
  • index_projects_on_pending_delete: size 3.81 GiB, can be reduced 17.27 times, by ~3.61 GiB (~94.21%)
  • index_projects_on_last_repository_check_failed: size 3.68 GiB, can be reduced 27.85 times, by ~3.56 GiB (~96.41%)
  • etc.

• [P2] There are 196 indexes with size > 1 MiB and index bloat estimate >= 40.00% and < 90.00%:

  • index_ci_builds_on_auto_canceled_by_id: size 15.39 GiB, can be reduced 4.20 times, by ~12.10 GiB (~76.19%)
  • index_ci_builds_on_protected: size 15.22 GiB, can be reduced 2.52 times, by ~9.79 GiB (~60.38%)
  • index_ci_builds_on_token_encrypted: size 16.15 GiB, can be reduced 2.22 times, by ~9.61 GiB (~54.89%)
  • index_ci_builds_on_token: size 17.01 GiB, can be reduced 2.03 times, by ~9.47 GiB (~50.83%)
  • index_ci_builds_on_runner_id: size 14.32 GiB, can be reduced 2.40 times, by ~8.94 GiB (~58.26%)
  • etc.

Recommendations

• [P1] Reduce and prevent a high level of index bloat:

  • to prevent a high level of bloat in the future, tune autovacuum: consider more aggressive autovacuum settings (see F001);
  • eliminate or reduce the current index bloat using one of the approaches listed below.

• If you want to get exact bloat numbers, clone the database, get index sizes, then apply database-wide VACUUM FULL (it eliminates all the bloat), and gets new table sizes. Then compare old and new numbers.

• To reduce the index bloat, consider one of the following approaches:

  • VACUUM FULL ( requires downtime / maintenance window),
  • REINDEX (REINDEX INDEX, REINDEX TABLE; requires downtime / maintenance window),
  • recreating indexes online using CREATE INDEX CONCURRENTLY, DROP INDEX CONCURRENTLY and renaming (not trivial for indexes supporting PK, FK) // REINDEX CONCURRENTLY is available in Postgres 12+,
  • one of the tools reducing the bloat online, without interrupting the operations:
    • pg_repack,
    • pg_squeeze,
    • pgcompacttable.

• Read more on this topic:

  • Index maintenance (PostgreSQL wiki)
  • Btree bloat query (2014, ioguix)
  • PostgreSQL Index bloat under a microscope (2017, Peter Geoghegan)
  • PostgreSQL Bloat: origins, monitoring and managing (2016, Compose)
  • Dealing with significant Postgres database bloat — what are your options? (2018, Compass)
  • Postgres database bloat analysis (2019, GitLab)

---

  Table of contents

F008 Autovacuum: Resource Usage

Observations

Data collected: 2019-09-30 13:37:48 +0000 UTC

Settings

Setting name	Value	Unit	Pretty value
autovacuum_max_workers	6
autovacuum_vacuum_cost_delay	5	ms
autovacuum_vacuum_cost_limit	6000
autovacuum_work_mem	-1	kB
log_autovacuum_min_duration	0	ms
maintenance_work_mem	2097152	kB	2.00 GiB
max_connections	300
shared_buffers	14450688	8kB	110.25 GiB
vacuum_cost_limit	200
work_mem	102400	kB	100.00 MiB

CPU

Cpu count you can see in report A001

RAM

Ram amount you can see in report A001

Max workers memory: 12 GiB

DISK

Warning: collection of current impact on disks is not yet implemented. Please refer to Postgres logs and see current read and write IO bandwidth caused by autovacuum.

Conclusions

Recommendations

---

  Table of contents

G001 Memory-related Settings

Observations

Data collected: 2019-09-30 13:56:33 +0000 UTC

Master (10.220.16.101)

Setting name	Value	Unit	Pretty value
autovacuum_max_workers	6
autovacuum_work_mem	-1	kB
effective_cache_size	43352064	8kB	330.75 GiB
maintenance_work_mem	2097152	kB	2.00 GiB
max_connections	300
shared_buffers	14450688	8kB	110.25 GiB
temp_buffers	1024	8kB	8.00 MiB
work_mem	102400	kB	100.00 MiB

Replica servers:

Replica (10.220.16.102)

Setting name	Value	Unit	Pretty value
autovacuum_max_workers	6
autovacuum_work_mem	-1	kB
effective_cache_size	43352064	8kB	330.75 GiB
maintenance_work_mem	2097152	kB	2.00 GiB
max_connections	300
shared_buffers	14450688	8kB	110.25 GiB
temp_buffers	1024	8kB	8.00 MiB
work_mem	102400	kB	100.00 MiB

Replica (10.220.16.103)

Setting name	Value	Unit	Pretty value
autovacuum_max_workers	6
autovacuum_work_mem	-1	kB
effective_cache_size	43352064	8kB	330.75 GiB
maintenance_work_mem	2097152	kB	2.00 GiB
max_connections	300
shared_buffers	14450688	8kB	110.25 GiB
temp_buffers	1024	8kB	8.00 MiB
work_mem	102400	kB	100.00 MiB

Replica (10.220.16.104)

Setting name	Value	Unit	Pretty value
autovacuum_max_workers	6
autovacuum_work_mem	-1	kB
effective_cache_size	43352064	8kB	330.75 GiB
maintenance_work_mem	2097152	kB	2.00 GiB
max_connections	300
shared_buffers	14450688	8kB	110.25 GiB
temp_buffers	1024	8kB	8.00 MiB
work_mem	102400	kB	100.00 MiB

Replica (10.220.16.105)

Setting name	Value	Unit	Pretty value
autovacuum_max_workers	6
autovacuum_work_mem	-1	kB
effective_cache_size	43352064	8kB	330.75 GiB
maintenance_work_mem	2097152	kB	2.00 GiB
max_connections	300
shared_buffers	14450688	8kB	110.25 GiB
temp_buffers	1024	8kB	8.00 MiB
work_mem	102400	kB	100.00 MiB

Replica (10.220.16.106)

Setting name	Value	Unit	Pretty value
autovacuum_max_workers	6
autovacuum_work_mem	-1	kB
effective_cache_size	43352064	8kB	330.75 GiB
maintenance_work_mem	2097152	kB	2.00 GiB
max_connections	300
shared_buffers	14450688	8kB	110.25 GiB
temp_buffers	1024	8kB	8.00 MiB
work_mem	102400	kB	100.00 MiB

Replica (10.220.16.107)

Setting name	Value	Unit	Pretty value
autovacuum_max_workers	6
autovacuum_work_mem	-1	kB
effective_cache_size	43352064	8kB	330.75 GiB
maintenance_work_mem	2097152	kB	2.00 GiB
max_connections	300
shared_buffers	14450688	8kB	110.25 GiB
temp_buffers	1024	8kB	8.00 MiB
work_mem	102400	kB	100.00 MiB

Replica (10.220.16.108)

Setting name	Value	Unit	Pretty value
autovacuum_max_workers	6
autovacuum_work_mem	-1	kB
effective_cache_size	43352064	8kB	330.75 GiB
maintenance_work_mem	2097152	kB	2.00 GiB
max_connections	300
shared_buffers	14450688	8kB	110.25 GiB
temp_buffers	1024	8kB	8.00 MiB
work_mem	102400	kB	100.00 MiB

Replica (10.220.16.109)

Setting name	Value	Unit	Pretty value
autovacuum_max_workers	6
autovacuum_work_mem	-1	kB
effective_cache_size	43352064	8kB	330.75 GiB
maintenance_work_mem	2097152	kB	2.00 GiB
max_connections	300
shared_buffers	14450688	8kB	110.25 GiB
temp_buffers	1024	8kB	8.00 MiB
work_mem	102400	kB	100.00 MiB

Conclusions

Recommendations

---

  Table of contents

G002 Connections and Current Activity

Observations

Data collected: 2019-09-30 13:56:35 +0000 UTC

Master (10.220.16.101)

The list is limited to 20 items. Total: 19.

#	User	DB	Current state	Count	State changed >1m ago	State changed >1h ago	Tx age >1m	Tx age >1h
1	ALL users	ALL databases	ALL states	261	2	1	1	0
2	gitlab	ALL databases	active	22	0	0	0	0
3	gitlab-psql	ALL databases	active	3	1	0	1	0
4	gitlab-checkup	ALL databases	active	1	0	0	0	0
5	gitlab	ALL databases	idle	190	1	1	0	0
6	pgbouncer	ALL databases	idle	4	0	0	0	0
7	gitlab-monitor	ALL databases	idle	1	0	0	0	0
8	gitlab-superuser	ALL databases	idle	1	0	0	0	0
9	postgres_exporter	ALL databases	idle	1	0	0	0	0
10	gitlab	ALL databases	idle in transaction	38	0	0	0	0
11	gitlab	gitlabhq_production	active	22	0	0	0	0
12	gitlab-psql	gitlabhq_production	active	3	1	0	1	0
13	gitlab-checkup	gitlabhq_production	active	1	0	0	0	0
14	gitlab	gitlabhq_production	idle	190	1	1	0	0
15	pgbouncer	gitlabhq_production	idle	4	0	0	0	0
16	postgres_exporter	gitlabhq_production	idle	1	0	0	0	0
17	gitlab-monitor	gitlabhq_production	idle	1	0	0	0	0
18	gitlab	gitlabhq_production	idle in transaction	38	0	0	0	0
19	gitlab-superuser	postgres	idle	1	0	0	0	0

Replica servers:

Replica (10.220.16.102)

#	User	DB	Current state	Count	State changed >1m ago	State changed >1h ago	Tx age >1m	Tx age >1h
1	ALL users	ALL databases	ALL states	46	0	0	0	0
2	gitlab	ALL databases	active	5	0	0	0	0
3	gitlab-checkup	ALL databases	active	1	0	0	0	0
4	gitlab	ALL databases	idle	33	0	0	0	0
5	pgbouncer	ALL databases	idle	4	0	0	0	0
6	gitlab-monitor	ALL databases	idle	1	0	0	0	0
7	gitlab-superuser	ALL databases	idle	1	0	0	0	0
8	postgres_exporter	ALL databases	idle	1	0	0	0	0
9	gitlab	gitlabhq_production	active	5	0	0	0	0
10	gitlab-checkup	gitlabhq_production	active	1	0	0	0	0
11	gitlab	gitlabhq_production	idle	33	0	0	0	0
12	pgbouncer	gitlabhq_production	idle	4	0	0	0	0
13	postgres_exporter	gitlabhq_production	idle	1	0	0	0	0
14	gitlab-monitor	gitlabhq_production	idle	1	0	0	0	0
15	gitlab-superuser	postgres	idle	1	0	0	0	0

Replica (10.220.16.103)

#	User	DB	Current state	Count	State changed >1m ago	State changed >1h ago	Tx age >1m	Tx age >1h
1	ALL users	ALL databases	ALL states	44	0	0	0	0
2	gitlab	ALL databases	active	6	0	0	0	0
3	gitlab-checkup	ALL databases	active	1	0	0	0	0
4	gitlab	ALL databases	idle	30	0	0	0	0
5	pgbouncer	ALL databases	idle	4	0	0	0	0
6	gitlab-monitor	ALL databases	idle	1	0	0	0	0
7	gitlab-superuser	ALL databases	idle	1	0	0	0	0
8	postgres_exporter	ALL databases	idle	1	0	0	0	0
9	gitlab	gitlabhq_production	active	6	0	0	0	0
10	gitlab-checkup	gitlabhq_production	active	1	0	0	0	0
11	gitlab	gitlabhq_production	idle	30	0	0	0	0
12	pgbouncer	gitlabhq_production	idle	4	0	0	0	0
13	postgres_exporter	gitlabhq_production	idle	1	0	0	0	0
14	gitlab-monitor	gitlabhq_production	idle	1	0	0	0	0
15	gitlab-superuser	postgres	idle	1	0	0	0	0

Replica (10.220.16.104)

#	User	DB	Current state	Count	State changed >1m ago	State changed >1h ago	Tx age >1m	Tx age >1h
1	ALL users	ALL databases	ALL states	38	0	0	0	0
2	gitlab	ALL databases	active	4	0	0	0	0
3	gitlab-checkup	ALL databases	active	1	0	0	0	0
4	gitlab	ALL databases	idle	26	0	0	0	0
5	pgbouncer	ALL databases	idle	4	0	0	0	0
6	gitlab-monitor	ALL databases	idle	1	0	0	0	0
7	gitlab-superuser	ALL databases	idle	1	0	0	0	0
8	postgres_exporter	ALL databases	idle	1	0	0	0	0
9	gitlab	gitlabhq_production	active	4	0	0	0	0
10	gitlab-checkup	gitlabhq_production	active	1	0	0	0	0
11	gitlab	gitlabhq_production	idle	26	0	0	0	0
12	pgbouncer	gitlabhq_production	idle	4	0	0	0	0
13	postgres_exporter	gitlabhq_production	idle	1	0	0	0	0
14	gitlab-monitor	gitlabhq_production	idle	1	0	0	0	0
15	gitlab-superuser	postgres	idle	1	0	0	0	0

Replica (10.220.16.105)

#	User	DB	Current state	Count	State changed >1m ago	State changed >1h ago	Tx age >1m	Tx age >1h
1	ALL users	ALL databases	ALL states	39	0	0	0	0
2	gitlab	ALL databases	active	3	0	0	0	0
3	gitlab-monitor	ALL databases	active	1	0	0	0	0
4	gitlab-checkup	ALL databases	active	1	0	0	0	0
5	gitlab	ALL databases	idle	28	0	0	0	0
6	pgbouncer	ALL databases	idle	4	0	0	0	0
7	gitlab-superuser	ALL databases	idle	1	0	0	0	0
8	postgres_exporter	ALL databases	idle	1	0	0	0	0
9	gitlab	gitlabhq_production	active	3	0	0	0	0
10	gitlab-checkup	gitlabhq_production	active	1	0	0	0	0
11	gitlab-monitor	gitlabhq_production	active	1	0	0	0	0
12	gitlab	gitlabhq_production	idle	28	0	0	0	0
13	pgbouncer	gitlabhq_production	idle	4	0	0	0	0
14	postgres_exporter	gitlabhq_production	idle	1	0	0	0	0
15	gitlab-superuser	postgres	idle	1	0	0	0	0

Replica (10.220.16.106)

#	User	DB	Current state	Count	State changed >1m ago	State changed >1h ago	Tx age >1m	Tx age >1h
1	ALL users	ALL databases	ALL states	48	0	0	0	0
2	gitlab	ALL databases	active	6	0	0	0	0
3	gitlab-checkup	ALL databases	active	1	0	0	0	0
4	gitlab	ALL databases	idle	33	0	0	0	0
5	pgbouncer	ALL databases	idle	4	0	0	0	0
6	gitlab-monitor	ALL databases	idle	2	0	0	0	0
7	gitlab-superuser	ALL databases	idle	1	0	0	0	0
8	postgres_exporter	ALL databases	idle	1	0	0	0	0
9	gitlab	gitlabhq_production	active	6	0	0	0	0
10	gitlab-checkup	gitlabhq_production	active	1	0	0	0	0
11	gitlab	gitlabhq_production	idle	33	0	0	0	0
12	pgbouncer	gitlabhq_production	idle	4	0	0	0	0
13	gitlab-monitor	gitlabhq_production	idle	2	0	0	0	0
14	postgres_exporter	gitlabhq_production	idle	1	0	0	0	0
15	gitlab-superuser	postgres	idle	1	0	0	0	0

Replica (10.220.16.107)

#	User	DB	Current state	Count	State changed >1m ago	State changed >1h ago	Tx age >1m	Tx age >1h
1	ALL users	ALL databases	ALL states	40	0	0	0	0
2	gitlab	ALL databases	active	2	0	0	0	0
3	gitlab-checkup	ALL databases	active	1	0	0	0	0
4	gitlab	ALL databases	idle	30	0	0	0	0
5	pgbouncer	ALL databases	idle	4	0	0	0	0
6	gitlab-monitor	ALL databases	idle	1	0	0	0	0
7	gitlab-superuser	ALL databases	idle	1	0	0	0	0
8	postgres_exporter	ALL databases	idle	1	0	0	0	0
9	gitlab	gitlabhq_production	active	2	0	0	0	0
10	gitlab-checkup	gitlabhq_production	active	1	0	0	0	0
11	gitlab	gitlabhq_production	idle	30	0	0	0	0
12	pgbouncer	gitlabhq_production	idle	4	0	0	0	0
13	postgres_exporter	gitlabhq_production	idle	1	0	0	0	0
14	gitlab-monitor	gitlabhq_production	idle	1	0	0	0	0
15	gitlab-superuser	postgres	idle	1	0	0	0	0

Replica (10.220.16.108)

#	User	DB	Current state	Count	State changed >1m ago	State changed >1h ago	Tx age >1m	Tx age >1h
1	ALL users	ALL databases	ALL states	40	0	0	0	0
2	gitlab	ALL databases	active	2	0	0	0	0
3	gitlab-checkup	ALL databases	active	1	0	0	0	0
4	gitlab	ALL databases	idle	31	0	0	0	0
5	pgbouncer	ALL databases	idle	3	0	0	0	0
6	gitlab-monitor	ALL databases	idle	1	0	0	0	0
7	gitlab-superuser	ALL databases	idle	1	0	0	0	0
8	postgres_exporter	ALL databases	idle	1	0	0	0	0
9	gitlab	gitlabhq_production	active	2	0	0	0	0
10	gitlab-checkup	gitlabhq_production	active	1	0	0	0	0
11	gitlab	gitlabhq_production	idle	31	0	0	0	0
12	pgbouncer	gitlabhq_production	idle	3	0	0	0	0
13	postgres_exporter	gitlabhq_production	idle	1	0	0	0	0
14	gitlab-monitor	gitlabhq_production	idle	1	0	0	0	0
15	gitlab-superuser	postgres	idle	1	0	0	0	0

Replica (10.220.16.109)

#	User	DB	Current state	Count	State changed >1m ago	State changed >1h ago	Tx age >1m	Tx age >1h
1	ALL users	ALL databases	ALL states	42	0	0	0	0
2	gitlab	ALL databases	active	6	0	0	0	0
3	gitlab-checkup	ALL databases	active	1	0	0	0	0
4	gitlab	ALL databases	idle	28	0	0	0	0
5	pgbouncer	ALL databases	idle	4	0	0	0	0
6	gitlab-monitor	ALL databases	idle	1	0	0	0	0
7	gitlab-superuser	ALL databases	idle	1	0	0	0	0
8	postgres_exporter	ALL databases	idle	1	0	0	0	0
9	gitlab	gitlabhq_production	active	6	0	0	0	0
10	gitlab-checkup	gitlabhq_production	active	1	0	0	0	0
11	gitlab	gitlabhq_production	idle	28	0	0	0	0
12	pgbouncer	gitlabhq_production	idle	4	0	0	0	0
13	postgres_exporter	gitlabhq_production	idle	1	0	0	0	0
14	gitlab-monitor	gitlabhq_production	idle	1	0	0	0	0
15	gitlab-superuser	postgres	idle	1	0	0	0	0

Conclusions

Recommendations

All good, no recommendations here.

---

  Table of contents

G003 Timeouts, Locks, Deadlocks

Observations

Data collected: 2019-09-30 13:56:38 +0000 UTC

Master (10.220.16.101)

Timeouts

Setting name	Value	Unit	Pretty value
authentication_timeout	60	s
idle_in_transaction_session_timeout	30000	ms
statement_timeout	15000	ms

Locks

Setting name	Value	Unit	Pretty value
deadlock_timeout	5000	ms
lock_timeout	0	ms
max_locks_per_transaction	128
max_pred_locks_per_transaction	64

Databases data

#	Database	Conflicts	▼ Deadlocks	Stats reset at	Stat reset
1	gitlabhq_production	0	20	2019-09-09T18:31:35.8238+00:00	20 days 19:06:17
2	postgres	0	0	2019-09-09T18:31:33.20897+00:00	20 days 19:06:19

Replica servers:

Replica (10.220.16.102)

Timeouts

Setting name	Value	Unit	Pretty value
authentication_timeout	60	s
idle_in_transaction_session_timeout	30000	ms
statement_timeout	15000	ms

Locks

Setting name	Value	Unit	Pretty value
deadlock_timeout	5000	ms
lock_timeout	0	ms
max_locks_per_transaction	128
max_pred_locks_per_transaction	64

Databases data

Database	Conflicts	▼ Deadlocks	Stats reset at	Stat reset
postgres	0	0	2019-09-09T20:34:43.543476+00:00	20 days 17:04:22
gitlabhq_production	0	0	2019-09-09T20:34:45.4209+00:00	20 days 17:04:20

Replica (10.220.16.103)

Timeouts

Setting name	Value	Unit	Pretty value
authentication_timeout	60	s
idle_in_transaction_session_timeout	30000	ms
statement_timeout	15000	ms

Locks

Setting name	Value	Unit	Pretty value
deadlock_timeout	5000	ms
lock_timeout	0	ms
max_locks_per_transaction	128
max_pred_locks_per_transaction	64

Databases data

Database	Conflicts	▼ Deadlocks	Stats reset at	Stat reset
postgres	0	0	2019-09-09T20:34:36.972379+00:00	20 days 17:05:45
gitlabhq_production	0	0	2019-09-09T20:34:37.497353+00:00	20 days 17:05:44

Replica (10.220.16.104)

Timeouts

Setting name	Value	Unit	Pretty value
authentication_timeout	60	s
idle_in_transaction_session_timeout	30000	ms
statement_timeout	15000	ms

Locks

Setting name	Value	Unit	Pretty value
deadlock_timeout	5000	ms
lock_timeout	0	ms
max_locks_per_transaction	128
max_pred_locks_per_transaction	64

Databases data

Database	Conflicts	▼ Deadlocks	Stats reset at	Stat reset
postgres	0	0	2019-09-09T20:34:35.218275+00:00	20 days 17:07:16
gitlabhq_production	1	0	2019-09-09T20:34:35.422453+00:00	20 days 17:07:15

Replica (10.220.16.105)

Timeouts

Setting name	Value	Unit	Pretty value
authentication_timeout	60	s
idle_in_transaction_session_timeout	30000	ms
statement_timeout	15000	ms

Locks

Setting name	Value	Unit	Pretty value
deadlock_timeout	5000	ms
lock_timeout	0	ms
max_locks_per_transaction	128
max_pred_locks_per_transaction	64

Databases data

Database	Conflicts	▼ Deadlocks	Stats reset at	Stat reset
postgres	0	0	2019-09-09T20:34:42.049288+00:00	20 days 17:09:06
gitlabhq_production	0	0	2019-09-09T20:34:43.018416+00:00	20 days 17:09:05

Replica (10.220.16.106)

Timeouts

Setting name	Value	Unit	Pretty value
authentication_timeout	60	s
idle_in_transaction_session_timeout	30000	ms
statement_timeout	15000	ms

Locks

Setting name	Value	Unit	Pretty value
deadlock_timeout	5000	ms
lock_timeout	0	ms
max_locks_per_transaction	128
max_pred_locks_per_transaction	64

Databases data

Database	Conflicts	▼ Deadlocks	Stats reset at	Stat reset
postgres	0	0	2019-09-09T20:34:05.869755+00:00	20 days 17:11:57
gitlabhq_production	3	0	2019-09-09T20:34:08.542607+00:00	20 days 17:11:54

Replica (10.220.16.107)

Timeouts

Setting name	Value	Unit	Pretty value
authentication_timeout	60	s
idle_in_transaction_session_timeout	30000	ms
statement_timeout	15000	ms

Locks

Setting name	Value	Unit	Pretty value
deadlock_timeout	5000	ms
lock_timeout	0	ms
max_locks_per_transaction	128
max_pred_locks_per_transaction	64

Databases data

Database	Conflicts	▼ Deadlocks	Stats reset at	Stat reset
postgres	0	0	2019-09-09T20:34:43.591948+00:00	20 days 17:14:16
gitlabhq_production	0	0	2019-09-09T20:34:46.149884+00:00	20 days 17:14:13

Replica (10.220.16.108)

Timeouts

Setting name	Value	Unit	Pretty value
authentication_timeout	60	s
idle_in_transaction_session_timeout	30000	ms
statement_timeout	15000	ms

Locks

Setting name	Value	Unit	Pretty value
deadlock_timeout	5000	ms
lock_timeout	0	ms
max_locks_per_transaction	128
max_pred_locks_per_transaction	64

Databases data

Database	Conflicts	▼ Deadlocks	Stats reset at	Stat reset
postgres	0	0	2019-09-09T23:15:38.234878+00:00	20 days 14:36:54
gitlabhq_production	0	0	2019-09-09T23:15:42.35404+00:00	20 days 14:36:50

Replica (10.220.16.109)

Timeouts

Setting name	Value	Unit	Pretty value
authentication_timeout	60	s
idle_in_transaction_session_timeout	30000	ms
statement_timeout	15000	ms

Locks

Setting name	Value	Unit	Pretty value
deadlock_timeout	5000	ms
lock_timeout	0	ms
max_locks_per_transaction	128
max_pred_locks_per_transaction	64

Databases data

Database	Conflicts	▼ Deadlocks	Stats reset at	Stat reset
postgres	0	0	2019-09-09T20:34:24.604946+00:00	20 days 17:22:15
gitlabhq_production	0	0	2019-09-09T20:34:25.124515+00:00	20 days 17:22:14

Conclusions

Recommendations

---

  Table of contents

H001 Invalid Indexes

Observations

Data collected: 2019-09-30 13:37:53 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.101)

Invalid indexes not found

Conclusions

Recommendations

---

  Table of contents

H002 Unused and Redundant Indexes

Observations

Data collected: 2019-09-30 13:56:40 +0000 UTC
Current database: gitlabhq_production
Stats reset: 20 days 19:06:00 ago (2019-09-09 18:31:36 +0000 UTC)
Statistics age is less than 30 days. Make decisions on index cleanup with caution!

Never Used Indexes

The list is limited to 20 items. Total: 64.

#	Table	Index	10.220.16.101 usage	10.220.16.102 usage	10.220.16.103 usage	10.220.16.104 usage	10.220.16.105 usage	10.220.16.106 usage	10.220.16.107 usage	10.220.16.108 usage	10.220.16.109 usage	▼ Index size	Table size	Supports FK
	=====TOTAL=====											142.08 GiB	1.32 TiB
1	ci_builds	index_ci_builds_on_protected	0	0	0	0	0	0	0	0	0	15.24 GiB	291.56 GiB
2	ci_builds	index_ci_builds_on_queued_at	0	0	0	0	0	0	0	0	0	12.73 GiB	291.56 GiB
3	ci_job_artifacts	index_ci_job_artifacts_on_file_store	0	0	0	0	0	0	0	0	0	6.65 GiB	36.69 GiB
4	audit_events	analytics_index_audit_events_on_created_at_and_author_id	0	0	0	0	0	0	0	0	0	5.48 GiB	42.52 GiB
6	projects	index_projects_on_pending_delete	0	0	0	0	0	0	0	0	0	3.81 GiB	2.79 GiB	Yes
8	projects	index_projects_on_last_repository_check_failed	0	0	0	0	0	0	0	0	0	3.68 GiB	2.79 GiB	Yes
17	projects	index_projects_on_mirror_last_successful_update_at	0	0	0	0	0	0	0	0	0	3.67 GiB	2.79 GiB	Yes
18	ci_stages	index_ci_stages_on_pipeline_id_and_position	0									3.50 GiB	11.69 GiB
19	merge_requests	index_merge_requests_on_title	0									2.71 GiB	13.42 GiB	Yes
25	ci_pipelines	index_ci_pipelines_on_status	0	0		0	0	0	0	0	0	2.37 GiB	16.48 GiB	Yes
30	users	index_users_on_accepted_term_id	0	0	0	0	0	0	0	0	0	1.65 GiB	1.95 GiB	Yes
31	merge_request_diffs	index_merge_request_diffs_on_merge_request_id_and_id_partial	0	0	0	0	0	0	0	0	0	1.30 GiB	10.47 GiB
32	deployments	index_deployments_on_created_at	0	0	0	0	0	0	0	0	0	1.26 GiB	4.92 GiB
33	projects	idx_project_repository_check_partial	0	0	0	0	0	0	0	0	0	1.17 GiB	2.79 GiB
34	merge_request_metrics	index_merge_request_metrics_on_first_deployed_to_production_at	0	0		0	0	0	0	0	0	1.15 GiB	2.07 GiB
35	ci_runners	index_ci_runners_on_is_shared	0	0	0	0	0	0	0	0	0	712.40 MiB	107.80 MiB
36	ci_runners	index_ci_runners_on_locked	0	0	0	0	0	0	0	0	0	692.37 MiB	107.80 MiB
37	ci_runners	index_ci_runners_on_contacted_at	0	0	0	0	0	0		0	0	600.68 MiB	107.80 MiB
48	projects	index_on_projects_lower_path	0	0	0	0	0	0	0	0	0	588.54 MiB	2.79 GiB	Yes
61	users	index_users_on_state_and_internal	0	0	0	0	0	0	0	0	0	391.38 MiB	1.95 GiB	Yes

Rarely Used Indexes

The list is limited to 20 items. Total: 224.

#	Table	Index	10.220.16.101 usage	10.220.16.102 usage	10.220.16.103 usage	10.220.16.104 usage	10.220.16.105 usage	10.220.16.106 usage	10.220.16.107 usage	10.220.16.108 usage	10.220.16.109 usage	▼ Index size	Table size	Comment	Supports FK
	=====TOTAL=====											1.02 TiB	8.23 TiB
1	ci_builds	index_ci_builds_on_commit_id_and_type_and_name_and_ref	scans: 15,184,478/hour, writes: 61,590,109/hour	scans: 540,055/hour, writes: 0/hour	scans: 540,132/hour, writes: 0/hour	scans: 538,338/hour, writes: 0/hour	scans: 537,333/hour, writes: 0/hour	scans: 539,133/hour, writes: 0/hour	scans: 537,786/hour, writes: 0/hour	scans: 526,334/hour, writes: 0/hour	scans: 539,417/hour, writes: 0/hour	46.57 GiB	291.56 GiB	Low Scans, High Writes
2	ci_build_trace_sections	index_ci_build_trace_sections_on_section_name_id	scans: 62,280/hour, writes: 94,958,723/hour									32.44 GiB	94.16 GiB	Low Scans, High Writes
3	ci_build_trace_sections	index_ci_build_trace_sections_on_project_id	scans: 92,791/hour, writes: 94,958,723/hour									31.68 GiB	94.16 GiB	Low Scans, High Writes
4	events	index_events_on_project_id_and_created_at	scans: 1,303/hour, writes: 21,500,162/hour	scans: 198,781/hour, writes: 0/hour	scans: 235,881/hour, writes: 0/hour	scans: 195,901/hour, writes: 0/hour	scans: 247,361/hour, writes: 0/hour	scans: 255,539/hour, writes: 0/hour	scans: 219,018/hour, writes: 0/hour	scans: 209,909/hour, writes: 0/hour	scans: 205,767/hour, writes: 0/hour	16.29 GiB	28.75 GiB	Low Scans, High Writes
5	notes	index_notes_on_discussion_id	scans: 7,141,296/hour, writes: 25,105,136/hour	scans: 81,208/hour, writes: 0/hour	scans: 80,707/hour, writes: 0/hour	scans: 80,786/hour, writes: 0/hour	scans: 80,784/hour, writes: 0/hour	scans: 81,633/hour, writes: 0/hour	scans: 80,685/hour, writes: 0/hour	scans: 78,162/hour, writes: 0/hour	scans: 80,935/hour, writes: 0/hour	15.56 GiB	122.34 GiB	Low Scans, High Writes
7	ci_builds	index_ci_builds_on_auto_canceled_by_id	scans: 462,028/hour, writes: 61,590,109/hour									15.42 GiB	291.56 GiB	Low Scans, High Writes	Yes
8	ci_builds	tmp_build_stage_position_index	scans: 442/hour, writes: 61,590,109/hour									15.20 GiB	291.56 GiB	Low Scans, High Writes
9	ci_builds	index_ci_builds_on_status_and_type_and_runner_id	scans: 37,318,642/hour, writes: 61,590,109/hour	scans: 4,777,664/hour, writes: 0/hour	scans: 4,771,316/hour, writes: 0/hour	scans: 4,774,908/hour, writes: 0/hour	scans: 4,783,094/hour, writes: 0/hour	scans: 4,776,569/hour, writes: 0/hour	scans: 4,787,719/hour, writes: 0/hour	scans: 4,642,701/hour, writes: 0/hour	scans: 4,781,555/hour, writes: 0/hour	14.66 GiB	291.56 GiB	Low Scans, High Writes
11	ci_builds	index_ci_builds_on_runner_id	scans: 8,230,782/hour, writes: 61,590,109/hour	scans: 1,338,618/hour, writes: 0/hour	scans: 1,336,588/hour, writes: 0/hour	scans: 1,337,588/hour, writes: 0/hour	scans: 1,338,962/hour, writes: 0/hour	scans: 1,338,756/hour, writes: 0/hour	scans: 1,340,959/hour, writes: 0/hour	scans: 1,295,198/hour, writes: 0/hour	scans: 1,337,986/hour, writes: 0/hour	14.35 GiB	291.56 GiB	Low Scans, High Writes	Yes
12	events	analytics_index_events_on_created_at_and_author_id	scans: 23,784/hour, writes: 21,500,162/hour	scans: 378,398/hour, writes: 0/hour	scans: 378,258/hour, writes: 0/hour	scans: 378,261/hour, writes: 0/hour	scans: 377,617/hour, writes: 0/hour	scans: 375,349/hour, writes: 0/hour	scans: 376,656/hour, writes: 0/hour	scans: 364,601/hour, writes: 0/hour	scans: 376,180/hour, writes: 0/hour	13.07 GiB	28.75 GiB	Low Scans, High Writes
13	ci_builds	index_ci_builds_on_updated_at	scans: 2,470/hour, writes: 61,590,109/hour									12.65 GiB	291.56 GiB	Low Scans, High Writes
14	ci_job_artifacts	index_ci_job_artifacts_on_expire_at_and_job_id	scans: 36,018/hour, writes: 38,258,788/hour				scans: 5/hour, writes: 0/hour					12.06 GiB	36.69 GiB	Low Scans, High Writes
15	ci_builds	ci_builds_gitlab_monitor_metrics	scans: 2,912/hour, writes: 61,590,109/hour	scans: 4,530/hour, writes: 0/hour	scans: 4,701/hour, writes: 0/hour	scans: 4,597/hour, writes: 0/hour	scans: 4,514/hour, writes: 0/hour	scans: 4,639/hour, writes: 0/hour	scans: 4,500/hour, writes: 0/hour	scans: 4,131/hour, writes: 0/hour	scans: 4,535/hour, writes: 0/hour	11.37 GiB	291.56 GiB	Low Scans, High Writes
16	events	index_events_on_author_id_and_project_id	scans: 4,174,240/hour, writes: 21,500,162/hour									11.36 GiB	28.75 GiB	Low Scans, High Writes
17	events	index_events_on_action	scans: 2/hour, writes: 21,500,162/hour	scans: 650/hour, writes: 0/hour	scans: 604/hour, writes: 0/hour	scans: 660/hour, writes: 0/hour	scans: 618/hour, writes: 0/hour	scans: 624/hour, writes: 0/hour	scans: 661/hour, writes: 0/hour	scans: 600/hour, writes: 0/hour	scans: 664/hour, writes: 0/hour	10.27 GiB	28.75 GiB	Low Scans, High Writes
18	events	index_events_on_project_id_and_id	scans: 3,363,724/hour, writes: 21,500,162/hour									10.14 GiB	28.75 GiB	Low Scans, High Writes
20	ci_builds	index_ci_builds_on_user_id	scans: 2,195/hour, writes: 61,590,109/hour									9.97 GiB	291.56 GiB	Low Scans, High Writes	Yes
21	ci_job_artifacts	index_ci_job_artifacts_on_project_id	scans: 92,794/hour, writes: 38,258,788/hour	scans: 3/hour, writes: 0/hour	scans: 24/hour, writes: 0/hour	scans: 5/hour, writes: 0/hour	scans: 9/hour, writes: 0/hour	scans: 12/hour, writes: 0/hour	scans: 5/hour, writes: 0/hour	scans: 4/hour, writes: 0/hour	scans: 6/hour, writes: 0/hour	8.03 GiB	36.69 GiB	Low Scans, High Writes
22	notes	index_notes_on_line_code	scans: 28,475/hour, writes: 25,105,136/hour									6.79 GiB	122.34 GiB	Low Scans, High Writes
23	ci_builds_metadata	index_ci_builds_metadata_on_project_id	scans: 92,790/hour, writes: 28,508,424/hour									5.80 GiB	11.05 GiB	Low Scans, High Writes

Redundant Indexes

#	Table	Index	Redundant to	10.220.16.101 usage	10.220.16.102 usage	10.220.16.103 usage	10.220.16.104 usage	10.220.16.105 usage	10.220.16.106 usage	10.220.16.107 usage	10.220.16.108 usage	10.220.16.109 usage	▼ Index size	Table size	Supports FK
	=====TOTAL=====												20.29 MiB	55.01 MiB
1	namespace_root_storage_statistics	index_namespace_root_storage_statistics_on_namespace_id	namespace_root_storage_statistics_pkey	0									14.68 MiB	47.50 MiB
2	namespace_aggregation_schedules	index_namespace_aggregation_schedules_on_namespace_id	namespace_aggregation_schedules_pkey	0									2.36 MiB	768.00 KiB
3	user_statuses	index_user_statuses_on_user_id	user_statuses_pkey	0									2.23 MiB	5.43 MiB
4	project_feature_usages	index_project_feature_usages_on_project_id	project_feature_usages_pkey	0									1.04 MiB	1.35 MiB

Conclusions

Recommendations

"DO" database migration code

DROP INDEX CONCURRENTLY index_analytics_ca_group_stages_on_group_id; -- 8192 bytes, index_analytics_ca_group_stages_on_group_id_and_name, table analytics_cycle_analytics_group_stages
DROP INDEX CONCURRENTLY index_analytics_ca_project_stages_on_project_id; -- 8192 bytes, index_analytics_ca_project_stages_on_project_id_and_name, table analytics_cycle_analytics_project_stages
DROP INDEX CONCURRENTLY index_approval_merge_request_rules_1; -- 124 MB, approval_rule_name_index_for_code_owners, table approval_merge_request_rules
DROP INDEX CONCURRENTLY index_approval_rules_code_owners_rule_type; -- 2864 kB, index_approval_rule_name_for_code_owners_rule_type, any_approver_merge_request_rule_type_unique_index, table approval_merge_request_rules
DROP INDEX CONCURRENTLY index_approval_project_rules_on_project_id; -- 2584 kB, any_approver_project_rule_type_unique_index, table approval_project_rules
DROP INDEX CONCURRENTLY analytics_index_audit_events_on_created_at_and_author_id; -- 5603 MB, Never Used Indexes, table audit_events
DROP INDEX CONCURRENTLY index_board_group_recent_visits_on_user_id; -- 616 kB, index_board_group_recent_visits_on_user_group_and_board, Never Used Indexes, table board_group_recent_visits
DROP INDEX CONCURRENTLY index_board_project_recent_visits_on_user_id; -- 2176 kB, index_board_project_recent_visits_on_user_project_and_board, table board_project_recent_visits
DROP INDEX CONCURRENTLY index_ci_builds_on_artifacts_expire_at; -- 82 MB, Never Used Indexes, table ci_builds
DROP INDEX CONCURRENTLY index_ci_builds_on_protected; -- 15 GB, Never Used Indexes, table ci_builds
DROP INDEX CONCURRENTLY index_ci_builds_on_queued_at; -- 13 GB, Never Used Indexes, table ci_builds
DROP INDEX CONCURRENTLY index_ci_builds_on_stage_id; -- 10 GB, tmp_build_stage_position_index, table ci_builds
DROP INDEX CONCURRENTLY index_ci_builds_metadata_on_build_id_and_interruptible; -- 80 kB, index_ci_builds_metadata_on_build_id, table ci_builds_metadata
DROP INDEX CONCURRENTLY index_ci_job_artifacts_on_file_store; -- 6802 MB, Never Used Indexes, table ci_job_artifacts
DROP INDEX CONCURRENTLY index_ci_job_artifacts_on_project_id; -- 8216 MB, index_ci_job_artifacts_on_project_id_for_security_reports, table ci_job_artifacts
DROP INDEX CONCURRENTLY index_ci_job_artifacts_on_project_id_for_security_reports; -- 16 MB, Never Used Indexes, index_ci_job_artifacts_on_project_id, table ci_job_artifacts
DROP INDEX CONCURRENTLY index_ci_pipeline_schedules_on_next_run_at_and_active; -- 22 MB, Never Used Indexes, table ci_pipeline_schedules
DROP INDEX CONCURRENTLY index_ci_pipelines_on_project_id; -- 2333 MB, index_ci_pipelines_on_project_idandrefandiddesc, index_ci_pipelines_on_project_id_and_sha, index_ci_pipelines_on_project_id_and_ref_and_status_and_id, index_ci_pipelines_on_project_id_and_source, index_ci_pipelines_on_project_id_and_iid, index_ci_pipelines_on_project_id_and_status_and_config_source, table ci_pipelines
DROP INDEX CONCURRENTLY index_ci_pipelines_on_status; -- 2425 MB, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, table ci_pipelines
DROP INDEX CONCURRENTLY index_ci_runners_on_contacted_at; -- 601 MB, Never Used Indexes, table ci_runners
DROP INDEX CONCURRENTLY index_ci_runners_on_is_shared; -- 712 MB, Never Used Indexes, table ci_runners
DROP INDEX CONCURRENTLY index_ci_runners_on_locked; -- 692 MB, Never Used Indexes, table ci_runners
DROP INDEX CONCURRENTLY index_ci_runners_on_token; -- 560 MB, index_ci_runners_on_token_encrypted, table ci_runners
DROP INDEX CONCURRENTLY index_ci_stages_on_pipeline_id; -- 3539 MB, index_ci_stages_on_pipeline_id_and_name, index_ci_stages_on_pipeline_id_and_position, table ci_stages
DROP INDEX CONCURRENTLY index_ci_stages_on_pipeline_id_and_position; -- 3578 MB, Never Used Indexes, table ci_stages
DROP INDEX CONCURRENTLY index_clusters_kubernetes_namespaces_on_cluster_id; -- 1232 kB, kubernetes_namespaces_cluster_and_namespace, index_kubernetes_namespaces_on_cluster_project_environment_id, table clusters_kubernetes_namespaces
DROP INDEX CONCURRENTLY index_container_repositories_on_project_id; -- 16 MB, index_container_repositories_on_project_id_and_name, table container_repositories
DROP INDEX CONCURRENTLY index_deploy_tokens_on_token_and_expires_at_and_id; -- 11 MB, Never Used Indexes, table deploy_tokens
DROP INDEX CONCURRENTLY index_deployments_on_created_at; -- 1290 MB, Never Used Indexes, table deployments
DROP INDEX CONCURRENTLY index_deployments_on_project_id_and_status; -- 878 MB, index_deployments_on_project_id_and_status_and_created_at, table deployments
DROP INDEX CONCURRENTLY partial_index_deployments_for_legacy_successful_deployments; -- 8192 bytes, index_deployments_on_cluster_id_and_status, deployments_pkey, Never Used Indexes, table deployments
DROP INDEX CONCURRENTLY index_design_management_designs_versions_on_design_id; -- 40 kB, design_management_designs_versions_uniqueness, table design_management_designs_versions
DROP INDEX CONCURRENTLY index_design_management_designs_versions_on_event; -- 48 kB, Never Used Indexes, table design_management_designs_versions
DROP INDEX CONCURRENTLY index_epics_on_end_date; -- 920 kB, Never Used Indexes, Never Used Indexes, table epics
DROP INDEX CONCURRENTLY index_epics_on_start_date; -- 912 kB, Never Used Indexes, Never Used Indexes, table epics
DROP INDEX CONCURRENTLY index_milestone; -- 1544 kB, index_epics_on_closed_by_id, index_epics_on_parent_id, table epics
DROP INDEX CONCURRENTLY index_geo_node_namespace_links_on_geo_node_id; -- 8192 bytes, index_geo_node_namespace_links_on_geo_node_id_and_namespace_id, table geo_node_namespace_links
DROP INDEX CONCURRENTLY index_issue_links_on_source_id; -- 9240 kB, index_issue_links_on_source_id_and_target_id, table issue_links
DROP INDEX CONCURRENTLY idx_jira_connect_subscriptions_on_installation_id; -- 16 kB, Never Used Indexes, idx_jira_connect_subscriptions_on_installation_id_namespace_id, table jira_connect_subscriptions
DROP INDEX CONCURRENTLY index_label_priorities_on_priority; -- 3256 kB, Never Used Indexes, table label_priorities
DROP INDEX CONCURRENTLY index_list_user_preferences_on_user_id; -- 744 kB, index_list_user_preferences_on_user_id_and_list_id, table list_user_preferences
DROP INDEX CONCURRENTLY index_members_on_access_level; -- 348 MB, Never Used Indexes, table members
DROP INDEX CONCURRENTLY index_merge_request_assignees_on_merge_request_id; -- 311 MB, index_merge_request_assignees_on_merge_request_id_and_user_id, table merge_request_assignees
DROP INDEX CONCURRENTLY index_merge_request_diffs_on_merge_request_id_and_id; -- 1247 MB, index_merge_request_diffs_on_merge_request_id_and_id_partial, table merge_request_diffs
DROP INDEX CONCURRENTLY index_merge_request_diffs_on_merge_request_id_and_id_partial; -- 1322 MB, index_merge_request_diffs_on_merge_request_id_and_id, Never Used Indexes, table merge_request_diffs
DROP INDEX CONCURRENTLY index_merge_request_metrics; -- 851 MB, index_merge_request_metrics_on_merge_request_id_and_merged_at, table merge_request_metrics
DROP INDEX CONCURRENTLY index_merge_request_metrics_on_first_deployed_to_production_at; -- 1170 MB, Never Used Indexes, table merge_request_metrics
DROP INDEX CONCURRENTLY index_merge_request_metrics_on_latest_closed_at; -- 35 MB, Never Used Indexes, table merge_request_metrics
DROP INDEX CONCURRENTLY index_merge_requests_on_source_project_and_branch_state_opened; -- 187 MB, index_merge_requests_on_source_project_id_and_source_branch, table merge_requests
DROP INDEX CONCURRENTLY index_merge_requests_on_source_project_id_and_source_branch; -- 2450 MB, index_merge_requests_on_source_project_and_branch_state_opened, table merge_requests
DROP INDEX CONCURRENTLY index_merge_requests_on_state_and_merge_status; -- 295 MB, Never Used Indexes, table merge_requests
DROP INDEX CONCURRENTLY index_merge_requests_on_target_project_id_and_iid_opened; -- 107 MB, index_merge_requests_on_target_project_id_and_iid, table merge_requests
DROP INDEX CONCURRENTLY index_merge_requests_on_title; -- 2765 MB, Never Used Indexes, table merge_requests
DROP INDEX CONCURRENTLY index_namespace_aggregation_schedules_on_namespace_id; -- 2408 kB, namespace_aggregation_schedules_pkey, table namespace_aggregation_schedules
DROP INDEX CONCURRENTLY index_namespace_root_storage_statistics_on_namespace_id; -- 15 MB, namespace_root_storage_statistics_pkey, table namespace_root_storage_statistics
DROP INDEX CONCURRENTLY index_namespaces_on_created_at; -- 236 MB, Never Used Indexes, Never Used Indexes, Never Used Indexes, table namespaces
DROP INDEX CONCURRENTLY index_namespaces_on_ldap_sync_last_successful_update_at; -- 291 MB, Never Used Indexes, Never Used Indexes, table namespaces
DROP INDEX CONCURRENTLY index_namespaces_on_ldap_sync_last_update_at; -- 291 MB, Never Used Indexes, Never Used Indexes, table namespaces
DROP INDEX CONCURRENTLY index_namespaces_on_path; -- 257 MB, index_namespaces_on_runners_token_encrypted, index_namespaces_on_runners_token, table namespaces
DROP INDEX CONCURRENTLY index_namespaces_on_plan_id; -- 291 MB, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, table namespaces
DROP INDEX CONCURRENTLY index_namespaces_on_require_two_factor_authentication; -- 291 MB, Never Used Indexes, table namespaces
DROP INDEX CONCURRENTLY index_namespaces_on_shared_and_extra_runners_minutes_limit; -- 291 MB, Never Used Indexes, table namespaces
DROP INDEX CONCURRENTLY index_namespaces_on_trial_ends_on; -- 760 kB, Never Used Indexes, Never Used Indexes, Never Used Indexes, table namespaces
DROP INDEX CONCURRENTLY index_on_namespaces_lower_name; -- 336 MB, index_on_namespaces_lower_path, table namespaces
DROP INDEX CONCURRENTLY index_on_namespaces_lower_path; -- 257 MB, index_on_namespaces_lower_name, table namespaces
DROP INDEX CONCURRENTLY index_notification_settings_on_user_id; -- 406 MB, index_notifications_on_user_id_and_source_id_and_source_type, table notification_settings
DROP INDEX CONCURRENTLY index_pages_domain_acme_orders_on_challenge_token; -- 56 kB, Never Used Indexes, table pages_domain_acme_orders
DROP INDEX CONCURRENTLY index_pages_domains_on_project_id; -- 3576 kB, index_pages_domains_on_project_id_and_enabled_until, table pages_domains
DROP INDEX CONCURRENTLY index_pages_domains_on_verified_at; -- 4584 kB, index_pages_domains_on_verified_at_and_enabled_until, table pages_domains
DROP INDEX CONCURRENTLY index_pages_domains_on_verified_at_and_enabled_until; -- 5920 kB, Never Used Indexes, table pages_domains
DROP INDEX CONCURRENTLY index_pool_repositories_on_shard_id; -- 296 kB, Never Used Indexes, table pool_repositories
DROP INDEX CONCURRENTLY index_project_custom_attributes_on_key_and_value; -- 16 kB, Never Used Indexes, table project_custom_attributes
DROP INDEX CONCURRENTLY index_project_feature_usages_on_project_id; -- 1064 kB, project_feature_usages_pkey, table project_feature_usages
DROP INDEX CONCURRENTLY index_project_mirror_data_on_jid; -- 236 MB, Never Used Indexes, table project_mirror_data
DROP INDEX CONCURRENTLY index_project_mirror_data_on_last_successful_update_at; -- 390 MB, Never Used Indexes, table project_mirror_data
DROP INDEX CONCURRENTLY index_project_repositories_on_shard_id; -- 259 MB, Never Used Indexes, table project_repositories
DROP INDEX CONCURRENTLY idx_repository_states_on_last_repository_verification_ran_at; -- 227 MB, Never Used Indexes, table project_repository_states
DROP INDEX CONCURRENTLY idx_repository_states_on_last_wiki_verification_ran_at; -- 243 MB, Never Used Indexes, table project_repository_states
DROP INDEX CONCURRENTLY idx_repository_states_on_repository_failure_partial; -- 16 kB, Never Used Indexes, table project_repository_states
DROP INDEX CONCURRENTLY idx_repository_states_on_wiki_failure_partial; -- 8192 bytes, Never Used Indexes, table project_repository_states
DROP INDEX CONCURRENTLY idx_repository_states_outdated_checksums; -- 23 MB, Never Used Indexes, index_project_repository_states_on_project_id, idx_repository_states_on_last_wiki_verification_ran_at, idx_repository_states_on_last_repository_verification_ran_at, table project_repository_states
DROP INDEX CONCURRENTLY idx_project_repository_check_partial; -- 1191 MB, Never Used Indexes, table projects
DROP INDEX CONCURRENTLY index_on_projects_lower_path; -- 589 MB, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, index_projects_on_lower_name, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, table projects
DROP INDEX CONCURRENTLY index_on_projects_path; -- 564 MB, index_projects_on_runners_token, table projects
DROP INDEX CONCURRENTLY index_projects_on_id_partial_for_visibility; -- 59 MB, projects_pkey, table projects
DROP INDEX CONCURRENTLY index_projects_on_last_repository_check_at; -- 16 kB, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, table projects
DROP INDEX CONCURRENTLY index_projects_on_last_repository_check_failed; -- 3762 MB, Never Used Indexes, Never Used Indexes, table projects
DROP INDEX CONCURRENTLY index_projects_on_lower_name; -- 592 MB, index_on_projects_lower_path, table projects
DROP INDEX CONCURRENTLY index_projects_on_mirror_and_mirror_trigger_builds_both_true; -- 3416 kB, projects_pkey, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, index_projects_on_star_count, index_projects_on_visibility_level, index_projects_on_id_partial_for_visibility, idx_projects_on_repository_storage_last_repository_updated_at, table projects
DROP INDEX CONCURRENTLY index_projects_on_mirror_last_successful_update_at; -- 3750 MB, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, table projects
DROP INDEX CONCURRENTLY index_projects_on_pending_delete; -- 3893 MB, Never Used Indexes, Never Used Indexes, table projects
DROP INDEX CONCURRENTLY index_projects_on_repository_storage; -- 4725 MB, idx_project_repository_check_partial, table projects
DROP INDEX CONCURRENTLY index_prometheus_metrics_on_group; -- 40 kB, Never Used Indexes, table prometheus_metrics
DROP INDEX CONCURRENTLY index_protected_branches_on_project_id; -- 514 MB, code_owner_approval_required, table protected_branches
DROP INDEX CONCURRENTLY index_protected_environments_on_project_id; -- 56 kB, index_protected_environments_on_project_id_and_name, table protected_environments
DROP INDEX CONCURRENTLY index_protected_tags_on_project_id; -- 544 kB, index_protected_tags_on_project_id_and_name, table protected_tags
DROP INDEX CONCURRENTLY index_remote_mirrors_on_last_successful_update_at; -- 37 MB, Never Used Indexes, table remote_mirrors
DROP INDEX CONCURRENTLY index_snippets_on_updated_at; -- 6552 kB, Never Used Indexes, table snippets
DROP INDEX CONCURRENTLY index_taggings_on_tag_id; -- 2823 MB, taggings_idx, table taggings
DROP INDEX CONCURRENTLY index_taggings_on_taggable_id_and_taggable_type; -- 4573 MB, index_taggings_on_taggable_id_and_taggable_type_and_context, table taggings
DROP INDEX CONCURRENTLY index_term_agreements_on_term_id; -- 89 MB, Never Used Indexes, table term_agreements
DROP INDEX CONCURRENTLY index_term_agreements_on_user_id; -- 76 MB, Never Used Indexes, term_agreements_unique_index, table term_agreements
DROP INDEX CONCURRENTLY index_timelogs_on_user_id; -- 31 MB, Never Used Indexes, table timelogs
DROP INDEX CONCURRENTLY index_todos_on_user_id; -- 910 MB, index_todos_on_user_id_and_id_pending, index_todos_on_user_id_and_id_done, table todos
DROP INDEX CONCURRENTLY index_todos_on_user_id_and_id_done; -- 725 MB, index_todos_on_user_id_and_id_pending, table todos
DROP INDEX CONCURRENTLY index_todos_on_user_id_and_id_pending; -- 214 MB, index_todos_on_user_id_and_id_done, table todos
DROP INDEX CONCURRENTLY index_u2f_registrations_on_key_handle; -- 2432 kB, Never Used Indexes, table u2f_registrations
DROP INDEX CONCURRENTLY index_user_callouts_on_user_id; -- 11 MB, Never Used Indexes, index_user_callouts_on_user_id_and_feature_name, table user_callouts
DROP INDEX CONCURRENTLY index_user_statuses_on_user_id; -- 2280 kB, user_statuses_pkey, table user_statuses
DROP INDEX CONCURRENTLY index_on_users_lower_email; -- 313 MB, index_on_users_name_lower, index_on_users_lower_username, table users
DROP INDEX CONCURRENTLY index_on_users_lower_username; -- 215 MB, index_on_users_lower_email, index_on_users_name_lower, table users
DROP INDEX CONCURRENTLY index_on_users_name_lower; -- 243 MB, index_on_users_lower_username, index_on_users_lower_email, table users
DROP INDEX CONCURRENTLY index_users_on_accepted_term_id; -- 1681 MB, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, table users
DROP INDEX CONCURRENTLY index_users_on_created_at; -- 281 MB, Never Used Indexes, Never Used Indexes, table users
DROP INDEX CONCURRENTLY index_users_on_email_trigram; -- 382 MB, index_users_on_username_trigram, table users
DROP INDEX CONCURRENTLY index_users_on_state; -- 1841 MB, index_users_on_state_and_internal, index_users_on_state_and_internal_ee, table users
DROP INDEX CONCURRENTLY index_users_on_state_and_internal; -- 391 MB, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, index_users_on_state, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, Never Used Indexes, index_users_on_state_and_internal_ee, Never Used Indexes, Never Used Indexes, table users
DROP INDEX CONCURRENTLY index_users_on_state_and_internal_ee; -- 392 MB, index_users_on_state_and_internal, index_users_on_state, table users

"UNDO" database migration code

CREATE INDEX CONCURRENTLY index_analytics_ca_group_stages_on_group_id ON public.analytics_cycle_analytics_group_stages USING btree (group_id); -- table analytics_cycle_analytics_group_stages
CREATE INDEX CONCURRENTLY index_analytics_ca_project_stages_on_project_id ON public.analytics_cycle_analytics_project_stages USING btree (project_id); -- table analytics_cycle_analytics_project_stages
CREATE INDEX CONCURRENTLY index_approval_merge_request_rules_1 ON public.approval_merge_request_rules USING btree (merge_request_id, code_owner); -- table approval_merge_request_rules
CREATE INDEX CONCURRENTLY index_approval_rules_code_owners_rule_type ON public.approval_merge_request_rules USING btree (merge_request_id, rule_type) WHERE (rule_type = 2); -- table approval_merge_request_rules
CREATE INDEX CONCURRENTLY index_approval_project_rules_on_project_id ON public.approval_project_rules USING btree (project_id); -- table approval_project_rules
CREATE INDEX CONCURRENTLY analytics_index_audit_events_on_created_at_and_author_id ON public.audit_events USING btree (created_at, author_id); -- table audit_events
CREATE INDEX CONCURRENTLY index_board_group_recent_visits_on_user_id ON public.board_group_recent_visits USING btree (user_id); -- table board_group_recent_visits
CREATE INDEX CONCURRENTLY index_board_project_recent_visits_on_user_id ON public.board_project_recent_visits USING btree (user_id); -- table board_project_recent_visits
CREATE INDEX CONCURRENTLY index_ci_builds_on_artifacts_expire_at ON public.ci_builds USING btree (artifacts_expire_at) WHERE (artifacts_file <> ''::text); -- table ci_builds
CREATE INDEX CONCURRENTLY index_ci_builds_on_protected ON public.ci_builds USING btree (protected); -- table ci_builds
CREATE INDEX CONCURRENTLY index_ci_builds_on_queued_at ON public.ci_builds USING btree (queued_at); -- table ci_builds
CREATE INDEX CONCURRENTLY index_ci_builds_on_stage_id ON public.ci_builds USING btree (stage_id); -- table ci_builds
CREATE INDEX CONCURRENTLY index_ci_builds_metadata_on_build_id_and_interruptible ON public.ci_builds_metadata USING btree (build_id) WHERE (interruptible = true); -- table ci_builds_metadata
CREATE INDEX CONCURRENTLY index_ci_job_artifacts_on_file_store ON public.ci_job_artifacts USING btree (file_store); -- table ci_job_artifacts
CREATE INDEX CONCURRENTLY index_ci_job_artifacts_on_project_id ON public.ci_job_artifacts USING btree (project_id); -- table ci_job_artifacts
CREATE INDEX CONCURRENTLY index_ci_job_artifacts_on_project_id_for_security_reports ON public.ci_job_artifacts USING btree (project_id) WHERE (file_type = ANY (ARRAY[5, 6, 7, 8])); -- table ci_job_artifacts
CREATE INDEX CONCURRENTLY index_ci_pipeline_schedules_on_next_run_at_and_active ON public.ci_pipeline_schedules USING btree (next_run_at, active); -- table ci_pipeline_schedules
CREATE INDEX CONCURRENTLY index_ci_pipelines_on_project_id ON public.ci_pipelines USING btree (project_id); -- table ci_pipelines
CREATE INDEX CONCURRENTLY index_ci_pipelines_on_status ON public.ci_pipelines USING btree (status); -- table ci_pipelines
CREATE INDEX CONCURRENTLY index_ci_runners_on_contacted_at ON public.ci_runners USING btree (contacted_at); -- table ci_runners
CREATE INDEX CONCURRENTLY index_ci_runners_on_is_shared ON public.ci_runners USING btree (is_shared); -- table ci_runners
CREATE INDEX CONCURRENTLY index_ci_runners_on_locked ON public.ci_runners USING btree (locked); -- table ci_runners
CREATE INDEX CONCURRENTLY index_ci_runners_on_token ON public.ci_runners USING btree (token); -- table ci_runners
CREATE INDEX CONCURRENTLY index_ci_stages_on_pipeline_id ON public.ci_stages USING btree (pipeline_id); -- table ci_stages
CREATE INDEX CONCURRENTLY index_ci_stages_on_pipeline_id_and_position ON public.ci_stages USING btree (pipeline_id, "position"); -- table ci_stages
CREATE INDEX CONCURRENTLY index_clusters_kubernetes_namespaces_on_cluster_id ON public.clusters_kubernetes_namespaces USING btree (cluster_id); -- table clusters_kubernetes_namespaces
CREATE INDEX CONCURRENTLY index_container_repositories_on_project_id ON public.container_repositories USING btree (project_id); -- table container_repositories
CREATE INDEX CONCURRENTLY index_deploy_tokens_on_token_and_expires_at_and_id ON public.deploy_tokens USING btree (token, expires_at, id) WHERE (revoked IS FALSE); -- table deploy_tokens
CREATE INDEX CONCURRENTLY index_deployments_on_created_at ON public.deployments USING btree (created_at); -- table deployments
CREATE INDEX CONCURRENTLY index_deployments_on_project_id_and_status ON public.deployments USING btree (project_id, status); -- table deployments
CREATE INDEX CONCURRENTLY partial_index_deployments_for_legacy_successful_deployments ON public.deployments USING btree (id) WHERE ((finished_at IS NULL) AND (status = 2)); -- table deployments
CREATE INDEX CONCURRENTLY index_design_management_designs_versions_on_design_id ON public.design_management_designs_versions USING btree (design_id); -- table design_management_designs_versions
CREATE INDEX CONCURRENTLY index_design_management_designs_versions_on_event ON public.design_management_designs_versions USING btree (event); -- table design_management_designs_versions
CREATE INDEX CONCURRENTLY index_epics_on_end_date ON public.epics USING btree (end_date); -- table epics
CREATE INDEX CONCURRENTLY index_epics_on_start_date ON public.epics USING btree (start_date); -- table epics
CREATE INDEX CONCURRENTLY index_milestone ON public.epics USING btree (milestone_id); -- table epics
CREATE INDEX CONCURRENTLY index_geo_node_namespace_links_on_geo_node_id ON public.geo_node_namespace_links USING btree (geo_node_id); -- table geo_node_namespace_links
CREATE INDEX CONCURRENTLY index_issue_links_on_source_id ON public.issue_links USING btree (source_id); -- table issue_links
CREATE INDEX CONCURRENTLY idx_jira_connect_subscriptions_on_installation_id ON public.jira_connect_subscriptions USING btree (jira_connect_installation_id); -- table jira_connect_subscriptions
CREATE INDEX CONCURRENTLY index_label_priorities_on_priority ON public.label_priorities USING btree (priority); -- table label_priorities
CREATE INDEX CONCURRENTLY index_list_user_preferences_on_user_id ON public.list_user_preferences USING btree (user_id); -- table list_user_preferences
CREATE INDEX CONCURRENTLY index_members_on_access_level ON public.members USING btree (access_level); -- table members
CREATE INDEX CONCURRENTLY index_merge_request_assignees_on_merge_request_id ON public.merge_request_assignees USING btree (merge_request_id); -- table merge_request_assignees
CREATE INDEX CONCURRENTLY index_merge_request_diffs_on_merge_request_id_and_id ON public.merge_request_diffs USING btree (merge_request_id, id); -- table merge_request_diffs
CREATE INDEX CONCURRENTLY index_merge_request_diffs_on_merge_request_id_and_id_partial ON public.merge_request_diffs USING btree (merge_request_id, id) WHERE ((NOT stored_externally) OR (stored_externally IS NULL)); -- table merge_request_diffs
CREATE INDEX CONCURRENTLY index_merge_request_metrics ON public.merge_request_metrics USING btree (merge_request_id); -- table merge_request_metrics
CREATE INDEX CONCURRENTLY index_merge_request_metrics_on_first_deployed_to_production_at ON public.merge_request_metrics USING btree (first_deployed_to_production_at); -- table merge_request_metrics
CREATE INDEX CONCURRENTLY index_merge_request_metrics_on_latest_closed_at ON public.merge_request_metrics USING btree (latest_closed_at) WHERE (latest_closed_at IS NOT NULL); -- table merge_request_metrics
CREATE INDEX CONCURRENTLY index_merge_requests_on_source_project_and_branch_state_opened ON public.merge_requests USING btree (source_project_id, source_branch) WHERE ((state)::text = 'opened'::text); -- table merge_requests
CREATE INDEX CONCURRENTLY index_merge_requests_on_source_project_id_and_source_branch ON public.merge_requests USING btree (source_project_id, source_branch); -- table merge_requests
CREATE INDEX CONCURRENTLY index_merge_requests_on_state_and_merge_status ON public.merge_requests USING btree (state, merge_status) WHERE (((state)::text = 'opened'::text) AND ((merge_status)::text = 'can_be_merged'::text)); -- table merge_requests
CREATE INDEX CONCURRENTLY index_merge_requests_on_target_project_id_and_iid_opened ON public.merge_requests USING btree (target_project_id, iid) WHERE ((state)::text = 'opened'::text); -- table merge_requests
CREATE INDEX CONCURRENTLY index_merge_requests_on_title ON public.merge_requests USING btree (title); -- table merge_requests
CREATE UNIQUE INDEX index_namespace_aggregation_schedules_on_namespace_id ON public.namespace_aggregation_schedules USING btree (namespace_id); -- table namespace_aggregation_schedules
CREATE UNIQUE INDEX index_namespace_root_storage_statistics_on_namespace_id ON public.namespace_root_storage_statistics USING btree (namespace_id); -- table namespace_root_storage_statistics
CREATE INDEX CONCURRENTLY index_namespaces_on_created_at ON public.namespaces USING btree (created_at); -- table namespaces
CREATE INDEX CONCURRENTLY index_namespaces_on_ldap_sync_last_successful_update_at ON public.namespaces USING btree (ldap_sync_last_successful_update_at); -- table namespaces
CREATE INDEX CONCURRENTLY index_namespaces_on_ldap_sync_last_update_at ON public.namespaces USING btree (ldap_sync_last_update_at); -- table namespaces
CREATE INDEX CONCURRENTLY index_namespaces_on_path ON public.namespaces USING btree (path); -- table namespaces
CREATE INDEX CONCURRENTLY index_namespaces_on_plan_id ON public.namespaces USING btree (plan_id); -- table namespaces
CREATE INDEX CONCURRENTLY index_namespaces_on_require_two_factor_authentication ON public.namespaces USING btree (require_two_factor_authentication); -- table namespaces
CREATE INDEX CONCURRENTLY index_namespaces_on_shared_and_extra_runners_minutes_limit ON public.namespaces USING btree (shared_runners_minutes_limit, extra_shared_runners_minutes_limit); -- table namespaces
CREATE INDEX CONCURRENTLY index_namespaces_on_trial_ends_on ON public.namespaces USING btree (trial_ends_on) WHERE (trial_ends_on IS NOT NULL); -- table namespaces
CREATE INDEX CONCURRENTLY index_on_namespaces_lower_name ON public.namespaces USING btree (lower((name)::text)); -- table namespaces
CREATE INDEX CONCURRENTLY index_on_namespaces_lower_path ON public.namespaces USING btree (lower((path)::text)); -- table namespaces
CREATE INDEX CONCURRENTLY index_notification_settings_on_user_id ON public.notification_settings USING btree (user_id); -- table notification_settings
CREATE INDEX CONCURRENTLY index_pages_domain_acme_orders_on_challenge_token ON public.pages_domain_acme_orders USING btree (challenge_token); -- table pages_domain_acme_orders
CREATE INDEX CONCURRENTLY index_pages_domains_on_project_id ON public.pages_domains USING btree (project_id); -- table pages_domains
CREATE INDEX CONCURRENTLY index_pages_domains_on_verified_at ON public.pages_domains USING btree (verified_at); -- table pages_domains
CREATE INDEX CONCURRENTLY index_pages_domains_on_verified_at_and_enabled_until ON public.pages_domains USING btree (verified_at, enabled_until); -- table pages_domains
CREATE INDEX CONCURRENTLY index_pool_repositories_on_shard_id ON public.pool_repositories USING btree (shard_id); -- table pool_repositories
CREATE INDEX CONCURRENTLY index_project_custom_attributes_on_key_and_value ON public.project_custom_attributes USING btree (key, value); -- table project_custom_attributes
CREATE INDEX CONCURRENTLY index_project_feature_usages_on_project_id ON public.project_feature_usages USING btree (project_id); -- table project_feature_usages
CREATE INDEX CONCURRENTLY index_project_mirror_data_on_jid ON public.project_mirror_data USING btree (jid); -- table project_mirror_data
CREATE INDEX CONCURRENTLY index_project_mirror_data_on_last_successful_update_at ON public.project_mirror_data USING btree (last_successful_update_at); -- table project_mirror_data
CREATE INDEX CONCURRENTLY index_project_repositories_on_shard_id ON public.project_repositories USING btree (shard_id); -- table project_repositories
CREATE INDEX CONCURRENTLY idx_repository_states_on_last_repository_verification_ran_at ON public.project_repository_states USING btree (project_id, last_repository_verification_ran_at) WHERE ((repository_verification_checksum IS NOT NULL) AND (last_repository_verification_failure IS NULL)); -- table project_repository_states
CREATE INDEX CONCURRENTLY idx_repository_states_on_last_wiki_verification_ran_at ON public.project_repository_states USING btree (project_id, last_wiki_verification_ran_at) WHERE ((wiki_verification_checksum IS NOT NULL) AND (last_wiki_verification_failure IS NULL)); -- table project_repository_states
CREATE INDEX CONCURRENTLY idx_repository_states_on_repository_failure_partial ON public.project_repository_states USING btree (last_repository_verification_failure) WHERE (last_repository_verification_failure IS NOT NULL); -- table project_repository_states
CREATE INDEX CONCURRENTLY idx_repository_states_on_wiki_failure_partial ON public.project_repository_states USING btree (last_wiki_verification_failure) WHERE (last_wiki_verification_failure IS NOT NULL); -- table project_repository_states
CREATE INDEX CONCURRENTLY idx_repository_states_outdated_checksums ON public.project_repository_states USING btree (project_id) WHERE (((repository_verification_checksum IS NULL) AND (last_repository_verification_failure IS NULL)) OR ((wiki_verification_checksum IS NULL) AND (last_wiki_verification_failure IS NULL))); -- table project_repository_states
CREATE INDEX CONCURRENTLY idx_project_repository_check_partial ON public.projects USING btree (repository_storage, created_at) WHERE (last_repository_check_at IS NULL); -- table projects
CREATE INDEX CONCURRENTLY index_on_projects_lower_path ON public.projects USING btree (lower((path)::text)); -- table projects
CREATE INDEX CONCURRENTLY index_on_projects_path ON public.projects USING btree (path); -- table projects
CREATE UNIQUE INDEX index_projects_on_id_partial_for_visibility ON public.projects USING btree (id) WHERE (visibility_level = ANY (ARRAY[10, 20])); -- table projects
CREATE INDEX CONCURRENTLY index_projects_on_last_repository_check_at ON public.projects USING btree (last_repository_check_at) WHERE (last_repository_check_at IS NOT NULL); -- table projects
CREATE INDEX CONCURRENTLY index_projects_on_last_repository_check_failed ON public.projects USING btree (last_repository_check_failed); -- table projects
CREATE INDEX CONCURRENTLY index_projects_on_lower_name ON public.projects USING btree (lower((name)::text)); -- table projects
CREATE INDEX CONCURRENTLY index_projects_on_mirror_and_mirror_trigger_builds_both_true ON public.projects USING btree (id) WHERE ((mirror IS TRUE) AND (mirror_trigger_builds IS TRUE)); -- table projects
CREATE INDEX CONCURRENTLY index_projects_on_mirror_last_successful_update_at ON public.projects USING btree (mirror_last_successful_update_at); -- table projects
CREATE INDEX CONCURRENTLY index_projects_on_pending_delete ON public.projects USING btree (pending_delete); -- table projects
CREATE INDEX CONCURRENTLY index_projects_on_repository_storage ON public.projects USING btree (repository_storage); -- table projects
CREATE INDEX CONCURRENTLY index_prometheus_metrics_on_group ON public.prometheus_metrics USING btree ("group"); -- table prometheus_metrics
CREATE INDEX CONCURRENTLY index_protected_branches_on_project_id ON public.protected_branches USING btree (project_id); -- table protected_branches
CREATE INDEX CONCURRENTLY index_protected_environments_on_project_id ON public.protected_environments USING btree (project_id); -- table protected_environments
CREATE INDEX CONCURRENTLY index_protected_tags_on_project_id ON public.protected_tags USING btree (project_id); -- table protected_tags
CREATE INDEX CONCURRENTLY index_remote_mirrors_on_last_successful_update_at ON public.remote_mirrors USING btree (last_successful_update_at); -- table remote_mirrors
CREATE INDEX CONCURRENTLY index_snippets_on_updated_at ON public.snippets USING btree (updated_at); -- table snippets
CREATE INDEX CONCURRENTLY index_taggings_on_tag_id ON public.taggings USING btree (tag_id); -- table taggings
CREATE INDEX CONCURRENTLY index_taggings_on_taggable_id_and_taggable_type ON public.taggings USING btree (taggable_id, taggable_type); -- table taggings
CREATE INDEX CONCURRENTLY index_term_agreements_on_term_id ON public.term_agreements USING btree (term_id); -- table term_agreements
CREATE INDEX CONCURRENTLY index_term_agreements_on_user_id ON public.term_agreements USING btree (user_id); -- table term_agreements
CREATE INDEX CONCURRENTLY index_timelogs_on_user_id ON public.timelogs USING btree (user_id); -- table timelogs
CREATE INDEX CONCURRENTLY index_todos_on_user_id ON public.todos USING btree (user_id); -- table todos
CREATE INDEX CONCURRENTLY index_todos_on_user_id_and_id_done ON public.todos USING btree (user_id, id) WHERE ((state)::text = 'done'::text); -- table todos
CREATE INDEX CONCURRENTLY index_todos_on_user_id_and_id_pending ON public.todos USING btree (user_id, id) WHERE ((state)::text = 'pending'::text); -- table todos
CREATE INDEX CONCURRENTLY index_u2f_registrations_on_key_handle ON public.u2f_registrations USING btree (key_handle); -- table u2f_registrations
CREATE INDEX CONCURRENTLY index_user_callouts_on_user_id ON public.user_callouts USING btree (user_id); -- table user_callouts
CREATE INDEX CONCURRENTLY index_user_statuses_on_user_id ON public.user_statuses USING btree (user_id); -- table user_statuses
CREATE INDEX CONCURRENTLY index_on_users_lower_email ON public.users USING btree (lower((email)::text)); -- table users
CREATE INDEX CONCURRENTLY index_on_users_lower_username ON public.users USING btree (lower((username)::text)); -- table users
CREATE INDEX CONCURRENTLY index_on_users_name_lower ON public.users USING btree (lower((name)::text)); -- table users
CREATE INDEX CONCURRENTLY index_users_on_accepted_term_id ON public.users USING btree (accepted_term_id); -- table users
CREATE INDEX CONCURRENTLY index_users_on_created_at ON public.users USING btree (created_at); -- table users
CREATE INDEX CONCURRENTLY index_users_on_email_trigram ON public.users USING gin (email gin_trgm_ops); -- table users
CREATE INDEX CONCURRENTLY index_users_on_state ON public.users USING btree (state); -- table users
CREATE INDEX CONCURRENTLY index_users_on_state_and_internal ON public.users USING btree (state) WHERE (ghost IS NOT TRUE); -- table users
CREATE INDEX CONCURRENTLY index_users_on_state_and_internal_ee ON public.users USING btree (state) WHERE ((ghost IS NOT TRUE) AND (bot_type IS NULL)); -- table users

---

  Table of contents

H003 Non-indexed Foreign Keys

Observations

Data collected: 2019-09-30 13:37:57 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.101)

Num	Schema name	Table name	FK name	Issue	Table mb	writes	Table scans	Parent name	Parent mb	Parent writes	Cols list	Indexdef
1	public	ci_builds	fk_87f4cefcda	questionable index	298553	61590381	0	ci_pipelines	16866	17087895	[upstream_pipeline_id]	CREATE INDEX index_ci_builds_on_upstream_pipeline_id ON public.ci_builds USING btree (upstream_pipeline_id) WHERE (upstream_pipeline_id IS NOT NULL)
2	public	ci_builds	fk_d3130c9a7f	questionable index	298553	61590381	0	ci_pipelines	16866	17087895	[commit_id]	CREATE INDEX index_ci_builds_on_commit_id_and_artifacts_expireatandidpartial ON public.ci_builds USING btree (commit_id, artifacts_expire_at, id) WHERE (((type)::text = 'Ci::Build'::text) AND ((retried = false) OR (retried IS NULL)) AND ((name)::text = ANY ((ARRAY['sast'::character varying, 'dependency_scanning'::character varying, 'sast:container'::character varying, 'container_scanning'::character varying, 'dast'::character varying])::text[])))
3	public	ci_builds	fk_d3130c9a7f	questionable index	298553	61590381	0	ci_pipelines	16866	17087895	[commit_id]	CREATE INDEX index_ci_builds_on_commit_id_and_type_and_ref ON public.ci_builds USING btree (commit_id, type, ref)
4	public	ci_builds	fk_d3130c9a7f	questionable index	298553	61590381	0	ci_pipelines	16866	17087895	[commit_id]	CREATE INDEX index_ci_builds_on_commit_id_and_stage_idx_and_created_at ON public.ci_builds USING btree (commit_id, stage_idx, created_at)
5	public	ci_builds	fk_d3130c9a7f	questionable index	298553	61590381	0	ci_pipelines	16866	17087895	[commit_id]	CREATE INDEX index_ci_builds_on_commit_id_and_type_and_name_and_ref ON public.ci_builds USING btree (commit_id, type, name, ref)
6	public	ci_builds	fk_d3130c9a7f	questionable index	298553	61590381	0	ci_pipelines	16866	17087895	[commit_id]	CREATE INDEX index_ci_builds_on_commit_id_and_status_and_type ON public.ci_builds USING btree (commit_id, status, type)
7	public	events	fk_61fbf6ca48	questionable index	29432	21500241	4	namespaces	718	896445	[group_id]	CREATE INDEX index_events_on_group_id_partial ON public.events USING btree (group_id) WHERE (group_id IS NOT NULL)
8	public	ci_pipelines	fk_190998ef09	questionable index	16866	17087895	2	external_pull_requests	0	12276	[external_pull_request_id]	CREATE INDEX index_ci_pipelines_on_external_pull_request_id ON public.ci_pipelines USING btree (external_pull_request_id) WHERE (external_pull_request_id IS NOT NULL)
9	public	ci_pipelines	fk_a23be95014	questionable index	16866	17087895	2	merge_requests	13737	27847227	[merge_request_id]	CREATE INDEX index_ci_pipelines_on_merge_request_id ON public.ci_pipelines USING btree (merge_request_id) WHERE (merge_request_id IS NOT NULL)
10	public	merge_requests	fk_641731faff	questionable index	13737	27847227	6	users	1996	15477966	[updated_by_id]	CREATE INDEX index_merge_requests_on_updated_by_id ON public.merge_requests USING btree (updated_by_id) WHERE (updated_by_id IS NOT NULL)
11	public	merge_requests	fk_ad525e1f87	questionable index	13737	27847227	6	users	1996	15477966	[merge_user_id]	CREATE INDEX index_merge_requests_on_merge_user_id ON public.merge_requests USING btree (merge_user_id) WHERE (merge_user_id IS NOT NULL)
12	public	issues	fk_9c4516d665	questionable index	12856	6677639	2222	issues	12856	6677639	[duplicated_to_id]	CREATE INDEX index_issues_on_duplicated_to_id ON public.issues USING btree (duplicated_to_id) WHERE (duplicated_to_id IS NOT NULL)
13	public	issues	fk_a194299be1	questionable index	12856	6677639	2222	issues	12856	6677639	[moved_to_id]	CREATE INDEX index_issues_on_moved_to_id ON public.issues USING btree (moved_to_id) WHERE (moved_to_id IS NOT NULL)
14	public	issues	fk_ffed080f01	questionable index	12856	6677639	2222	users	1996	15477966	[updated_by_id]	CREATE INDEX index_issues_on_updated_by_id ON public.issues USING btree (updated_by_id) WHERE (updated_by_id IS NOT NULL)
15	public	projects	fk_6e5c14658a	questionable index	2850	22381071	93440	pool_repositories	1	11033	[pool_repository_id]	CREATE INDEX index_projects_on_pool_repository_id ON public.projects USING btree (pool_repository_id) WHERE (pool_repository_id IS NOT NULL)
16	public	project_authorizations	fk_rails_11e7aa3ed9	questionable index	1727	6398788	0	users	1996	15477966	[user_id]	CREATE UNIQUE INDEX index_project_authorizations_on_user_id_project_id_access_level ON public.project_authorizations USING btree (user_id, project_id, access_level)
17	public	labels	fk_rails_c1ac5161d8	questionable index	928	378327	20	namespaces	718	896445	[group_id]	CREATE UNIQUE INDEX index_labels_on_group_id_and_project_id_and_title ON public.labels USING btree (group_id, project_id, title)
18	public	vulnerability_occurrences	fk_rails_90fed4faba	questionable index	731	144337	0	projects	2850	22381071	[project_id]	CREATE UNIQUE INDEX index_vulnerability_occurrences_on_unique_keys ON public.vulnerability_occurrences USING btree (project_id, primary_identifier_id, location_fingerprint, scanner_id)
19	public	namespaces	fk_e7a0b20a6b	questionable index	718	896445	20086	namespaces	718	896445	[custom_project_templates_group_id]	CREATE INDEX index_namespaces_on_custom_project_templates_group_id_and_type ON public.namespaces USING btree (custom_project_templates_group_id, type) WHERE (custom_project_templates_group_id IS NOT NULL)
20	public	ci_variables	fk_ada5eb64b3	questionable index	225	824385	0	projects	2850	22381071	[project_id]	CREATE UNIQUE INDEX index_ci_variables_on_project_id_and_key_and_environment_scope ON public.ci_variables USING btree (project_id, key, environment_scope)

Conclusions

Recommendations

---

  Table of contents

K001 Globally Aggregated Query Metrics

Observations

Data collected: 2019-09-30 13:57:28 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.101)

Start: 2019-09-30T13:06:11.519966+00:00
End: 2019-09-30T13:37:59.792427+00:00
Period seconds: 1908.27246
Period age: 00:31:48.272461

Error (calls): 0.00 (0.00%)
Error (total time): 0.00 (0.00%)

Calls	Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
29,701,831 15.57K/sec 1.00/call 100.00%	6,113,505.21 ms 3203.686 ms/sec 0.206 ms/call 100.00%	126,260,839 66.17K/sec 4.25/call 100.00%	1,225,683,429 blks 642.31K blks/sec 41.27 blks/call 100.00%	1,204,989 blks 631.46 blks/sec 0.04 blks/call 100.00%	2,224,707 blks 1.17K blks/sec 0.07 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	563,630.82 ms 295.362 ms/sec 0.019 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica servers:

Replica (10.220.16.102)

Start: 2019-09-30T13:06:14.399331+00:00
End: 2019-09-30T13:39:13.29931+00:00
Period seconds: 1978.89998
Period age: 00:32:58.899979

Calls	Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
9,075,973 4.59K/sec 1.00/call 100.00%	3,216,173.56 ms 1625.233 ms/sec 0.354 ms/call 100.00%	30,031,897 15.18K/sec 3.31/call 100.00%	1,218,572,413 blks 615.79K blks/sec 134.26 blks/call 100.00%	1,544,001 blks 780.23 blks/sec 0.17 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	156,240.06 ms 78.953 ms/sec 0.017 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.103)

Start: 2019-09-30T13:06:17.052557+00:00
End: 2019-09-30T13:40:33.301941+00:00
Period seconds: 2056.24938
Period age: 00:34:16.249384

Calls	Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
9,623,347 4.69K/sec 1.00/call 100.00%	3,461,444.67 ms 1683.378 ms/sec 0.360 ms/call 100.00%	31,127,854 15.14K/sec 3.23/call 100.00%	1,357,604,276 blks 660.24K blks/sec 141.07 blks/call 100.00%	1,543,432 blks 750.61 blks/sec 0.16 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	152,011.98 ms 73.927 ms/sec 0.016 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.104)

Start: 2019-09-30T13:06:20.434612+00:00
End: 2019-09-30T13:42:07.554666+00:00
Period seconds: 2147.12006
Period age: 00:35:47.120054

Calls	Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
9,969,462 4.65K/sec 1.00/call 100.00%	3,258,244.34 ms 1517.495 ms/sec 0.327 ms/call 100.00%	32,380,580 15.09K/sec 3.25/call 100.00%	1,402,260,910 blks 653.09K blks/sec 140.66 blks/call 100.00%	1,864,136 blks 868.20 blks/sec 0.19 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	192,584.40 ms 89.694 ms/sec 0.019 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.105)

Start: 2019-09-30T13:06:23.43192+00:00
End: 2019-09-30T13:44:07.798909+00:00
Period seconds: 2264.36699
Period age: 00:37:44.366989

Calls	Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
10,512,382 4.65K/sec 1.00/call 100.00%	3,289,304.64 ms 1452.638 ms/sec 0.313 ms/call 100.00%	34,381,232 15.19K/sec 3.27/call 100.00%	1,427,241,439 blks 630.31K blks/sec 135.77 blks/call 100.00%	1,795,630 blks 792.99 blks/sec 0.17 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	149,260.34 ms 65.917 ms/sec 0.014 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.106)

Start: 2019-09-30T13:06:25.816873+00:00
End: 2019-09-30T13:46:29.215889+00:00
Period seconds: 2403.39902
Period age: 00:40:03.399016

Calls	Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
11,314,919 4.71K/sec 1.00/call 100.00%	3,283,939.05 ms 1366.373 ms/sec 0.290 ms/call 100.00%	36,477,028 15.18K/sec 3.22/call 100.00%	1,351,662,312 blks 562.40K blks/sec 119.46 blks/call 100.00%	2,087,432 blks 868.53 blks/sec 0.18 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	129,672.34 ms 53.954 ms/sec 0.011 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.107)

Start: 2019-09-30T13:06:28.521327+00:00
End: 2019-09-30T13:49:34.408897+00:00
Period seconds: 2585.88757
Period age: 00:43:05.88757

Calls	Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
12,071,806 4.67K/sec 1.00/call 100.00%	3,955,324.76 ms 1529.581 ms/sec 0.328 ms/call 100.00%	39,290,972 15.20K/sec 3.25/call 100.00%	1,631,062,292 blks 630.76K blks/sec 135.11 blks/call 100.00%	1,992,208 blks 770.42 blks/sec 0.17 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	123,828.77 ms 47.886 ms/sec 0.010 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.108)

Start: 2019-09-30T13:06:29.835321+00:00
End: 2019-09-30T13:53:16.626097+00:00
Period seconds: 2806.79078
Period age: 00:46:46.790776

Calls	Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
13,673,685 4.88K/sec 1.00/call 100.00%	4,802,648.32 ms 1711.082 ms/sec 0.351 ms/call 100.00%	41,000,330 14.61K/sec 3.00/call 100.00%	1,874,453,340 blks 667.83K blks/sec 137.08 blks/call 100.00%	2,401,212 blks 855.50 blks/sec 0.18 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	148,591.43 ms 52.940 ms/sec 0.011 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.109)

Start: 2019-09-30T13:06:31.947164+00:00
End: 2019-09-30T13:57:29.023066+00:00
Period seconds: 3057.07591
Period age: 00:50:57.075902

Calls	Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
14,197,116 4.65K/sec 1.00/call 100.00%	5,176,462.98 ms 1693.273 ms/sec 0.365 ms/call 100.00%	46,548,326 15.23K/sec 3.28/call 100.00%	2,046,043,112 blks 669.29K blks/sec 144.12 blks/call 100.00%	2,536,458 blks 829.70 blks/sec 0.18 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	136,801.17 ms 44.749 ms/sec 0.010 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Conclusions

Recommendations

---

  Table of contents

K002 Workload Type ("The First Word" Analysis)

Observations

Data collected: 2019-09-30 13:57:28 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.101)

Start: 2019-09-30T13:06:11.519966+00:00
End: 2019-09-30T13:37:59.792427+00:00
Period seconds: 1908.27246
Period age: 00:31:48.272461

Error (calls): 0.00 (0.00%)
Error (total time): 0.00 (0.00%)

#	Workload type	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	select	25,969,558 13.61K/sec 1.00/call 87.43%	3,589,921.18 ms 1881.241 ms/sec 0.138 ms/call 58.72%	122,913,284 64.42K/sec 4.73/call 97.35%	1,171,560,253 blks 613.94K blks/sec 45.11 blks/call 95.58%	780,113 blks 408.81 blks/sec 0.03 blks/call 64.74%	91,804 blks 48.11 blks/sec 0.00 blks/call 4.13%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	355,598.20 ms 186.346 ms/sec 0.014 ms/call 63.09%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	update	752,177 394.17/sec 1.00/call 2.53%	2,035,787.87 ms 1066.822 ms/sec 2.707 ms/call 33.30%	560,477 293.71/sec 0.75/call 0.44%	30,957,908 blks 16.23K blks/sec 41.16 blks/call 2.53%	228,764 blks 119.88 blks/sec 0.30 blks/call 18.98%	1,517,812 blks 795.39 blks/sec 2.02 blks/call 68.23%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	88,938.11 ms 46.607 ms/sec 0.118 ms/call 15.78%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
3	select ... for [no key] update	44,475 23.31/sec 1.00/call 0.15%	223,189.44 ms 116.959 ms/sec 5.018 ms/call 3.65%	44,475 23.31/sec 1.00/call 0.04%	226,742 blks 118.82 blks/sec 5.10 blks/call 0.02%	1,003 blks 0.53 blks/sec 0.02 blks/call 0.08%	3,801 blks 1.99 blks/sec 0.09 blks/call 0.17%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	56.09 ms 0.029 ms/sec 0.001 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
4	insert	181,192 94.95/sec 1.00/call 0.61%	184,006.18 ms 96.426 ms/sec 1.016 ms/call 3.01%	326,720 171.21/sec 1.80/call 0.26%	13,591,664 blks 7.13K blks/sec 75.01 blks/call 1.11%	164,134 blks 86.01 blks/sec 0.91 blks/call 13.62%	583,809 blks 305.94 blks/sec 3.22 blks/call 26.24%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	99,757.49 ms 52.276 ms/sec 0.551 ms/call 17.70%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
5	set	2,745,747 1.44K/sec 1.00/call 9.24%	31,957.87 ms 16.747 ms/sec 0.012 ms/call 0.52%	0 0.00/sec 0.00/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
6	with	6,379 3.34/sec 1.00/call 0.02%	24,690.51 ms 12.939 ms/sec 3.871 ms/call 0.40%	2,365,872 1.24K/sec 370.88/call 1.87%	8,635,765 blks 4.53K blks/sec 1.36K blks/call 0.70%	2,303 blks 1.21 blks/sec 0.36 blks/call 0.19%	290 blks 0.15 blks/sec 0.05 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1,203.32 ms 0.631 ms/sec 0.189 ms/call 0.21%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
7	delete	2,303 1.21/sec 1.00/call 0.01%	23,952.17 ms 12.552 ms/sec 10.400 ms/call 0.39%	50,011 26.21/sec 21.72/call 0.04%	711,097 blks 372.64 blks/sec 308.77 blks/call 0.06%	28,672 blks 15.03 blks/sec 12.45 blks/call 2.38%	27,191 blks 14.25 blks/sec 11.81 blks/call 1.22%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	18,077.61 ms 9.473 ms/sec 7.850 ms/call 3.21%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica servers:

Replica (10.220.16.102)

Start: 2019-09-30T13:06:14.399331+00:00
End: 2019-09-30T13:39:13.29931+00:00
Period seconds: 1978.89998
Period age: 00:32:58.899979

#	Workload type	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	select	8,609,343 4.36K/sec 1.00/call 94.86%	3,000,363.83 ms 1516.178 ms/sec 0.349 ms/call 93.29%	29,205,642 14.76K/sec 3.39/call 97.25%	1,130,615,691 blks 571.34K blks/sec 131.32 blks/call 92.78%	1,543,966 blks 780.21 blks/sec 0.18 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	156,219.72 ms 78.943 ms/sec 0.018 ms/call 99.99%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	with	466,630 235.80/sec 1.00/call 5.14%	215,809.73 ms 109.055 ms/sec 0.462 ms/call 6.71%	826,255 417.53/sec 1.77/call 2.75%	87,956,722 blks 44.45K blks/sec 188.49 blks/call 7.22%	35 blks 0.02 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	20.34 ms 0.010 ms/sec 0.000 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.103)

Start: 2019-09-30T13:06:17.052557+00:00
End: 2019-09-30T13:40:33.301941+00:00
Period seconds: 2056.24938
Period age: 00:34:16.249384

#	Workload type	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	select	9,147,692 4.45K/sec 1.00/call 95.06%	3,275,101.45 ms 1592.755 ms/sec 0.358 ms/call 94.62%	30,291,523 14.74K/sec 3.31/call 97.31%	1,283,348,294 blks 624.13K blks/sec 140.29 blks/call 94.53%	1,543,429 blks 750.60 blks/sec 0.17 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	152,011.94 ms 73.927 ms/sec 0.017 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	with	475,655 231.32/sec 1.00/call 4.94%	186,343.22 ms 90.623 ms/sec 0.392 ms/call 5.38%	836,331 406.73/sec 1.76/call 2.69%	74,255,982 blks 36.12K blks/sec 156.11 blks/call 5.47%	3 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.04 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.104)

Start: 2019-09-30T13:06:20.434612+00:00
End: 2019-09-30T13:42:07.554666+00:00
Period seconds: 2147.12006
Period age: 00:35:47.120054

#	Workload type	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	select	9,465,311 4.41K/sec 1.00/call 94.94%	3,100,529.96 ms 1444.041 ms/sec 0.328 ms/call 95.16%	31,486,265 14.67K/sec 3.33/call 97.24%	1,333,396,833 blks 621.02K blks/sec 140.87 blks/call 95.09%	1,864,131 blks 868.20 blks/sec 0.20 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	192,583.52 ms 89.694 ms/sec 0.020 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	with	504,151 234.80/sec 1.00/call 5.06%	157,714.39 ms 73.454 ms/sec 0.313 ms/call 4.84%	894,315 416.52/sec 1.77/call 2.76%	68,864,077 blks 32.08K blks/sec 136.59 blks/call 4.91%	5 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.88 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.105)

Start: 2019-09-30T13:06:23.43192+00:00
End: 2019-09-30T13:44:07.798909+00:00
Period seconds: 2264.36699
Period age: 00:37:44.366989

#	Workload type	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	select	9,977,839 4.41K/sec 1.00/call 94.92%	3,119,842.05 ms 1377.799 ms/sec 0.313 ms/call 94.85%	33,435,290 14.77K/sec 3.35/call 97.25%	1,353,450,510 blks 597.72K blks/sec 135.65 blks/call 94.83%	1,795,611 blks 792.99 blks/sec 0.18 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	149,259.22 ms 65.917 ms/sec 0.015 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	with	534,543 236.07/sec 1.00/call 5.08%	169,462.59 ms 74.839 ms/sec 0.317 ms/call 5.15%	945,942 417.75/sec 1.77/call 2.75%	73,790,929 blks 32.59K blks/sec 138.04 blks/call 5.17%	19 blks 0.01 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1.12 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.106)

Start: 2019-09-30T13:06:25.816873+00:00
End: 2019-09-30T13:46:29.215889+00:00
Period seconds: 2403.39902
Period age: 00:40:03.399016

#	Workload type	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	select	10,749,555 4.48K/sec 1.00/call 95.00%	3,116,959.07 ms 1296.896 ms/sec 0.290 ms/call 94.92%	35,485,183 14.77K/sec 3.30/call 97.28%	1,288,544,409 blks 536.14K blks/sec 119.87 blks/call 95.33%	2,087,412 blks 868.52 blks/sec 0.19 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	129,671.95 ms 53.954 ms/sec 0.012 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	with	565,364 235.24/sec 1.00/call 5.00%	166,979.99 ms 69.477 ms/sec 0.295 ms/call 5.08%	991,845 412.68/sec 1.75/call 2.72%	63,117,903 blks 26.27K blks/sec 111.64 blks/call 4.67%	20 blks 0.01 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.39 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.107)

Start: 2019-09-30T13:06:28.521327+00:00
End: 2019-09-30T13:49:34.408897+00:00
Period seconds: 2585.88757
Period age: 00:43:05.88757

#	Workload type	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	select	11,450,260 4.43K/sec 1.00/call 94.85%	3,761,352.05 ms 1454.569 ms/sec 0.328 ms/call 95.10%	38,183,938 14.77K/sec 3.33/call 97.18%	1,545,746,832 blks 597.77K blks/sec 135.00 blks/call 94.77%	1,992,183 blks 770.41 blks/sec 0.17 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	123,821.57 ms 47.884 ms/sec 0.011 ms/call 99.99%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	with	621,546 240.36/sec 1.00/call 5.15%	193,972.72 ms 75.012 ms/sec 0.312 ms/call 4.90%	1,107,034 428.11/sec 1.78/call 2.82%	85,315,460 blks 33.00K blks/sec 137.26 blks/call 5.23%	25 blks 0.01 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	7.20 ms 0.003 ms/sec 0.000 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.108)

Start: 2019-09-30T13:06:29.835321+00:00
End: 2019-09-30T13:53:16.626097+00:00
Period seconds: 2806.79078
Period age: 00:46:46.790776

#	Workload type	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	select	13,003,918 4.64K/sec 1.00/call 95.10%	4,548,868.54 ms 1620.665 ms/sec 0.350 ms/call 94.72%	39,812,402 14.19K/sec 3.06/call 97.10%	1,782,251,750 blks 634.98K blks/sec 137.05 blks/call 95.08%	2,401,180 blks 855.49 blks/sec 0.18 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	148,590.90 ms 52.940 ms/sec 0.011 ms/call 100.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	with	669,767 238.62/sec 1.00/call 4.90%	253,779.78 ms 90.416 ms/sec 0.379 ms/call 5.28%	1,187,928 423.23/sec 1.77/call 2.90%	92,201,590 blks 32.85K blks/sec 137.66 blks/call 4.92%	32 blks 0.01 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.53 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.109)

Start: 2019-09-30T13:06:31.947164+00:00
End: 2019-09-30T13:57:29.023066+00:00
Period seconds: 3057.07591
Period age: 00:50:57.075902

#	Workload type	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	select	13,475,550 4.41K/sec 1.00/call 94.92%	4,923,472.70 ms 1610.517 ms/sec 0.365 ms/call 95.11%	45,276,021 14.82K/sec 3.36/call 97.27%	1,946,765,513 blks 636.81K blks/sec 144.47 blks/call 95.15%	2,536,417 blks 829.69 blks/sec 0.19 blks/call 100.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	136,791.15 ms 44.746 ms/sec 0.010 ms/call 99.99%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	with	721,566 236.03/sec 1.00/call 5.08%	252,990.29 ms 82.756 ms/sec 0.351 ms/call 4.89%	1,272,305 416.18/sec 1.76/call 2.73%	99,277,599 blks 32.48K blks/sec 137.59 blks/call 4.85%	41 blks 0.01 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	10.02 ms 0.003 ms/sec 0.000 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Conclusions

Recommendations

---

  Table of contents

K003 Top-20 Queries by total_time

Observations

Data collected: 2019-09-30 13:57:28 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.101)

Start: 2019-09-30T13:06:11.519966+00:00
End: 2019-09-30T13:37:59.792427+00:00
Period seconds: 1908.27246
Period age: 00:31:48.272461

Error (calls): 0.00 (0.00%)
Error (total time): 0.00 (0.00%)

The list is limited to 20 items.

#	Query	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	SELECT COUNT(*) FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) Full query	220 0.12/sec 1.00/call 0.00%	737,993.22 ms 386.734 ms/sec 3354.515 ms/call 12.07%	220 0.12/sec 1.00/call 0.00%	80,241,577 blks 42.05K blks/sec 364.74K blks/call 6.55%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	3,589 blks 1.88 blks/sec 16.31 blks/call 0.16%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	UPDATE "project_statistics" SET  build_artifacts_size = COALESCE( build_artifacts_size, ?) + (?),  storage_size = COALESCE(storage_size,  ?) + (?) WHERE "project_statistics"."project_id"  = ? Full query	40,624 21.29/sec 1.00/call 0.14%	729,680.80 ms 382.378 ms/sec 17.962 ms/call 11.94%	40,624 21.29/sec 1.00/call 0.03%	1,415,705 blks 741.88 blks/sec 34.85 blks/call 0.12%	456 blks 0.24 blks/sec 0.01 blks/call 0.04%	34,647 blks 18.16 blks/sec 0.85 blks/call 1.56%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	20.96 ms 0.011 ms/sec 0.001 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
3	UPDATE "ci_builds" SET "status"  = ?, "updated_at" = ?, "lock_version"  = ? WHERE "ci_builds"."type" IN  (?) AND ("ci_builds"."lock_version"  = ? OR "ci_builds"."lock_version"  IS NULL) AND "ci_builds"."id" =  ? Full query	166,790 87.40/sec 1.00/call 0.56%	467,495.82 ms 244.984 ms/sec 2.803 ms/call 7.65%	8,410 4.41/sec 0.05/call 0.01%	1,757,493 blks 920.99 blks/sec 10.54 blks/call 0.14%	3,468 blks 1.82 blks/sec 0.02 blks/call 0.29%	13,628 blks 7.14 blks/sec 0.08 blks/call 0.61%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1,221.66 ms 0.640 ms/sec 0.007 ms/call 0.22%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
4	SELECT  "projects".* FROM "projects"  INNER JOIN project_mirror_data  import_state ON import_state.project_id  = projects.id WHERE "projects"."mirror"  = TRUE AND "import_state"."status"  NOT IN ('scheduled', 'started')  AND (import_state.next_execution_timestamp  <= '2019-09-09 20:35:08.814893')  AND (import_state.retry_count <=  14) ORDER BY import_state.next_execution_timestamp  LIMIT 500 Full query	2,295 1.20/sec 1.00/call 0.01%	460,944.67 ms 241.551 ms/sec 200.847 ms/call 7.54%	96,686 50.67/sec 42.13/call 0.08%	262,104,122 blks 137.36K blks/sec 114.21K blks/call 21.38%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	886 blks 0.46 blks/sec 0.39 blks/call 0.04%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
5	SELECT COUNT(*) FROM "projects"  WHERE "projects"."namespace_id"  IN (WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" INNER JOIN "members"  ON "namespaces"."id" = "members"."source_id"  WHERE "members"."type" IN (?) AND  "members"."source_type" = ? AND  "namespaces"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND (members.access_level  >= ?) UNION SELECT "namespaces".*  FROM "namespaces", "base_and_descendants"  WHERE "namespaces"."type" IN (?)  AND "namespaces"."parent_id" =  "base_and_descendants"."id") SELECT  DISTINCT "namespaces"."id" FROM  "base_and_descendants" AS "namespaces"  WHERE "namespaces"."id" IN (WITH  RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces... Full query	25 0.01/sec 1.00/call 0.00%	241,547.16 ms 126.579 ms/sec 9661.887 ms/call 3.95%	25 0.01/sec 1.00/call 0.00%	198,174,152 blks 103.86K blks/sec 7.93M blks/call 16.17%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	12,190 blks 6.39 blks/sec 487.60 blks/call 0.55%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
6	SELECT  "internal_ids".* FROM "internal_ids"  WHERE "internal_ids"."id" = ? LIMIT  ? FOR UPDATE Full query	44,475 23.31/sec 1.00/call 0.15%	223,189.44 ms 116.959 ms/sec 5.018 ms/call 3.65%	44,475 23.31/sec 1.00/call 0.04%	226,742 blks 118.82 blks/sec 5.10 blks/call 0.02%	1,003 blks 0.53 blks/sec 0.02 blks/call 0.08%	3,801 blks 1.99 blks/sec 0.09 blks/call 0.17%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	56.09 ms 0.029 ms/sec 0.001 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
7	UPDATE "project_mirror_data" SET  "status" = ?, "last_update_scheduled_at"  = ? WHERE "project_mirror_data"."id"  = ? Full query	91,512 47.96/sec 1.00/call 0.31%	220,568.75 ms 115.586 ms/sec 2.410 ms/call 3.61%	91,512 47.96/sec 1.00/call 0.07%	3,092,237 blks 1.63K blks/sec 33.79 blks/call 0.25%	7 blks 0.00 blks/sec 0.00 blks/call 0.00%	221,867 blks 116.27 blks/sec 2.42 blks/call 9.97%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
8	SELECT "web_hooks".* FROM "web_hooks"  WHERE "web_hooks"."type" IN (?)  AND "web_hooks"."group_id" = ?  AND "web_hooks"."job_events" =  ? Full query	43,998 23.06/sec 1.00/call 0.15%	194,149.11 ms 101.741 ms/sec 4.413 ms/call 3.18%	761 0.40/sec 0.02/call 0.00%	117,518,658 blks 61.59K blks/sec 2.68K blks/call 9.59%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
9	SELECT (CASE           WHEN (SELECT  count(*) FROM "ci_builds" WHERE  "ci_builds"."stage_id" = ? AND  ("ci_builds"."retried" = ? OR "ci_builds"."retried"  IS NULL) AND (allow_failure = ?  OR status IN (?,?,?,?,?,?,?)))=( SELECT count(*) FROM "ci_builds"  WHERE "ci_builds"."stage_id" =  ? AND ("ci_builds"."retried" =  ? OR "ci_builds"."retried" IS NULL)  AND (allow_failure = ? OR status  IN (?,?,?,?,?,?,?)) AND ("ci_builds"."status"  IN (?))) AND (SELECT count(*) >  ? FROM "ci_builds" WHERE "ci_builds"."stage_id"  = ? AND ("ci_builds"."retried"  = ? OR "ci_builds"."retried" IS  NULL) AND "ci_builds"."allow_failure"  = ? AND "ci_builds"."status" IN  (?, ?)) THEN ?           WHEN ( SELECT count(*) FROM "ci_builds"  WHERE "ci_builds"."stage_id" =  ? AND ("ci_builds"."retried" =  ? OR "ci_builds"."retried" IS NULL)  AND (allow_failure = ? OR status  IN (?,?,?,?,?,?,?)))=(SELECT count( *) FROM "ci_builds" WHERE "ci_builds"."stage_id"  = ? AND ("ci_builds"."retried"  = ? OR "ci_builds"."retried" IS  NULL) AND (allow_fa... Full query	96,459 50.55/sec 1.00/call 0.32%	193,585.51 ms 101.445 ms/sec 2.007 ms/call 3.17%	1,008,764 528.63/sec 10.46/call 0.80%	80,329,197 blks 42.10K blks/sec 832.78 blks/call 6.55%	137 blks 0.07 blks/sec 0.00 blks/call 0.01%	662 blks 0.35 blks/sec 0.01 blks/call 0.03%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	85.53 ms 0.045 ms/sec 0.001 ms/call 0.02%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
10	UPDATE "project_mirror_data" SET  "status" = ?, "last_update_started_at"  = ? WHERE "project_mirror_data"."id"  = ? Full query	90,581 47.47/sec 1.00/call 0.30%	186,163.95 ms 97.556 ms/sec 2.055 ms/call 3.05%	90,581 47.47/sec 1.00/call 0.07%	3,422,785 blks 1.80K blks/sec 37.79 blks/call 0.28%	5 blks 0.00 blks/sec 0.00 blks/call 0.00%	1,755 blks 0.92 blks/sec 0.02 blks/call 0.08%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
11	SELECT  "users".* FROM "users" WHERE  "users"."unlock_token" = '55800f789c61ad88d5b37088e4a2c9708de838f236be56c31edef601baedbfec'  ORDER BY "users"."id" ASC LIMIT  1 Full query	19 0.01/sec 1.00/call 0.00%	131,771.74 ms 69.053 ms/sec 6935.355 ms/call 2.16%	0 0.00/sec 0.00/call 0.00%	80,430,563 blks 42.15K blks/sec 4.24M blks/call 6.56%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	12,269 blks 6.43 blks/sec 645.74 blks/call 0.55%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
12	SELECT  ? AS one FROM (SELECT "projects".*  FROM "projects" WHERE "projects"."namespace_id"  IN (WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."id"  IN (SELECT "elasticsearch_indexed_namespaces"."namespace_id"  FROM "elasticsearch_indexed_namespaces") UNION SELECT  "namespaces".* FROM "namespaces",  "base_and_descendants" WHERE "namespaces"."parent_id"  = "base_and_descendants"."id")  SELECT "id" FROM "base_and_descendants"  AS "namespaces") UNION SELECT "projects".*  FROM "projects" WHERE "projects"."id"  IN (SELECT "elasticsearch_indexed_projects"."project_id"  FROM "elasticsearch_indexed_projects"))  projects WHERE "projects"."id"  = ? LIMIT ? Full query	166,649 87.33/sec 1.00/call 0.56%	123,999.00 ms 64.980 ms/sec 0.744 ms/call 2.03%	2,465 1.29/sec 0.01/call 0.00%	44,709,266 blks 23.43K blks/sec 268.28 blks/call 3.65%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1,994 blks 1.04 blks/sec 0.01 blks/call 0.09%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
13	UPDATE "project_mirror_data" SET  "jid" = ? WHERE "project_mirror_data"."id"  = ? Full query	91,564 47.98/sec 1.00/call 0.31%	86,137.18 ms 45.139 ms/sec 0.941 ms/call 1.41%	91,564 47.98/sec 1.00/call 0.07%	3,331,983 blks 1.75K blks/sec 36.39 blks/call 0.27%	6 blks 0.00 blks/sec 0.00 blks/call 0.00%	58,164 blks 30.48 blks/sec 0.64 blks/call 2.61%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
14	UPDATE "notes" SET "note_html" =  ?, "cached_markdown_version" =  ? WHERE "notes"."id" = ? Full query	29,522 15.47/sec 1.00/call 0.10%	82,923.71 ms 43.455 ms/sec 2.809 ms/call 1.36%	29,522 15.47/sec 1.00/call 0.02%	2,139,723 blks 1.13K blks/sec 72.48 blks/call 0.17%	111,439 blks 58.40 blks/sec 3.77 blks/call 9.25%	262,922 blks 137.78 blks/sec 8.91 blks/call 11.82%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	64,721.70 ms 33.916 ms/sec 2.192 ms/call 11.48%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
15	SELECT (CASE           WHEN (SELECT  count(*) FROM "ci_builds" WHERE  "ci_builds"."commit_id" = ? AND  ("ci_builds"."retried" = ? OR "ci_builds"."retried"  IS NULL) AND (allow_failure = ?  OR status IN (?,?,?,?,?,?,?)))=( SELECT count(*) FROM "ci_builds"  WHERE "ci_builds"."commit_id" =  ? AND ("ci_builds"."retried" =  ? OR "ci_builds"."retried" IS NULL)  AND (allow_failure = ? OR status  IN (?,?,?,?,?,?,?)) AND ("ci_builds"."status"  IN (?))) AND (SELECT count(*) >  ? FROM "ci_builds" WHERE "ci_builds"."commit_id"  = ? AND ("ci_builds"."retried"  = ? OR "ci_builds"."retried" IS  NULL) AND "ci_builds"."allow_failure"  = ? AND "ci_builds"."status" IN  (?, ?)) THEN ?           WHEN ( SELECT count(*) FROM "ci_builds"  WHERE "ci_builds"."commit_id" =  ? AND ("ci_builds"."retried" =  ? OR "ci_builds"."retried" IS NULL)  AND (allow_failure = ? OR status  IN (?,?,?,?,?,?,?)))=(SELECT count( *) FROM "ci_builds" WHERE "ci_builds"."commit_id"  = ? AND ("ci_builds"."retried"  = ? OR "ci_builds"."retried" IS  NULL) AND (all... Full query	116,654 61.13/sec 1.00/call 0.39%	66,336.49 ms 34.763 ms/sec 0.569 ms/call 1.09%	2,323,325 1.22K/sec 19.92/call 1.84%	45,849,688 blks 24.03K blks/sec 393.04 blks/call 3.74%	278 blks 0.15 blks/sec 0.00 blks/call 0.02%	545 blks 0.29 blks/sec 0.00 blks/call 0.02%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	173.39 ms 0.091 ms/sec 0.001 ms/call 0.03%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
16	SELECT  "projects".* FROM "projects"  WHERE "projects"."id" = ? LIMIT  ? Full query	1,207,668 632.86/sec 1.00/call 4.07%	58,080.94 ms 30.436 ms/sec 0.048 ms/call 0.95%	1,207,251 632.64/sec 1.00/call 0.96%	4,945,834 blks 2.60K blks/sec 4.10 blks/call 0.40%	5 blks 0.00 blks/sec 0.00 blks/call 0.00%	38 blks 0.02 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.73 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
17	SELECT  "clusters".* FROM "clusters"  WHERE "clusters"."cluster_type"  = ? AND "clusters"."enabled" =  ? AND "clusters"."environment_scope"  = ? ORDER BY "clusters"."id" ASC  LIMIT ? Full query	6,045 3.17/sec 1.00/call 0.02%	56,294.52 ms 29.500 ms/sec 9.313 ms/call 0.92%	0 0.00/sec 0.00/call 0.00%	22,438,136 blks 11.76K blks/sec 3.72K blks/call 1.83%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	2 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
18	SELECT  "routes".* FROM "routes"  WHERE "routes"."source_id" = ?  AND "routes"."source_type" = ?  LIMIT ? Full query	1,273,223 667.21/sec 1.00/call 4.29%	54,833.81 ms 28.735 ms/sec 0.043 ms/call 0.90%	1,273,223 667.21/sec 1.00/call 1.01%	6,368,982 blks 3.34K blks/sec 5.00 blks/call 0.52%	1,706 blks 0.89 blks/sec 0.00 blks/call 0.14%	193 blks 0.10 blks/sec 0.00 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	627.50 ms 0.329 ms/sec 0.000 ms/call 0.11%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
19	INSERT INTO "notes" ("note", "noteable_type",  "author_id", "created_at", "updated_at",  "project_id", "noteable_id", "system",  "discussion_id", "note_html",  "cached_markdown_version") VALUES  (?, ?, ?, ?, ?, ?, ?, ?, ?, ?,  ?) RETURNING "id" Full query	19,450 10.19/sec 1.00/call 0.07%	51,516.89 ms 26.997 ms/sec 2.649 ms/call 0.84%	19,450 10.19/sec 1.00/call 0.02%	1,706,845 blks 894.45 blks/sec 87.76 blks/call 0.14%	57,976 blks 30.38 blks/sec 2.98 blks/call 4.81%	163,091 blks 85.47 blks/sec 8.39 blks/call 7.33%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	35,859.01 ms 18.791 ms/sec 1.844 ms/call 6.36%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
20	SELECT  ? AS one FROM (WITH RECURSIVE  "base_and_ancestors" AS (SELECT  "namespaces".* FROM (SELECT "namespaces".*  FROM "namespaces" INNER JOIN "members"  ON "namespaces"."id" = "members"."source_id"  WHERE "members"."type" IN (?) AND  "members"."source_type" = ? AND  "namespaces"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL UNION SELECT namespaces.*  FROM "projects" INNER JOIN "namespaces"  ON "namespaces"."id" = "projects"."namespace_id"  INNER JOIN "project_authorizations"  ON "projects"."id" = "project_authorizations"."project_id"  WHERE "project_authorizations"."user_id"  = ?) namespaces WHERE "namespaces"."type"  IN (?) UNION SELECT "namespaces".*  FROM "namespaces", "base_and_ancestors"  WHERE "namespaces"."type" IN (?)  AND "namespaces"."id" = "base_and_ancestors"."parent_id"),  "base_and_descendants" AS (SELECT  "namespaces".* FROM "namespaces"  INNER JOIN "members" ON "namespaces"."id"  = "members"."source_id" WHERE "members"."type"  IN (?) AND "members"."source... Full query	46 0.02/sec 1.00/call 0.00%	50,636.17 ms 26.535 ms/sec 1100.786 ms/call 0.83%	46 0.02/sec 1.00/call 0.00%	11,066,322 blks 5.80K blks/sec 240.58K blks/call 0.90%	21 blks 0.01 blks/sec 0.46 blks/call 0.00%	430 blks 0.23 blks/sec 9.35 blks/call 0.02%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	14.44 ms 0.008 ms/sec 0.314 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica servers:

Replica (10.220.16.102)

Start: 2019-09-30T13:06:14.399331+00:00
End: 2019-09-30T13:39:13.29931+00:00
Period seconds: 1978.89998
Period age: 00:32:58.899979

Top 50 rows

#	Query	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	SELECT COUNT(*) FROM "projects"  WHERE "projects"."namespace_id"  IN (WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" INNER JOIN "members"  ON "namespaces"."id" = "members"."source_id"  WHERE "members"."type" IN (?) AND  "members"."source_type" = ? AND  "namespaces"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND (members.access_level  >= ?) UNION SELECT "namespaces".*  FROM "namespaces", "base_and_descendants"  WHERE "namespaces"."type" IN (?)  AND "namespaces"."parent_id" =  "base_and_descendants"."id") SELECT  DISTINCT "namespaces"."id" FROM  "base_and_descendants" AS "namespaces"  WHERE "namespaces"."id" IN (WITH  RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces... Full query	52 0.03/sec 1.00/call 0.00%	451,082.95 ms 227.946 ms/sec 8674.672 ms/call 14.03%	52 0.03/sec 1.00/call 0.00%	394,807,081 blks 199.51K blks/sec 7.60M blks/call 32.40%	33 blks 0.02 blks/sec 0.63 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1.81 ms 0.001 ms/sec 0.035 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."pending_delete"  = ? ORDER BY "projects"."created_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	34 0.02/sec 1.00/call 0.00%	236,908.33 ms 119.717 ms/sec 6967.892 ms/call 7.37%	1,674 0.85/sec 49.24/call 0.01%	12,401,998 blks 6.27K blks/sec 364.77K blks/call 1.02%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
3	WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces.id))) UNION SELECT  "namespaces".* FROM "namespaces",  "base_and_descendants" WHERE "namespaces"."type"  IN (?) AND "namespaces"."parent_id"  = "base_and_descendants"."id")  SELECT  "namespaces".* FROM "base_and_descendants"  AS "namespaces" WHERE "namespaces"."id"  = ? LIMIT ? Full query	31 0.02/sec 1.00/call 0.00%	187,810.70 ms 94.907 ms/sec 6058.410 ms/call 5.84%	1 0.00/sec 0.03/call 0.00%	83,445,623 blks 42.17K blks/sec 2.70M blks/call 6.85%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
4	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	42 0.02/sec 1.00/call 0.00%	163,579.17 ms 82.662 ms/sec 3894.742 ms/call 5.09%	4,152 2.10/sec 98.86/call 0.01%	57,287,866 blks 28.95K blks/sec 1.37M blks/call 4.70%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
5	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	559 0.28/sec 1.00/call 0.01%	162,246.43 ms 81.988 ms/sec 290.244 ms/call 5.04%	6,212 3.14/sec 11.11/call 0.02%	100,758,779 blks 50.92K blks/sec 180.25K blks/call 8.27%	113 blks 0.06 blks/sec 0.20 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	4.42 ms 0.002 ms/sec 0.008 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
6	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	546 0.28/sec 1.00/call 0.01%	160,752.82 ms 81.233 ms/sec 294.419 ms/call 5.00%	5,371 2.71/sec 9.84/call 0.02%	98,070,050 blks 49.56K blks/sec 179.62K blks/call 8.05%	139 blks 0.07 blks/sec 0.25 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	27.80 ms 0.014 ms/sec 0.051 ms/call 0.02%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
7	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	10 0.01/sec 1.00/call 0.00%	129,722.60 ms 65.553 ms/sec 12972.260 ms/call 4.03%	10 0.01/sec 1.00/call 0.00%	49,930,977 blks 25.24K blks/sec 5.00M blks/call 4.10%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
8	SELECT COUNT(*) FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) Full query	32 0.02/sec 1.00/call 0.00%	121,173.32 ms 61.233 ms/sec 3786.666 ms/call 3.77%	32 0.02/sec 1.00/call 0.00%	11,671,502 blks 5.90K blks/sec 364.74K blks/call 0.96%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
9	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	9 0.00/sec 1.00/call 0.00%	115,950.76 ms 58.594 ms/sec 12883.418 ms/call 3.61%	490 0.25/sec 54.44/call 0.00%	44,961,851 blks 22.73K blks/sec 5.00M blks/call 3.69%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
10	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	9 0.00/sec 1.00/call 0.00%	114,797.43 ms 58.011 ms/sec 12755.269 ms/call 3.57%	19 0.01/sec 2.11/call 0.00%	44,963,940 blks 22.73K blks/sec 5.00M blks/call 3.69%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
11	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  projects.id FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	24 0.01/sec 1.00/call 0.00%	102,680.51 ms 51.888 ms/sec 4278.355 ms/call 3.19%	55 0.03/sec 2.29/call 0.00%	41,830,019 blks 21.14K blks/sec 1.75M blks/call 3.43%	33 blks 0.02 blks/sec 1.38 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.32 ms 0.000 ms/sec 0.013 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
12	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	23 0.01/sec 1.00/call 0.00%	89,190.35 ms 45.071 ms/sec 3877.841 ms/call 2.77%	33 0.02/sec 1.43/call 0.00%	36,851,446 blks 18.63K blks/sec 1.61M blks/call 3.02%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
13	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	23 0.01/sec 1.00/call 0.00%	64,618.58 ms 32.654 ms/sec 2809.504 ms/call 2.01%	23 0.01/sec 1.00/call 0.00%	25,108,701 blks 12.69K blks/sec 1.10M blks/call 2.06%	5 blks 0.00 blks/sec 0.22 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.85 ms 0.000 ms/sec 0.037 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
14	SELECT  "snippets".* FROM (SELECT  "snippets".* FROM "snippets" WHERE  (snippets.visibility_level IN ( ?,?,?) OR snippets.author_id =  ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? ORDER BY created_at DESC... Full query	13 0.01/sec 1.00/call 0.00%	61,682.81 ms 31.170 ms/sec 4744.832 ms/call 1.92%	273 0.14/sec 21.00/call 0.00%	4,955,461 blks 2.51K blks/sec 381.19K blks/call 0.41%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
15	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM (SELECT "snippets".*  FROM "snippets" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? LIMIT ... Full query	13 0.01/sec 1.00/call 0.00%	60,630.47 ms 30.638 ms/sec 4663.882 ms/call 1.89%	13 0.01/sec 1.00/call 0.00%	4,951,153 blks 2.51K blks/sec 380.86K blks/call 0.41%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
16	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  "issues"."confidential" = ? AND  (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_level"  = ? AND EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)))) AND ("issues"."title"  ILIKE ? OR "issues"."description"  ILIKE ?) LIMIT ?) subquery_for_count Full query	12 0.01/sec 1.00/call 0.00%	52,907.81 ms 26.736 ms/sec 4408.984 ms/call 1.65%	12 0.01/sec 1.00/call 0.00%	983,484 blks 496.99 blks/sec 81.96K blks/call 0.08%	37,372 blks 18.89 blks/sec 3.12K blks/call 2.42%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	965.74 ms 0.488 ms/sec 80.478 ms/call 0.62%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
17	SELECT COUNT(*) FROM "deploy_keys_projects"  WHERE "deploy_keys_projects"."deploy_key_id"  = ? Full query	997 0.50/sec 1.00/call 0.01%	47,081.55 ms 23.792 ms/sec 47.223 ms/call 1.46%	997 0.50/sec 1.00/call 0.00%	3,647,026 blks 1.85K blks/sec 3.66K blks/call 0.30%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
18	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  (       issues.confidential IS  NOT TRUE       OR (issues.confidential  = ?         AND (issues.author_id  = ?           OR EXISTS (SELECT  ? FROM issue_assignees WHERE user_id  = ? AND issue_id = issues.id)          OR EXISTS (SELECT ? FROM  "project_authorizations" WHERE  "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = issues.project_id) AND (project_authorizations.access_level  >= ?))))) AND (EXISTS (SELECT ?  FROM "project_authorizations" WHERE  "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_l... Full query	9 0.00/sec 1.00/call 0.00%	46,152.25 ms 23.322 ms/sec 5128.027 ms/call 1.44%	9 0.00/sec 1.00/call 0.00%	867,386 blks 438.32 blks/sec 96.38K blks/call 0.07%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
19	SELECT  "projects"."id" AS t0_r0,  "projects"."name" AS t0_r1, "projects"."path"  AS t0_r2, "projects"."description"  AS t0_r3, "projects"."created_at"  AS t0_r4, "projects"."updated_at"  AS t0_r5, "projects"."creator_id"  AS t0_r6, "projects"."namespace_id"  AS t0_r7, "projects"."last_activity_at"  AS t0_r8, "projects"."import_url"  AS t0_r9, "projects"."visibility_level"  AS t0_r10, "projects"."archived"  AS t0_r11, "projects"."merge_requests_template"  AS t0_r12, "projects"."star_count"  AS t0_r13, "projects"."merge_requests_rebase_enabled"  AS t0_r14, "projects"."import_type"  AS t0_r15, "projects"."import_source"  AS t0_r16, "projects"."avatar"  AS t0_r17, "projects"."approvals_before_merge"  AS t0_r18, "projects"."reset_approvals_on_push"  AS t0_r19, "projects"."merge_requests_ff_only_enabled"  AS t0_r20, "projects"."issues_template"  AS t0_r21, "projects"."mirror"  AS t0_r22, "projects"."mirror_user_id"  AS t0_r23, "projects"."shared_runners_enabled"  AS t0_r24, "projects"."runners_token"  AS t0_r25, "... Full query	677,646 342.44/sec 1.00/call 7.47%	44,137.69 ms 22.304 ms/sec 0.065 ms/call 1.37%	668,231 337.68/sec 0.99/call 2.23%	6,066,227 blks 3.07K blks/sec 8.95 blks/call 0.50%	8,374 blks 4.23 blks/sec 0.01 blks/call 0.54%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	685.72 ms 0.347 ms/sec 0.001 ms/call 0.44%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
20	SELECT  "deployments".* FROM "deployments"  WHERE "deployments"."environment_id"  = ? AND "deployments"."status"  = ? ORDER BY deployments.id DESC  LIMIT ? Full query	9,928 5.02/sec 1.00/call 0.11%	36,081.84 ms 18.233 ms/sec 3.634 ms/call 1.12%	8,186 4.14/sec 0.82/call 0.03%	12,223,256 blks 6.18K blks/sec 1.24K blks/call 1.00%	3,457 blks 1.75 blks/sec 0.35 blks/call 0.22%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	238.93 ms 0.121 ms/sec 0.024 ms/call 0.15%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.103)

Start: 2019-09-30T13:06:17.052557+00:00
End: 2019-09-30T13:40:33.301941+00:00
Period seconds: 2056.24938
Period age: 00:34:16.249384

Top 50 rows

#	Query	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	SELECT COUNT(*) FROM "projects"  WHERE "projects"."namespace_id"  IN (WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" INNER JOIN "members"  ON "namespaces"."id" = "members"."source_id"  WHERE "members"."type" IN (?) AND  "members"."source_type" = ? AND  "namespaces"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND (members.access_level  >= ?) UNION SELECT "namespaces".*  FROM "namespaces", "base_and_descendants"  WHERE "namespaces"."type" IN (?)  AND "namespaces"."parent_id" =  "base_and_descendants"."id") SELECT  DISTINCT "namespaces"."id" FROM  "base_and_descendants" AS "namespaces"  WHERE "namespaces"."id" IN (WITH  RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces... Full query	62 0.03/sec 1.00/call 0.00%	561,372.89 ms 273.008 ms/sec 9054.402 ms/call 16.22%	62 0.03/sec 1.00/call 0.00%	501,464,515 blks 243.88K blks/sec 8.09M blks/call 36.94%	125 blks 0.06 blks/sec 2.02 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	4.91 ms 0.002 ms/sec 0.079 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."pending_delete"  = ? ORDER BY "projects"."created_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	38 0.02/sec 1.00/call 0.00%	267,439.82 ms 130.062 ms/sec 7037.890 ms/call 7.73%	1,278 0.62/sec 33.63/call 0.00%	13,862,554 blks 6.75K blks/sec 364.81K blks/call 1.02%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
3	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT 1 FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = 3611581 AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (10,20)) AND "projects"."visibility_level"  = 20 AND "projects"."archived"  = FALSE AND "projects"."pending_delete"  = FALSE ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  101 OFFSET 100 Full query	37 0.02/sec 1.00/call 0.00%	179,191.59 ms 87.145 ms/sec 4843.016 ms/call 5.18%	3,620 1.76/sec 97.84/call 0.01%	66,893,207 blks 32.54K blks/sec 1.81M blks/call 4.93%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
4	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	567 0.28/sec 1.00/call 0.01%	166,643.33 ms 81.042 ms/sec 293.904 ms/call 4.81%	5,614 2.73/sec 9.90/call 0.02%	102,389,171 blks 49.80K blks/sec 180.59K blks/call 7.54%	126 blks 0.06 blks/sec 0.22 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	13.86 ms 0.007 ms/sec 0.024 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
5	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	551 0.27/sec 1.00/call 0.01%	158,427.72 ms 77.047 ms/sec 287.528 ms/call 4.58%	5,786 2.81/sec 10.50/call 0.02%	99,071,170 blks 48.19K blks/sec 179.81K blks/call 7.30%	128 blks 0.06 blks/sec 0.23 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	9.23 ms 0.004 ms/sec 0.017 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
6	WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces.id))) UNION SELECT  "namespaces".* FROM "namespaces",  "base_and_descendants" WHERE "namespaces"."type"  IN (?) AND "namespaces"."parent_id"  = "base_and_descendants"."id")  SELECT  "namespaces".* FROM "base_and_descendants"  AS "namespaces" WHERE "namespaces"."id"  = ? LIMIT ? Full query	26 0.01/sec 1.00/call 0.00%	157,614.46 ms 76.651 ms/sec 6062.095 ms/call 4.55%	1 0.00/sec 0.04/call 0.00%	69,658,168 blks 33.88K blks/sec 2.68M blks/call 5.13%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
7	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  projects.id FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	24 0.01/sec 1.00/call 0.00%	151,120.29 ms 73.493 ms/sec 6296.679 ms/call 4.37%	60 0.03/sec 2.50/call 0.00%	60,910,107 blks 29.63K blks/sec 2.54M blks/call 4.49%	22 blks 0.01 blks/sec 0.92 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.20 ms 0.000 ms/sec 0.008 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
8	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	23 0.01/sec 1.00/call 0.00%	138,540.35 ms 67.375 ms/sec 6023.493 ms/call 4.00%	58 0.03/sec 2.52/call 0.00%	55,903,581 blks 27.19K blks/sec 2.44M blks/call 4.12%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
9	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	24 0.01/sec 1.00/call 0.00%	127,105.34 ms 61.814 ms/sec 5296.056 ms/call 3.67%	24 0.01/sec 1.00/call 0.00%	50,142,813 blks 24.39K blks/sec 2.09M blks/call 3.69%	2 blks 0.00 blks/sec 0.08 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.02 ms 0.000 ms/sec 0.001 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
10	SELECT COUNT(*) FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) Full query	28 0.01/sec 1.00/call 0.00%	103,456.12 ms 50.313 ms/sec 3694.861 ms/call 2.99%	28 0.01/sec 1.00/call 0.00%	10,212,574 blks 4.97K blks/sec 364.74K blks/call 0.75%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
11	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	7 0.00/sec 1.00/call 0.00%	90,003.33 ms 43.771 ms/sec 12857.619 ms/call 2.60%	274 0.13/sec 39.14/call 0.00%	34,897,574 blks 16.98K blks/sec 4.99M blks/call 2.57%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
12	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	7 0.00/sec 1.00/call 0.00%	89,049.36 ms 43.307 ms/sec 12721.338 ms/call 2.57%	11 0.01/sec 1.57/call 0.00%	34,899,872 blks 16.98K blks/sec 4.99M blks/call 2.57%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
13	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	7 0.00/sec 1.00/call 0.00%	88,862.12 ms 43.216 ms/sec 12694.589 ms/call 2.57%	7 0.00/sec 1.00/call 0.00%	34,895,771 blks 16.98K blks/sec 4.99M blks/call 2.57%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
14	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  "issues"."confidential" = ? AND  (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_level"  = ? AND EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)))) AND ("issues"."title"  ILIKE ? OR "issues"."description"  ILIKE ?) LIMIT ?) subquery_for_count Full query	15 0.01/sec 1.00/call 0.00%	78,303.15 ms 38.081 ms/sec 5220.210 ms/call 2.26%	15 0.01/sec 1.00/call 0.00%	1,325,925 blks 644.83 blks/sec 88.40K blks/call 0.10%	61,529 blks 29.92 blks/sec 4.11K blks/call 3.99%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1,732.31 ms 0.842 ms/sec 115.487 ms/call 1.14%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
15	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  (       issues.confidential IS  NOT TRUE       OR (issues.confidential  = ?         AND (issues.author_id  = ?           OR EXISTS (SELECT  ? FROM issue_assignees WHERE user_id  = ? AND issue_id = issues.id)          OR EXISTS (SELECT ? FROM  "project_authorizations" WHERE  "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = issues.project_id) AND (project_authorizations.access_level  >= ?))))) AND (EXISTS (SELECT ?  FROM "project_authorizations" WHERE  "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_l... Full query	13 0.01/sec 1.00/call 0.00%	70,936.47 ms 34.498 ms/sec 5456.651 ms/call 2.05%	13 0.01/sec 1.00/call 0.00%	1,291,857 blks 628.26 blks/sec 99.38K blks/call 0.10%	2 blks 0.00 blks/sec 0.15 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.89 ms 0.000 ms/sec 0.069 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
16	SELECT  "snippets".* FROM (SELECT  "snippets".* FROM "snippets" WHERE  (snippets.visibility_level IN ( ?,?,?) OR snippets.author_id =  ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? ORDER BY created_at DESC... Full query	13 0.01/sec 1.00/call 0.00%	61,415.22 ms 29.868 ms/sec 4724.247 ms/call 1.77%	273 0.13/sec 21.00/call 0.00%	4,954,599 blks 2.41K blks/sec 381.13K blks/call 0.36%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
17	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM (SELECT "snippets".*  FROM "snippets" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? LIMIT ... Full query	12 0.01/sec 1.00/call 0.00%	57,159.14 ms 27.798 ms/sec 4763.262 ms/call 1.65%	12 0.01/sec 1.00/call 0.00%	4,570,166 blks 2.23K blks/sec 380.85K blks/call 0.34%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
18	SELECT COUNT(*) FROM "deploy_keys_projects"  WHERE "deploy_keys_projects"."deploy_key_id"  = ? Full query	1,048 0.51/sec 1.00/call 0.01%	47,570.75 ms 23.135 ms/sec 45.392 ms/call 1.37%	1,048 0.51/sec 1.00/call 0.00%	3,833,584 blks 1.87K blks/sec 3.66K blks/call 0.28%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
19	SELECT  "projects"."id" AS t0_r0,  "projects"."name" AS t0_r1, "projects"."path"  AS t0_r2, "projects"."description"  AS t0_r3, "projects"."created_at"  AS t0_r4, "projects"."updated_at"  AS t0_r5, "projects"."creator_id"  AS t0_r6, "projects"."namespace_id"  AS t0_r7, "projects"."last_activity_at"  AS t0_r8, "projects"."import_url"  AS t0_r9, "projects"."visibility_level"  AS t0_r10, "projects"."archived"  AS t0_r11, "projects"."merge_requests_template"  AS t0_r12, "projects"."star_count"  AS t0_r13, "projects"."merge_requests_rebase_enabled"  AS t0_r14, "projects"."import_type"  AS t0_r15, "projects"."import_source"  AS t0_r16, "projects"."avatar"  AS t0_r17, "projects"."approvals_before_merge"  AS t0_r18, "projects"."reset_approvals_on_push"  AS t0_r19, "projects"."merge_requests_ff_only_enabled"  AS t0_r20, "projects"."issues_template"  AS t0_r21, "projects"."mirror"  AS t0_r22, "projects"."mirror_user_id"  AS t0_r23, "projects"."shared_runners_enabled"  AS t0_r24, "projects"."runners_token"  AS t0_r25, "... Full query	703,424 342.09/sec 1.00/call 7.31%	46,593.10 ms 22.659 ms/sec 0.066 ms/call 1.35%	693,549 337.29/sec 0.99/call 2.23%	6,296,578 blks 3.07K blks/sec 8.95 blks/call 0.46%	8,784 blks 4.27 blks/sec 0.01 blks/call 0.57%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	807.26 ms 0.393 ms/sec 0.001 ms/call 0.53%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
20	SELECT  "deployments".* FROM "deployments"  WHERE "deployments"."environment_id"  = ? AND "deployments"."status"  = ? ORDER BY deployments.id DESC  LIMIT ? Full query	10,045 4.89/sec 1.00/call 0.10%	41,793.34 ms 20.325 ms/sec 4.161 ms/call 1.21%	8,450 4.11/sec 0.84/call 0.03%	15,401,046 blks 7.49K blks/sec 1.54K blks/call 1.13%	2,681 blks 1.30 blks/sec 0.27 blks/call 0.17%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	181.49 ms 0.088 ms/sec 0.018 ms/call 0.12%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.104)

Start: 2019-09-30T13:06:20.434612+00:00
End: 2019-09-30T13:42:07.554666+00:00
Period seconds: 2147.12006
Period age: 00:35:47.120054

Top 50 rows

#	Query	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	SELECT COUNT(*) FROM "projects"  WHERE "projects"."namespace_id"  IN (WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" INNER JOIN "members"  ON "namespaces"."id" = "members"."source_id"  WHERE "members"."type" IN (?) AND  "members"."source_type" = ? AND  "namespaces"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND (members.access_level  >= ?) UNION SELECT "namespaces".*  FROM "namespaces", "base_and_descendants"  WHERE "namespaces"."type" IN (?)  AND "namespaces"."parent_id" =  "base_and_descendants"."id") SELECT  DISTINCT "namespaces"."id" FROM  "base_and_descendants" AS "namespaces"  WHERE "namespaces"."id" IN (WITH  RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces... Full query	54 0.03/sec 1.00/call 0.00%	425,151.83 ms 198.010 ms/sec 7873.182 ms/call 13.05%	54 0.03/sec 1.00/call 0.00%	427,229,864 blks 198.98K blks/sec 7.92M blks/call 30.47%	131 blks 0.06 blks/sec 2.43 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	3.69 ms 0.002 ms/sec 0.068 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	49 0.02/sec 1.00/call 0.00%	196,013.92 ms 91.292 ms/sec 4000.284 ms/call 6.02%	4,792 2.23/sec 97.80/call 0.01%	77,319,716 blks 36.02K blks/sec 1.58M blks/call 5.51%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
3	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	610 0.28/sec 1.00/call 0.01%	157,051.53 ms 73.145 ms/sec 257.462 ms/call 4.82%	6,445 3.00/sec 10.57/call 0.02%	110,031,873 blks 51.25K blks/sec 180.39K blks/call 7.85%	151 blks 0.07 blks/sec 0.25 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	13.95 ms 0.006 ms/sec 0.023 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
4	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."pending_delete"  = ? ORDER BY "projects"."created_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	24 0.01/sec 1.00/call 0.00%	154,982.97 ms 72.182 ms/sec 6457.624 ms/call 4.76%	984 0.46/sec 41.00/call 0.00%	8,754,360 blks 4.08K blks/sec 364.77K blks/call 0.62%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
5	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	573 0.27/sec 1.00/call 0.01%	148,325.30 ms 69.081 ms/sec 258.857 ms/call 4.55%	5,873 2.74/sec 10.25/call 0.02%	103,429,024 blks 48.18K blks/sec 180.51K blks/call 7.38%	155 blks 0.07 blks/sec 0.27 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	17.06 ms 0.008 ms/sec 0.030 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
6	WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces.id))) UNION SELECT  "namespaces".* FROM "namespaces",  "base_and_descendants" WHERE "namespaces"."type"  IN (?) AND "namespaces"."parent_id"  = "base_and_descendants"."id")  SELECT  "namespaces".* FROM "base_and_descendants"  AS "namespaces" WHERE "namespaces"."id"  = ? LIMIT ? Full query	27 0.01/sec 1.00/call 0.00%	130,309.01 ms 60.690 ms/sec 4826.260 ms/call 4.00%	5 0.00/sec 0.19/call 0.00%	63,997,855 blks 29.81K blks/sec 2.38M blks/call 4.56%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
7	SELECT COUNT(*) FROM (SELECT  1  AS one FROM "projects" WHERE (EXISTS  (SELECT 1 FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = 2344453 AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (10,20)) AND "projects"."visibility_level"  = 0 AND "projects"."pending_delete"  = FALSE LIMIT 10001) subquery_for_count Full query	11 0.01/sec 1.00/call 0.00%	122,387.00 ms 57.001 ms/sec 11126.091 ms/call 3.76%	11 0.01/sec 1.00/call 0.00%	54,857,519 blks 25.55K blks/sec 4.99M blks/call 3.91%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
8	SELECT COUNT(*) FROM "projects"  WHERE (EXISTS (SELECT 1 FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = 4465626 AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (10,20)) Full query	38 0.02/sec 1.00/call 0.00%	122,346.72 ms 56.982 ms/sec 3219.650 ms/call 3.75%	38 0.02/sec 1.00/call 0.00%	13,859,952 blks 6.46K blks/sec 364.74K blks/call 0.99%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
9	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  projects.id FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	28 0.01/sec 1.00/call 0.00%	122,252.17 ms 56.938 ms/sec 4366.149 ms/call 3.75%	56 0.03/sec 2.00/call 0.00%	56,142,832 blks 26.15K blks/sec 2.01M blks/call 4.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
10	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	27 0.01/sec 1.00/call 0.00%	109,269.60 ms 50.891 ms/sec 4047.022 ms/call 3.35%	82 0.04/sec 3.04/call 0.00%	51,152,887 blks 23.83K blks/sec 1.90M blks/call 3.65%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
11	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  'Namespace' AND "members"."type"  IN ('GroupMember') AND "members"."user_id"  = 2344453 AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT 1 FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = 2344453 AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (10,20)) AND "projects"."visibility_level"  = 0 AND "projects"."pending_delete"  = FALSE ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  50 OFFSET 0) Full query	9 0.00/sec 1.00/call 0.00%	100,659.04 ms 46.881 ms/sec 11184.338 ms/call 3.09%	18 0.01/sec 2.00/call 0.00%	44,919,710 blks 20.93K blks/sec 5.00M blks/call 3.20%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
12	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	9 0.00/sec 1.00/call 0.00%	100,592.23 ms 46.850 ms/sec 11176.915 ms/call 3.09%	427 0.20/sec 47.44/call 0.00%	44,916,931 blks 20.92K blks/sec 5.00M blks/call 3.20%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
13	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	28 0.01/sec 1.00/call 0.00%	79,299.71 ms 36.933 ms/sec 2832.133 ms/call 2.43%	28 0.01/sec 1.00/call 0.00%	35,194,901 blks 16.40K blks/sec 1.26M blks/call 2.51%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
14	SELECT  "snippets".* FROM (SELECT  "snippets".* FROM "snippets" WHERE  (snippets.visibility_level IN ( ?,?,?) OR snippets.author_id =  ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? ORDER BY created_at DESC... Full query	16 0.01/sec 1.00/call 0.00%	74,717.10 ms 34.799 ms/sec 4669.819 ms/call 2.29%	336 0.16/sec 21.00/call 0.00%	6,099,856 blks 2.85K blks/sec 381.25K blks/call 0.44%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
15	SELECT  issues.*, (SELECT MIN("label_priorities"."priority")  FROM "labels" INNER JOIN "label_links"  ON "label_links"."label_id" = "labels"."id"  LEFT OUTER JOIN "label_priorities"  ON "labels"."id" = "label_priorities"."label_id"  WHERE (label_priorities.project_id  = issues.project_id) AND (label_links.target_id  = issues.id) AND "label_links"."target_type"  = ?) AS highest_priority FROM "issues"  WHERE "issues"."project_id" = ?  AND ("issues"."state" IN (?)) GROUP  BY "issues"."id" ORDER BY relative_position  ASC NULLS LAST, highest_priority  ASC NULLS LAST, id DESC LIMIT ?  OFFSET ? Full query	1,039 0.48/sec 1.00/call 0.01%	73,925.58 ms 34.430 ms/sec 71.151 ms/call 2.27%	19,013 8.86/sec 18.30/call 0.06%	56,639,127 blks 26.38K blks/sec 54.52K blks/call 4.04%	19,159 blks 8.92 blks/sec 18.44 blks/call 1.03%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1,616.21 ms 0.753 ms/sec 1.556 ms/call 0.84%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
16	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM (SELECT "snippets".*  FROM "snippets" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? LIMIT ... Full query	16 0.01/sec 1.00/call 0.00%	73,810.69 ms 34.377 ms/sec 4613.168 ms/call 2.27%	16 0.01/sec 1.00/call 0.00%	6,094,077 blks 2.84K blks/sec 380.88K blks/call 0.43%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
17	SELECT COUNT(*) FROM "deploy_keys_projects"  WHERE "deploy_keys_projects"."deploy_key_id"  = ? Full query	1,446 0.67/sec 1.00/call 0.01%	63,633.56 ms 29.637 ms/sec 44.007 ms/call 1.95%	1,446 0.67/sec 1.00/call 0.00%	5,289,468 blks 2.47K blks/sec 3.66K blks/call 0.38%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
18	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  "issues"."confidential" = ? AND  (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_level"  = ? AND EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)))) AND ("issues"."title"  ILIKE ? OR "issues"."description"  ILIKE ?) LIMIT ?) subquery_for_count Full query	10 0.00/sec 1.00/call 0.00%	57,700.37 ms 26.873 ms/sec 5770.037 ms/call 1.77%	10 0.00/sec 1.00/call 0.00%	899,435 blks 418.90 blks/sec 89.95K blks/call 0.06%	28,382 blks 13.22 blks/sec 2.84K blks/call 1.52%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	779.49 ms 0.363 ms/sec 77.949 ms/call 0.40%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
19	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  (       issues.confidential IS  NOT TRUE       OR (issues.confidential  = ?         AND (issues.author_id  = ?           OR EXISTS (SELECT  ? FROM issue_assignees WHERE user_id  = ? AND issue_id = issues.id)          OR EXISTS (SELECT ? FROM  "project_authorizations" WHERE  "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = issues.project_id) AND (project_authorizations.access_level  >= ?))))) AND (EXISTS (SELECT ?  FROM "project_authorizations" WHERE  "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_l... Full query	9 0.00/sec 1.00/call 0.00%	55,167.79 ms 25.694 ms/sec 6129.754 ms/call 1.69%	9 0.00/sec 1.00/call 0.00%	877,642 blks 408.75 blks/sec 97.52K blks/call 0.06%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
20	SELECT  "projects"."id" AS t0_r0,  "projects"."name" AS t0_r1, "projects"."path"  AS t0_r2, "projects"."description"  AS t0_r3, "projects"."created_at"  AS t0_r4, "projects"."updated_at"  AS t0_r5, "projects"."creator_id"  AS t0_r6, "projects"."namespace_id"  AS t0_r7, "projects"."last_activity_at"  AS t0_r8, "projects"."import_url"  AS t0_r9, "projects"."visibility_level"  AS t0_r10, "projects"."archived"  AS t0_r11, "projects"."merge_requests_template"  AS t0_r12, "projects"."star_count"  AS t0_r13, "projects"."merge_requests_rebase_enabled"  AS t0_r14, "projects"."import_type"  AS t0_r15, "projects"."import_source"  AS t0_r16, "projects"."avatar"  AS t0_r17, "projects"."approvals_before_merge"  AS t0_r18, "projects"."reset_approvals_on_push"  AS t0_r19, "projects"."merge_requests_ff_only_enabled"  AS t0_r20, "projects"."issues_template"  AS t0_r21, "projects"."mirror"  AS t0_r22, "projects"."mirror_user_id"  AS t0_r23, "projects"."shared_runners_enabled"  AS t0_r24, "projects"."runners_token"  AS t0_r25, "... Full query	735,933 342.75/sec 1.00/call 7.38%	44,158.09 ms 20.566 ms/sec 0.060 ms/call 1.36%	725,714 337.99/sec 0.99/call 2.24%	6,587,082 blks 3.07K blks/sec 8.95 blks/call 0.47%	9,898 blks 4.61 blks/sec 0.01 blks/call 0.53%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	904.31 ms 0.421 ms/sec 0.001 ms/call 0.47%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.105)

Start: 2019-09-30T13:06:23.43192+00:00
End: 2019-09-30T13:44:07.798909+00:00
Period seconds: 2264.36699
Period age: 00:37:44.366989

Top 50 rows

#	Query	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	SELECT COUNT(*) FROM "projects"  WHERE "projects"."namespace_id"  IN (WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" INNER JOIN "members"  ON "namespaces"."id" = "members"."source_id"  WHERE "members"."type" IN (?) AND  "members"."source_type" = ? AND  "namespaces"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND (members.access_level  >= ?) UNION SELECT "namespaces".*  FROM "namespaces", "base_and_descendants"  WHERE "namespaces"."type" IN (?)  AND "namespaces"."parent_id" =  "base_and_descendants"."id") SELECT  DISTINCT "namespaces"."id" FROM  "base_and_descendants" AS "namespaces"  WHERE "namespaces"."id" IN (WITH  RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces... Full query	63 0.03/sec 1.00/call 0.00%	503,485.08 ms 222.351 ms/sec 7991.827 ms/call 15.31%	63 0.03/sec 1.00/call 0.00%	491,189,587 blks 216.93K blks/sec 7.80M blks/call 34.42%	11 blks 0.00 blks/sec 0.17 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1.04 ms 0.000 ms/sec 0.017 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	50 0.02/sec 1.00/call 0.00%	234,527.29 ms 103.573 ms/sec 4690.546 ms/call 7.13%	4,747 2.10/sec 94.94/call 0.01%	91,966,683 blks 40.62K blks/sec 1.84M blks/call 6.44%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
3	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."pending_delete"  = ? ORDER BY "projects"."created_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	30 0.01/sec 1.00/call 0.00%	202,768.20 ms 89.547 ms/sec 6758.940 ms/call 6.16%	1,072 0.47/sec 35.73/call 0.00%	10,942,725 blks 4.84K blks/sec 364.76K blks/call 0.77%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
4	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	624 0.28/sec 1.00/call 0.01%	164,677.31 ms 72.726 ms/sec 263.906 ms/call 5.01%	6,766 2.99/sec 10.84/call 0.02%	113,486,706 blks 50.12K blks/sec 181.87K blks/call 7.95%	139 blks 0.06 blks/sec 0.22 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	6.66 ms 0.003 ms/sec 0.011 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
5	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	612 0.27/sec 1.00/call 0.01%	162,182.85 ms 71.624 ms/sec 265.005 ms/call 4.93%	6,560 2.90/sec 10.72/call 0.02%	111,052,793 blks 49.05K blks/sec 181.46K blks/call 7.78%	166 blks 0.07 blks/sec 0.27 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	17.70 ms 0.008 ms/sec 0.029 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
6	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  projects.id FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	30 0.01/sec 1.00/call 0.00%	160,481.69 ms 70.873 ms/sec 5349.390 ms/call 4.88%	248 0.11/sec 8.27/call 0.00%	70,899,799 blks 31.32K blks/sec 2.37M blks/call 4.97%	2 blks 0.00 blks/sec 0.07 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.03 ms 0.000 ms/sec 0.001 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
7	WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN ('Group') AND (EXISTS (SELECT  1 FROM "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN ('silver', 'gold') AND (gitlab_subscriptions.namespace_id  = namespaces.id))) UNION SELECT  "namespaces".* FROM "namespaces",  "base_and_descendants" WHERE "namespaces"."type"  IN ('Group') AND "namespaces"."parent_id"  = "base_and_descendants"."id")  SELECT  "namespaces".* FROM "base_and_descendants"  AS "namespaces" WHERE "namespaces"."id"  = 5982968 LIMIT 1 Full query	25 0.01/sec 1.00/call 0.00%	140,430.99 ms 62.018 ms/sec 5617.240 ms/call 4.27%	0 0.00/sec 0.00/call 0.00%	68,626,563 blks 30.31K blks/sec 2.75M blks/call 4.81%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
8	SELECT COUNT(*) FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) Full query	39 0.02/sec 1.00/call 0.00%	137,340.65 ms 60.653 ms/sec 3521.555 ms/call 4.18%	39 0.02/sec 1.00/call 0.00%	14,224,684 blks 6.29K blks/sec 364.74K blks/call 1.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
9	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	31 0.01/sec 1.00/call 0.00%	119,214.06 ms 52.648 ms/sec 3845.615 ms/call 3.62%	31 0.01/sec 1.00/call 0.00%	50,060,479 blks 22.11K blks/sec 1.62M blks/call 3.51%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
10	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	25 0.01/sec 1.00/call 0.00%	102,157.57 ms 45.115 ms/sec 4086.303 ms/call 3.11%	38 0.02/sec 1.52/call 0.00%	45,973,593 blks 20.31K blks/sec 1.84M blks/call 3.22%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
11	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	7 0.00/sec 1.00/call 0.00%	82,751.08 ms 36.545 ms/sec 11821.583 ms/call 2.52%	16 0.01/sec 2.29/call 0.00%	34,957,240 blks 15.44K blks/sec 5.00M blks/call 2.45%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
12	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	7 0.00/sec 1.00/call 0.00%	81,811.54 ms 36.130 ms/sec 11687.363 ms/call 2.49%	7 0.00/sec 1.00/call 0.00%	34,953,370 blks 15.44K blks/sec 5.00M blks/call 2.45%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
13	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	7 0.00/sec 1.00/call 0.00%	80,851.20 ms 35.706 ms/sec 11550.171 ms/call 2.46%	310 0.14/sec 44.29/call 0.00%	34,954,953 blks 15.44K blks/sec 5.00M blks/call 2.45%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
14	SELECT COUNT(*) FROM "deploy_keys_projects"  WHERE "deploy_keys_projects"."deploy_key_id"  = ? Full query	1,380 0.61/sec 1.00/call 0.01%	60,794.49 ms 26.848 ms/sec 44.054 ms/call 1.85%	1,380 0.61/sec 1.00/call 0.00%	5,048,040 blks 2.23K blks/sec 3.66K blks/call 0.35%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
15	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  "issues"."confidential" = ? AND  (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_level"  = ? AND EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)))) AND ("issues"."title"  ILIKE ? OR "issues"."description"  ILIKE ?) LIMIT ?) subquery_for_count Full query	16 0.01/sec 1.00/call 0.00%	60,698.07 ms 26.806 ms/sec 3793.629 ms/call 1.85%	16 0.01/sec 1.00/call 0.00%	1,139,728 blks 503.33 blks/sec 71.24K blks/call 0.08%	30,341 blks 13.40 blks/sec 1.90K blks/call 1.69%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	398.32 ms 0.176 ms/sec 24.895 ms/call 0.27%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
16	SELECT  "deployments".* FROM "deployments"  WHERE "deployments"."environment_id"  = ? AND "deployments"."status"  = ? ORDER BY deployments.id DESC  LIMIT ? Full query	12,996 5.74/sec 1.00/call 0.12%	53,703.72 ms 23.717 ms/sec 4.132 ms/call 1.63%	10,555 4.66/sec 0.81/call 0.03%	20,930,517 blks 9.25K blks/sec 1.62K blks/call 1.47%	6,694 blks 2.96 blks/sec 0.52 blks/call 0.37%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	288.61 ms 0.127 ms/sec 0.022 ms/call 0.19%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
17	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  (       issues.confidential IS  NOT TRUE       OR (issues.confidential  = ?         AND (issues.author_id  = ?           OR EXISTS (SELECT  ? FROM issue_assignees WHERE user_id  = ? AND issue_id = issues.id)          OR EXISTS (SELECT ? FROM  "project_authorizations" WHERE  "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = issues.project_id) AND (project_authorizations.access_level  >= ?))))) AND (EXISTS (SELECT ?  FROM "project_authorizations" WHERE  "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_l... Full query	10 0.00/sec 1.00/call 0.00%	53,394.19 ms 23.580 ms/sec 5339.419 ms/call 1.62%	10 0.00/sec 1.00/call 0.00%	970,791 blks 428.73 blks/sec 97.08K blks/call 0.07%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
18	SELECT  "projects"."id" AS t0_r0,  "projects"."name" AS t0_r1, "projects"."path"  AS t0_r2, "projects"."description"  AS t0_r3, "projects"."created_at"  AS t0_r4, "projects"."updated_at"  AS t0_r5, "projects"."creator_id"  AS t0_r6, "projects"."namespace_id"  AS t0_r7, "projects"."last_activity_at"  AS t0_r8, "projects"."import_url"  AS t0_r9, "projects"."visibility_level"  AS t0_r10, "projects"."archived"  AS t0_r11, "projects"."merge_requests_template"  AS t0_r12, "projects"."star_count"  AS t0_r13, "projects"."merge_requests_rebase_enabled"  AS t0_r14, "projects"."import_type"  AS t0_r15, "projects"."import_source"  AS t0_r16, "projects"."avatar"  AS t0_r17, "projects"."approvals_before_merge"  AS t0_r18, "projects"."reset_approvals_on_push"  AS t0_r19, "projects"."merge_requests_ff_only_enabled"  AS t0_r20, "projects"."issues_template"  AS t0_r21, "projects"."mirror"  AS t0_r22, "projects"."mirror_user_id"  AS t0_r23, "projects"."shared_runners_enabled"  AS t0_r24, "projects"."runners_token"  AS t0_r25, "... Full query	775,154 342.33/sec 1.00/call 7.37%	46,425.47 ms 20.503 ms/sec 0.060 ms/call 1.41%	764,234 337.50/sec 0.99/call 2.22%	6,938,109 blks 3.07K blks/sec 8.95 blks/call 0.49%	10,283 blks 4.54 blks/sec 0.01 blks/call 0.57%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	808.13 ms 0.357 ms/sec 0.001 ms/call 0.54%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
19	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	143 0.06/sec 1.00/call 0.00%	35,356.14 ms 15.614 ms/sec 247.246 ms/call 1.07%	4,227 1.87/sec 29.56/call 0.01%	24,531,827 blks 10.84K blks/sec 171.56K blks/call 1.72%	23 blks 0.01 blks/sec 0.16 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	27.52 ms 0.012 ms/sec 0.192 ms/call 0.02%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
20	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  projects.id FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	7 0.00/sec 1.00/call 0.00%	34,439.11 ms 15.209 ms/sec 4919.874 ms/call 1.05%	3 0.00/sec 0.43/call 0.00%	15,001,685 blks 6.63K blks/sec 2.15M blks/call 1.05%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.106)

Start: 2019-09-30T13:06:25.816873+00:00
End: 2019-09-30T13:46:29.215889+00:00
Period seconds: 2403.39902
Period age: 00:40:03.399016

Top 50 rows

#	Query	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	SELECT COUNT(*) FROM "projects"  WHERE "projects"."namespace_id"  IN (WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" INNER JOIN "members"  ON "namespaces"."id" = "members"."source_id"  WHERE "members"."type" IN (?) AND  "members"."source_type" = ? AND  "namespaces"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND (members.access_level  >= ?) UNION SELECT "namespaces".*  FROM "namespaces", "base_and_descendants"  WHERE "namespaces"."type" IN (?)  AND "namespaces"."parent_id" =  "base_and_descendants"."id") SELECT  DISTINCT "namespaces"."id" FROM  "base_and_descendants" AS "namespaces"  WHERE "namespaces"."id" IN (WITH  RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces... Full query	64 0.03/sec 1.00/call 0.00%	567,927.93 ms 236.302 ms/sec 8873.874 ms/call 17.29%	64 0.03/sec 1.00/call 0.00%	509,167,046 blks 211.86K blks/sec 7.96M blks/call 37.67%	63 blks 0.03 blks/sec 0.98 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	4.01 ms 0.002 ms/sec 0.063 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	706 0.29/sec 1.00/call 0.01%	196,001.25 ms 81.552 ms/sec 277.622 ms/call 5.97%	8,407 3.50/sec 11.91/call 0.02%	129,006,154 blks 53.68K blks/sec 182.73K blks/call 9.54%	163 blks 0.07 blks/sec 0.23 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	7.52 ms 0.003 ms/sec 0.011 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
3	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."pending_delete"  = ? ORDER BY "projects"."created_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	28 0.01/sec 1.00/call 0.00%	185,559.72 ms 77.207 ms/sec 6627.133 ms/call 5.65%	908 0.38/sec 32.43/call 0.00%	10,214,258 blks 4.25K blks/sec 364.80K blks/call 0.76%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
4	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	621 0.26/sec 1.00/call 0.01%	179,793.10 ms 74.808 ms/sec 289.522 ms/call 5.47%	7,118 2.96/sec 11.46/call 0.02%	113,646,053 blks 47.29K blks/sec 183.01K blks/call 8.41%	131 blks 0.05 blks/sec 0.21 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	29.82 ms 0.012 ms/sec 0.048 ms/call 0.02%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
5	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	37 0.02/sec 1.00/call 0.00%	159,182.14 ms 66.232 ms/sec 4302.220 ms/call 4.85%	3,620 1.51/sec 97.84/call 0.01%	59,131,195 blks 24.61K blks/sec 1.60M blks/call 4.37%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
6	SELECT COUNT(*) FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) Full query	44 0.02/sec 1.00/call 0.00%	146,659.20 ms 61.022 ms/sec 3333.164 ms/call 4.47%	44 0.02/sec 1.00/call 0.00%	16,048,479 blks 6.68K blks/sec 364.74K blks/call 1.19%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
7	WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces.id))) UNION SELECT  "namespaces".* FROM "namespaces",  "base_and_descendants" WHERE "namespaces"."type"  IN (?) AND "namespaces"."parent_id"  = "base_and_descendants"."id")  SELECT  "namespaces".* FROM "base_and_descendants"  AS "namespaces" WHERE "namespaces"."id"  = ? LIMIT ? Full query	21 0.01/sec 1.00/call 0.00%	133,447.18 ms 55.524 ms/sec 6354.628 ms/call 4.06%	0 0.00/sec 0.00/call 0.00%	57,657,806 blks 24.00K blks/sec 2.75M blks/call 4.27%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
8	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	8 0.00/sec 1.00/call 0.00%	96,590.50 ms 40.189 ms/sec 12073.813 ms/call 2.94%	8 0.00/sec 1.00/call 0.00%	39,879,164 blks 16.60K blks/sec 4.99M blks/call 2.95%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
9	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	7 0.00/sec 1.00/call 0.00%	82,330.25 ms 34.256 ms/sec 11761.464 ms/call 2.51%	357 0.15/sec 51.00/call 0.00%	34,910,367 blks 14.53K blks/sec 4.99M blks/call 2.58%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
10	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	7 0.00/sec 1.00/call 0.00%	81,301.40 ms 33.828 ms/sec 11614.486 ms/call 2.48%	13 0.01/sec 1.86/call 0.00%	34,912,337 blks 14.53K blks/sec 4.99M blks/call 2.58%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
11	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  'Project' AND "members"."type"  IN ('ProjectMember') AND "members"."user_id"  = 3611581 AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  projects.id FROM "projects"  WHERE (EXISTS (SELECT 1 FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = 3611581 AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (10,20)) AND "projects"."visibility_level"  = 0 AND "projects"."archived" =  FALSE AND "projects"."pending_delete"  = FALSE ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  100 OFFSET 0) Full query	20 0.01/sec 1.00/call 0.00%	77,967.87 ms 32.441 ms/sec 3898.394 ms/call 2.37%	98 0.04/sec 4.90/call 0.00%	37,044,489 blks 15.42K blks/sec 1.86M blks/call 2.74%	26 blks 0.01 blks/sec 1.30 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.50 ms 0.000 ms/sec 0.025 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
12	SELECT COUNT(*) FROM "deploy_keys_projects"  WHERE "deploy_keys_projects"."deploy_key_id"  = ? Full query	1,367 0.57/sec 1.00/call 0.01%	64,716.68 ms 26.927 ms/sec 47.342 ms/call 1.97%	1,367 0.57/sec 1.00/call 0.00%	5,000,486 blks 2.09K blks/sec 3.66K blks/call 0.37%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
13	SELECT COUNT(*) FROM (SELECT  1  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  "issues"."confidential" = FALSE  AND (EXISTS (SELECT 1 FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = 4590286 AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= 10)) OR projects.visibility_level  IN (10,20)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (20,30) OR ("project_features"."issues_access_level"  = 10 AND EXISTS (SELECT 1 FROM  "project_authorizations" WHERE  "project_authorizations"."user_id"  = 4590286 AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= 10)))) AND ("issues"."title"  ILIKE '%tetrix%' OR "issues"."description"  ILIKE '%tetrix%') LIMIT 1001) subquery_for_count Full query	12 0.00/sec 1.00/call 0.00%	64,593.19 ms 26.876 ms/sec 5382.766 ms/call 1.97%	12 0.00/sec 1.00/call 0.00%	1,126,817 blks 468.84 blks/sec 93.91K blks/call 0.08%	86,501 blks 35.99 blks/sec 7.21K blks/call 4.14%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1,388.89 ms 0.578 ms/sec 115.741 ms/call 1.07%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
14	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  (       issues.confidential IS  NOT TRUE       OR (issues.confidential  = ?         AND (issues.author_id  = ?           OR EXISTS (SELECT  ? FROM issue_assignees WHERE user_id  = ? AND issue_id = issues.id)          OR EXISTS (SELECT ? FROM  "project_authorizations" WHERE  "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = issues.project_id) AND (project_authorizations.access_level  >= ?))))) AND (EXISTS (SELECT ?  FROM "project_authorizations" WHERE  "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_l... Full query	10 0.00/sec 1.00/call 0.00%	61,049.63 ms 25.401 ms/sec 6104.963 ms/call 1.86%	10 0.00/sec 1.00/call 0.00%	1,137,027 blks 473.09 blks/sec 113.71K blks/call 0.08%	1 blks 0.00 blks/sec 0.10 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.02 ms 0.000 ms/sec 0.002 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
15	SELECT  "deployments".* FROM "deployments"  WHERE "deployments"."environment_id"  = ? AND "deployments"."status"  = ? ORDER BY deployments.id DESC  LIMIT ? Full query	13,535 5.63/sec 1.00/call 0.12%	58,725.86 ms 24.435 ms/sec 4.339 ms/call 1.79%	11,227 4.67/sec 0.83/call 0.03%	20,499,721 blks 8.53K blks/sec 1.52K blks/call 1.52%	3,259 blks 1.36 blks/sec 0.24 blks/call 0.16%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	147.46 ms 0.061 ms/sec 0.011 ms/call 0.11%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
16	SELECT  "snippets".* FROM (SELECT  "snippets".* FROM "snippets" WHERE  (snippets.visibility_level IN ( ?,?,?) OR snippets.author_id =  ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? ORDER BY created_at DESC... Full query	12 0.00/sec 1.00/call 0.00%	56,140.74 ms 23.359 ms/sec 4678.395 ms/call 1.71%	252 0.10/sec 21.00/call 0.00%	4,574,727 blks 1.91K blks/sec 381.23K blks/call 0.34%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
17	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	18 0.01/sec 1.00/call 0.00%	55,848.24 ms 23.237 ms/sec 3102.680 ms/call 1.70%	5 0.00/sec 0.28/call 0.00%	27,056,223 blks 11.26K blks/sec 1.51M blks/call 2.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
18	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM (SELECT "snippets".*  FROM "snippets" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? LIMIT ... Full query	12 0.00/sec 1.00/call 0.00%	55,737.74 ms 23.191 ms/sec 4644.811 ms/call 1.70%	12 0.00/sec 1.00/call 0.00%	4,570,833 blks 1.91K blks/sec 380.91K blks/call 0.34%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
19	SELECT  "projects"."id" AS t0_r0,  "projects"."name" AS t0_r1, "projects"."path"  AS t0_r2, "projects"."description"  AS t0_r3, "projects"."created_at"  AS t0_r4, "projects"."updated_at"  AS t0_r5, "projects"."creator_id"  AS t0_r6, "projects"."namespace_id"  AS t0_r7, "projects"."last_activity_at"  AS t0_r8, "projects"."import_url"  AS t0_r9, "projects"."visibility_level"  AS t0_r10, "projects"."archived"  AS t0_r11, "projects"."merge_requests_template"  AS t0_r12, "projects"."star_count"  AS t0_r13, "projects"."merge_requests_rebase_enabled"  AS t0_r14, "projects"."import_type"  AS t0_r15, "projects"."import_source"  AS t0_r16, "projects"."avatar"  AS t0_r17, "projects"."approvals_before_merge"  AS t0_r18, "projects"."reset_approvals_on_push"  AS t0_r19, "projects"."merge_requests_ff_only_enabled"  AS t0_r20, "projects"."issues_template"  AS t0_r21, "projects"."mirror"  AS t0_r22, "projects"."mirror_user_id"  AS t0_r23, "projects"."shared_runners_enabled"  AS t0_r24, "projects"."runners_token"  AS t0_r25, "... Full query	824,108 342.89/sec 1.00/call 7.28%	53,063.22 ms 22.078 ms/sec 0.064 ms/call 1.62%	812,308 337.98/sec 0.99/call 2.23%	7,374,590 blks 3.07K blks/sec 8.95 blks/call 0.55%	11,233 blks 4.67 blks/sec 0.01 blks/call 0.54%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	429.20 ms 0.179 ms/sec 0.001 ms/call 0.33%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
20	SELECT  "users".* FROM "users" WHERE  ("users"."state" IN (?)) AND (ghost  IS NOT TRUE) AND "users"."bot_type"  IS NULL AND (("users"."name" ILIKE  ? OR "users"."username" ILIKE ?)  OR "users"."email" = ?) ORDER BY  CASE   WHEN users.name = ? THEN  ?   WHEN users.username = ? THEN  ?   WHEN users.email = ? THEN ?  ELSE ? END , "users"."name" ASC  LIMIT ? Full query	702 0.29/sec 1.00/call 0.01%	42,994.35 ms 17.889 ms/sec 61.246 ms/call 1.31%	6,560 2.73/sec 9.34/call 0.02%	4,482,508 blks 1.87K blks/sec 6.39K blks/call 0.33%	1,618 blks 0.67 blks/sec 2.30 blks/call 0.08%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	87.95 ms 0.037 ms/sec 0.125 ms/call 0.07%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.107)

Start: 2019-09-30T13:06:28.521327+00:00
End: 2019-09-30T13:49:34.408897+00:00
Period seconds: 2585.88757
Period age: 00:43:05.88757

Top 50 rows

#	Query	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	SELECT COUNT(*) FROM "projects"  WHERE "projects"."namespace_id"  IN (WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" INNER JOIN "members"  ON "namespaces"."id" = "members"."source_id"  WHERE "members"."type" IN (?) AND  "members"."source_type" = ? AND  "namespaces"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND (members.access_level  >= ?) UNION SELECT "namespaces".*  FROM "namespaces", "base_and_descendants"  WHERE "namespaces"."type" IN (?)  AND "namespaces"."parent_id" =  "base_and_descendants"."id") SELECT  DISTINCT "namespaces"."id" FROM  "base_and_descendants" AS "namespaces"  WHERE "namespaces"."id" IN (WITH  RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces... Full query	60 0.02/sec 1.00/call 0.00%	477,767.68 ms 184.760 ms/sec 7962.795 ms/call 12.08%	60 0.02/sec 1.00/call 0.00%	474,724,055 blks 183.59K blks/sec 7.92M blks/call 29.11%	101 blks 0.04 blks/sec 1.68 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1.90 ms 0.001 ms/sec 0.032 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."pending_delete"  = ? ORDER BY "projects"."created_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	37 0.01/sec 1.00/call 0.00%	254,838.33 ms 98.550 ms/sec 6887.523 ms/call 6.44%	1,097 0.42/sec 29.65/call 0.00%	13,496,632 blks 5.22K blks/sec 364.78K blks/call 0.83%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
3	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	59 0.02/sec 1.00/call 0.00%	194,234.21 ms 75.113 ms/sec 3292.105 ms/call 4.91%	5,803 2.24/sec 98.36/call 0.01%	77,341,363 blks 29.91K blks/sec 1.32M blks/call 4.74%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
4	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	17 0.01/sec 1.00/call 0.00%	193,833.97 ms 74.958 ms/sec 11401.998 ms/call 4.90%	17 0.01/sec 1.00/call 0.00%	84,874,727 blks 32.83K blks/sec 5.00M blks/call 5.20%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
5	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	728 0.28/sec 1.00/call 0.01%	192,104.03 ms 74.289 ms/sec 263.879 ms/call 4.86%	6,634 2.57/sec 9.11/call 0.02%	134,316,602 blks 51.95K blks/sec 184.51K blks/call 8.23%	185 blks 0.07 blks/sec 0.25 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	15.31 ms 0.006 ms/sec 0.021 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
6	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	707 0.27/sec 1.00/call 0.01%	187,742.47 ms 72.603 ms/sec 265.548 ms/call 4.75%	7,073 2.74/sec 10.00/call 0.02%	130,425,234 blks 50.44K blks/sec 184.48K blks/call 8.00%	187 blks 0.07 blks/sec 0.26 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	14.81 ms 0.006 ms/sec 0.021 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
7	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	16 0.01/sec 1.00/call 0.00%	182,431.48 ms 70.549 ms/sec 11401.968 ms/call 4.61%	29 0.01/sec 1.81/call 0.00%	79,867,869 blks 30.89K blks/sec 5.00M blks/call 4.90%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
8	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	16 0.01/sec 1.00/call 0.00%	181,596.24 ms 70.226 ms/sec 11349.765 ms/call 4.59%	801 0.31/sec 50.06/call 0.00%	79,862,999 blks 30.89K blks/sec 5.00M blks/call 4.90%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
9	WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN ('Group') AND (EXISTS (SELECT  1 FROM "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN ('silver', 'gold') AND (gitlab_subscriptions.namespace_id  = namespaces.id))) UNION SELECT  "namespaces".* FROM "namespaces",  "base_and_descendants" WHERE "namespaces"."type"  IN ('Group') AND "namespaces"."parent_id"  = "base_and_descendants"."id")  SELECT  "namespaces".* FROM "base_and_descendants"  AS "namespaces" WHERE "namespaces"."id"  = 5834003 LIMIT 1 Full query	32 0.01/sec 1.00/call 0.00%	159,553.29 ms 61.702 ms/sec 4986.040 ms/call 4.03%	4 0.00/sec 0.12/call 0.00%	79,311,562 blks 30.68K blks/sec 2.48M blks/call 4.86%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
10	SELECT COUNT(*) FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) Full query	40 0.02/sec 1.00/call 0.00%	142,350.07 ms 55.049 ms/sec 3558.752 ms/call 3.60%	40 0.02/sec 1.00/call 0.00%	14,589,455 blks 5.65K blks/sec 364.74K blks/call 0.89%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
11	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  projects.id FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	33 0.01/sec 1.00/call 0.00%	125,675.28 ms 48.600 ms/sec 3808.342 ms/call 3.18%	58 0.02/sec 1.76/call 0.00%	56,113,723 blks 21.70K blks/sec 1.71M blks/call 3.44%	41 blks 0.02 blks/sec 1.24 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.41 ms 0.000 ms/sec 0.012 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
12	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	32 0.01/sec 1.00/call 0.00%	111,737.04 ms 43.210 ms/sec 3491.782 ms/call 2.82%	74 0.03/sec 2.31/call 0.00%	51,114,383 blks 19.77K blks/sec 1.60M blks/call 3.13%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
13	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	34 0.01/sec 1.00/call 0.00%	105,210.79 ms 40.687 ms/sec 3094.435 ms/call 2.66%	34 0.01/sec 1.00/call 0.00%	45,170,150 blks 17.47K blks/sec 1.33M blks/call 2.77%	3 blks 0.00 blks/sec 0.09 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.03 ms 0.000 ms/sec 0.001 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
14	SELECT  "snippets".* FROM (SELECT  "snippets".* FROM "snippets" WHERE  (snippets.visibility_level IN ( 0,10,20) OR snippets.author_id  = 2166229) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (0,10,20) OR snippets.author_id  = 2166229) AND (projects.visibility_level  IN (0,10,20)) AND "project_features"."snippets_access_level"  IN (20, 30) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (20, 30, 10) AND (EXISTS (SELECT  1 FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = 2166229))) snippets WHERE "snippets"."visibility_level"... Full query	21 0.01/sec 1.00/call 0.00%	98,810.10 ms 38.211 ms/sec 4705.243 ms/call 2.50%	441 0.17/sec 21.00/call 0.00%	8,003,525 blks 3.10K blks/sec 381.13K blks/call 0.49%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
15	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM (SELECT "snippets".*  FROM "snippets" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? LIMIT ... Full query	21 0.01/sec 1.00/call 0.00%	97,146.27 ms 37.568 ms/sec 4626.013 ms/call 2.46%	21 0.01/sec 1.00/call 0.00%	7,997,653 blks 3.10K blks/sec 380.85K blks/call 0.49%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
16	SELECT COUNT(*) FROM "deploy_keys_projects"  WHERE "deploy_keys_projects"."deploy_key_id"  = ? Full query	1,417 0.55/sec 1.00/call 0.01%	63,513.47 ms 24.562 ms/sec 44.822 ms/call 1.61%	1,417 0.55/sec 1.00/call 0.00%	5,183,386 blks 2.01K blks/sec 3.66K blks/call 0.32%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
17	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  "issues"."confidential" = ? AND  (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_level"  = ? AND EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)))) AND ("issues"."title"  ILIKE ? OR "issues"."description"  ILIKE ?) LIMIT ?) subquery_for_count Full query	15 0.01/sec 1.00/call 0.00%	61,314.33 ms 23.711 ms/sec 4087.622 ms/call 1.55%	15 0.01/sec 1.00/call 0.00%	1,119,111 blks 432.78 blks/sec 74.61K blks/call 0.07%	51,695 blks 19.99 blks/sec 3.45K blks/call 2.59%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	772.49 ms 0.299 ms/sec 51.499 ms/call 0.62%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
18	SELECT  "deployments".* FROM "deployments"  WHERE "deployments"."environment_id"  = ? AND "deployments"."status"  = ? ORDER BY deployments.id DESC  LIMIT ? Full query	14,618 5.65/sec 1.00/call 0.12%	60,733.22 ms 23.486 ms/sec 4.155 ms/call 1.54%	11,997 4.64/sec 0.82/call 0.03%	23,546,028 blks 9.11K blks/sec 1.62K blks/call 1.44%	8,131 blks 3.14 blks/sec 0.56 blks/call 0.41%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	217.73 ms 0.084 ms/sec 0.015 ms/call 0.18%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
19	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  projects.id FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	16 0.01/sec 1.00/call 0.00%	56,278.69 ms 21.764 ms/sec 3517.418 ms/call 1.42%	5 0.00/sec 0.31/call 0.00%	24,996,979 blks 9.67K blks/sec 1.57M blks/call 1.53%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
20	SELECT  "projects"."id" AS t0_r0,  "projects"."name" AS t0_r1, "projects"."path"  AS t0_r2, "projects"."description"  AS t0_r3, "projects"."created_at"  AS t0_r4, "projects"."updated_at"  AS t0_r5, "projects"."creator_id"  AS t0_r6, "projects"."namespace_id"  AS t0_r7, "projects"."last_activity_at"  AS t0_r8, "projects"."import_url"  AS t0_r9, "projects"."visibility_level"  AS t0_r10, "projects"."archived"  AS t0_r11, "projects"."merge_requests_template"  AS t0_r12, "projects"."star_count"  AS t0_r13, "projects"."merge_requests_rebase_enabled"  AS t0_r14, "projects"."import_type"  AS t0_r15, "projects"."import_source"  AS t0_r16, "projects"."avatar"  AS t0_r17, "projects"."approvals_before_merge"  AS t0_r18, "projects"."reset_approvals_on_push"  AS t0_r19, "projects"."merge_requests_ff_only_enabled"  AS t0_r20, "projects"."issues_template"  AS t0_r21, "projects"."mirror"  AS t0_r22, "projects"."mirror_user_id"  AS t0_r23, "projects"."shared_runners_enabled"  AS t0_r24, "projects"."runners_token"  AS t0_r25, "... Full query	886,278 342.74/sec 1.00/call 7.34%	53,884.62 ms 20.838 ms/sec 0.061 ms/call 1.36%	873,792 337.91/sec 0.99/call 2.22%	7,930,771 blks 3.07K blks/sec 8.95 blks/call 0.49%	12,668 blks 4.90 blks/sec 0.01 blks/call 0.64%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	407.61 ms 0.158 ms/sec 0.000 ms/call 0.33%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.108)

Start: 2019-09-30T13:06:29.835321+00:00
End: 2019-09-30T13:53:16.626097+00:00
Period seconds: 2806.79078
Period age: 00:46:46.790776

Top 50 rows

#	Query	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	SELECT COUNT(*) FROM "projects"  WHERE "projects"."namespace_id"  IN (WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" INNER JOIN "members"  ON "namespaces"."id" = "members"."source_id"  WHERE "members"."type" IN (?) AND  "members"."source_type" = ? AND  "namespaces"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND (members.access_level  >= ?) UNION SELECT "namespaces".*  FROM "namespaces", "base_and_descendants"  WHERE "namespaces"."type" IN (?)  AND "namespaces"."parent_id" =  "base_and_descendants"."id") SELECT  DISTINCT "namespaces"."id" FROM  "base_and_descendants" AS "namespaces"  WHERE "namespaces"."id" IN (WITH  RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces... Full query	78 0.03/sec 1.00/call 0.00%	739,369.91 ms 263.422 ms/sec 9479.101 ms/call 15.40%	78 0.03/sec 1.00/call 0.00%	595,768,928 blks 212.26K blks/sec 7.64M blks/call 31.78%	120 blks 0.04 blks/sec 1.54 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	3.71 ms 0.001 ms/sec 0.048 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."pending_delete"  = ? ORDER BY "projects"."created_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	42 0.01/sec 1.00/call 0.00%	301,333.12 ms 107.359 ms/sec 7174.598 ms/call 6.27%	1,682 0.60/sec 40.05/call 0.00%	15,320,260 blks 5.46K blks/sec 364.77K blks/call 0.82%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
3	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	48 0.02/sec 1.00/call 0.00%	231,153.46 ms 82.355 ms/sec 4815.697 ms/call 4.81%	4,512 1.61/sec 94.00/call 0.01%	76,708,931 blks 27.33K blks/sec 1.60M blks/call 4.09%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
4	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	815 0.29/sec 1.00/call 0.01%	230,082.57 ms 81.974 ms/sec 282.310 ms/call 4.79%	6,784 2.42/sec 8.32/call 0.02%	151,017,579 blks 53.81K blks/sec 185.30K blks/call 8.06%	186 blks 0.07 blks/sec 0.23 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	6.19 ms 0.002 ms/sec 0.008 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
5	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	799 0.28/sec 1.00/call 0.01%	226,630.57 ms 80.744 ms/sec 283.643 ms/call 4.72%	7,556 2.69/sec 9.46/call 0.02%	148,719,082 blks 52.99K blks/sec 186.14K blks/call 7.93%	188 blks 0.07 blks/sec 0.24 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	5.77 ms 0.002 ms/sec 0.007 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
6	WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN ('Group') AND (EXISTS (SELECT  1 FROM "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN ('silver', 'gold') AND (gitlab_subscriptions.namespace_id  = namespaces.id))) UNION SELECT  "namespaces".* FROM "namespaces",  "base_and_descendants" WHERE "namespaces"."type"  IN ('Group') AND "namespaces"."parent_id"  = "base_and_descendants"."id")  SELECT  "namespaces".* FROM "base_and_descendants"  AS "namespaces" WHERE "namespaces"."id"  = 4526358 LIMIT 1 Full query	34 0.01/sec 1.00/call 0.00%	216,681.89 ms 77.199 ms/sec 6372.997 ms/call 4.51%	4 0.00/sec 0.12/call 0.00%	85,723,474 blks 30.55K blks/sec 2.53M blks/call 4.57%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
7	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	16 0.01/sec 1.00/call 0.00%	211,658.47 ms 75.409 ms/sec 13228.654 ms/call 4.41%	16 0.01/sec 1.00/call 0.00%	79,794,911 blks 28.43K blks/sec 4.99M blks/call 4.26%	6 blks 0.00 blks/sec 0.38 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.09 ms 0.000 ms/sec 0.006 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
8	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	14 0.00/sec 1.00/call 0.00%	189,330.37 ms 67.454 ms/sec 13523.598 ms/call 3.94%	537 0.19/sec 38.36/call 0.00%	69,828,127 blks 24.88K blks/sec 4.99M blks/call 3.73%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
9	SELECT COUNT(*) FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) Full query	44 0.02/sec 1.00/call 0.00%	169,945.59 ms 60.548 ms/sec 3862.400 ms/call 3.54%	44 0.02/sec 1.00/call 0.00%	16,048,355 blks 5.72K blks/sec 364.74K blks/call 0.86%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
10	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	12 0.00/sec 1.00/call 0.00%	164,325.38 ms 58.546 ms/sec 13693.781 ms/call 3.42%	25 0.01/sec 2.08/call 0.00%	59,847,067 blks 21.33K blks/sec 4.99M blks/call 3.19%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
11	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  projects.id FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	28 0.01/sec 1.00/call 0.00%	154,179.31 ms 54.931 ms/sec 5506.404 ms/call 3.21%	118 0.04/sec 4.21/call 0.00%	58,334,685 blks 20.79K blks/sec 2.09M blks/call 3.11%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
12	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	26 0.01/sec 1.00/call 0.00%	123,004.99 ms 43.824 ms/sec 4730.961 ms/call 2.56%	74 0.03/sec 2.85/call 0.00%	48,334,592 blks 17.23K blks/sec 1.86M blks/call 2.58%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
13	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	29 0.01/sec 1.00/call 0.00%	120,496.53 ms 42.930 ms/sec 4155.053 ms/call 2.51%	29 0.01/sec 1.00/call 0.00%	45,157,450 blks 16.09K blks/sec 1.56M blks/call 2.41%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
14	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  "issues"."confidential" = ? AND  (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_level"  = ? AND EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)))) AND ("issues"."title"  ILIKE ? OR "issues"."description"  ILIKE ?) LIMIT ?) subquery_for_count Full query	20 0.01/sec 1.00/call 0.00%	116,600.10 ms 41.542 ms/sec 5830.005 ms/call 2.43%	20 0.01/sec 1.00/call 0.00%	1,895,207 blks 675.22 blks/sec 94.77K blks/call 0.10%	122,831 blks 43.76 blks/sec 6.15K blks/call 5.12%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1,814.17 ms 0.646 ms/sec 90.708 ms/call 1.22%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
15	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM (SELECT "snippets".*  FROM "snippets" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? LIMIT ... Full query	20 0.01/sec 1.00/call 0.00%	94,497.60 ms 33.667 ms/sec 4724.880 ms/call 1.97%	20 0.01/sec 1.00/call 0.00%	7,618,530 blks 2.72K blks/sec 380.93K blks/call 0.41%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
16	SELECT  "snippets".* FROM (SELECT  "snippets".* FROM "snippets" WHERE  (snippets.visibility_level IN ( ?,?,?) OR snippets.author_id =  ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? ORDER BY created_at DESC... Full query	20 0.01/sec 1.00/call 0.00%	93,854.19 ms 33.438 ms/sec 4692.710 ms/call 1.95%	420 0.15/sec 21.00/call 0.00%	7,624,885 blks 2.72K blks/sec 381.25K blks/call 0.41%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
17	SELECT  issues.*, (SELECT MIN("label_priorities"."priority")  FROM "labels" INNER JOIN "label_links"  ON "label_links"."label_id" = "labels"."id"  LEFT OUTER JOIN "label_priorities"  ON "labels"."id" = "label_priorities"."label_id"  WHERE (label_priorities.project_id  = issues.project_id) AND (label_links.target_id  = issues.id) AND "label_links"."target_type"  = ?) AS highest_priority FROM "issues"  WHERE "issues"."project_id" = ?  AND ("issues"."state" IN (?)) GROUP  BY "issues"."id" ORDER BY relative_position  ASC NULLS LAST, highest_priority  ASC NULLS LAST, id DESC LIMIT ?  OFFSET ? Full query	1,252 0.45/sec 1.00/call 0.01%	83,733.51 ms 29.832 ms/sec 66.880 ms/call 1.74%	22,836 8.14/sec 18.24/call 0.06%	61,319,982 blks 21.85K blks/sec 48.98K blks/call 3.27%	25,500 blks 9.09 blks/sec 20.37 blks/call 1.06%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	794.20 ms 0.283 ms/sec 0.634 ms/call 0.53%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
18	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  projects.id FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	14 0.00/sec 1.00/call 0.00%	80,149.58 ms 28.556 ms/sec 5724.970 ms/call 1.67%	5 0.00/sec 0.36/call 0.00%	29,897,902 blks 10.66K blks/sec 2.14M blks/call 1.60%	5 blks 0.00 blks/sec 0.36 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.04 ms 0.000 ms/sec 0.003 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
19	SELECT COUNT(*) FROM "deploy_keys_projects"  WHERE "deploy_keys_projects"."deploy_key_id"  = ? Full query	1,391 0.50/sec 1.00/call 0.01%	64,908.87 ms 23.126 ms/sec 46.663 ms/call 1.35%	1,391 0.50/sec 1.00/call 0.00%	5,088,278 blks 1.82K blks/sec 3.66K blks/call 0.27%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
20	SELECT  "projects"."id" AS t0_r0,  "projects"."name" AS t0_r1, "projects"."path"  AS t0_r2, "projects"."description"  AS t0_r3, "projects"."created_at"  AS t0_r4, "projects"."updated_at"  AS t0_r5, "projects"."creator_id"  AS t0_r6, "projects"."namespace_id"  AS t0_r7, "projects"."last_activity_at"  AS t0_r8, "projects"."import_url"  AS t0_r9, "projects"."visibility_level"  AS t0_r10, "projects"."archived"  AS t0_r11, "projects"."merge_requests_template"  AS t0_r12, "projects"."star_count"  AS t0_r13, "projects"."merge_requests_rebase_enabled"  AS t0_r14, "projects"."import_type"  AS t0_r15, "projects"."import_source"  AS t0_r16, "projects"."avatar"  AS t0_r17, "projects"."approvals_before_merge"  AS t0_r18, "projects"."reset_approvals_on_push"  AS t0_r19, "projects"."merge_requests_ff_only_enabled"  AS t0_r20, "projects"."issues_template"  AS t0_r21, "projects"."mirror"  AS t0_r22, "projects"."mirror_user_id"  AS t0_r23, "projects"."shared_runners_enabled"  AS t0_r24, "projects"."runners_token"  AS t0_r25, "... Full query	963,120 343.14/sec 1.00/call 7.04%	57,831.01 ms 20.604 ms/sec 0.060 ms/call 1.20%	949,749 338.38/sec 0.99/call 2.32%	8,619,618 blks 3.08K blks/sec 8.95 blks/call 0.46%	13,383 blks 4.77 blks/sec 0.01 blks/call 0.56%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	449.00 ms 0.160 ms/sec 0.000 ms/call 0.30%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Replica (10.220.16.109)

Start: 2019-09-30T13:06:31.947164+00:00
End: 2019-09-30T13:57:29.023066+00:00
Period seconds: 3057.07591
Period age: 00:50:57.075902

Top 50 rows

#	Query	Calls	▼ Total time	Rows	shared_blks_hit	shared_blks_read	shared_blks_dirtied	shared_blks_written	blk_read_time	blk_write_time	kcache_reads	kcache_writes	kcache_user_time_ms	kcache_system_time
1	SELECT COUNT(*) FROM "projects"  WHERE "projects"."namespace_id"  IN (WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" INNER JOIN "members"  ON "namespaces"."id" = "members"."source_id"  WHERE "members"."type" IN (?) AND  "members"."source_type" = ? AND  "namespaces"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND (members.access_level  >= ?) UNION SELECT "namespaces".*  FROM "namespaces", "base_and_descendants"  WHERE "namespaces"."type" IN (?)  AND "namespaces"."parent_id" =  "base_and_descendants"."id") SELECT  DISTINCT "namespaces"."id" FROM  "base_and_descendants" AS "namespaces"  WHERE "namespaces"."id" IN (WITH  RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces... Full query	79 0.03/sec 1.00/call 0.00%	714,066.45 ms 233.578 ms/sec 9038.816 ms/call 13.79%	79 0.03/sec 1.00/call 0.00%	651,475,681 blks 213.11K blks/sec 8.25M blks/call 31.84%	96 blks 0.03 blks/sec 1.22 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1.57 ms 0.001 ms/sec 0.020 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
2	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT 1 FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = 4546324 AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (10,20)) AND "projects"."pending_delete"  = FALSE ORDER BY "projects"."created_at"  DESC, "projects"."id" DESC LIMIT  21 OFFSET 0 Full query	58 0.02/sec 1.00/call 0.00%	393,050.74 ms 128.571 ms/sec 6776.737 ms/call 7.59%	1,858 0.61/sec 32.03/call 0.00%	21,156,999 blks 6.93K blks/sec 364.78K blks/call 1.03%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
3	SELECT COUNT(*) FROM (SELECT  1  AS one FROM "projects" WHERE (EXISTS  (SELECT 1 FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = 516057 AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (10,20)) AND "projects"."visibility_level"  = 0 AND "projects"."pending_delete"  = FALSE LIMIT 10001) subquery_for_count Full query	22 0.01/sec 1.00/call 0.00%	263,213.10 ms 86.100 ms/sec 11964.232 ms/call 5.08%	22 0.01/sec 1.00/call 0.00%	109,790,412 blks 35.92K blks/sec 5.00M blks/call 5.37%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
4	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	826 0.27/sec 1.00/call 0.01%	242,547.70 ms 79.340 ms/sec 293.641 ms/call 4.69%	7,876 2.58/sec 9.54/call 0.02%	154,599,498 blks 50.58K blks/sec 187.17K blks/call 7.56%	195 blks 0.06 blks/sec 0.24 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	5.53 ms 0.002 ms/sec 0.007 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
5	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	65 0.02/sec 1.00/call 0.00%	239,255.31 ms 78.263 ms/sec 3680.851 ms/call 4.62%	6,332 2.07/sec 97.42/call 0.01%	93,188,492 blks 30.49K blks/sec 1.44M blks/call 4.55%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
6	SELECT  "projects".* FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ? Full query	20 0.01/sec 1.00/call 0.00%	237,025.00 ms 77.533 ms/sec 11851.250 ms/call 4.58%	891 0.29/sec 44.55/call 0.00%	99,824,763 blks 32.66K blks/sec 5.00M blks/call 4.88%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
7	SELECT "ci_builds".* FROM "ci_builds"  INNER JOIN "projects" ON "projects"."id"  = "ci_builds"."project_id" LEFT  JOIN project_features ON ci_builds.project_id  = project_features.project_id LEFT  JOIN (SELECT "ci_builds"."project_id",  count(*) AS running_builds FROM  "ci_builds" WHERE "ci_builds"."type"  IN (?) AND ("ci_builds"."status"  IN (?)) AND "ci_builds"."runner_id"  IN (SELECT "ci_runners"."id" FROM  "ci_runners" WHERE "ci_runners"."runner_type"  = ?) GROUP BY "ci_builds"."project_id")  AS project_builds ON ci_builds.project_id=project_builds.project_id  WHERE "ci_builds"."type" IN (?)  AND ("ci_builds"."status" IN (?))  AND "ci_builds"."runner_id" IS  NULL AND "projects"."shared_runners_enabled"  = ? AND "projects"."pending_delete"  = ? AND (project_features.builds_access_level  IS NULL or project_features.builds_access_level  > ?) AND (projects.visibility_level=?  OR (WITH RECURSIVE "base_and_ancestors"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE (namespaces.id  = projects.namespace_id) UN... Full query	806 0.26/sec 1.00/call 0.01%	233,503.98 ms 76.381 ms/sec 289.707 ms/call 4.51%	7,193 2.35/sec 8.92/call 0.02%	151,157,820 blks 49.45K blks/sec 187.55K blks/call 7.39%	197 blks 0.06 blks/sec 0.24 blks/call 0.01%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	17.70 ms 0.006 ms/sec 0.022 ms/call 0.01%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
8	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	19 0.01/sec 1.00/call 0.00%	228,489.25 ms 74.741 ms/sec 12025.750 ms/call 4.41%	36 0.01/sec 1.89/call 0.00%	94,828,669 blks 31.02K blks/sec 5.00M blks/call 4.63%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
9	WITH RECURSIVE "base_and_descendants"  AS (SELECT "namespaces".* FROM  "namespaces" WHERE "namespaces"."type"  IN (?) AND (EXISTS (SELECT ? FROM  "plans" INNER JOIN "gitlab_subscriptions"  ON "gitlab_subscriptions"."hosted_plan_id"  = "plans"."id" WHERE "plans"."name"  IN (?, ?) AND (gitlab_subscriptions.namespace_id  = namespaces.id))) UNION SELECT  "namespaces".* FROM "namespaces",  "base_and_descendants" WHERE "namespaces"."type"  IN (?) AND "namespaces"."parent_id"  = "base_and_descendants"."id")  SELECT  "namespaces".* FROM "base_and_descendants"  AS "namespaces" WHERE "namespaces"."id"  = ? LIMIT ? Full query	35 0.01/sec 1.00/call 0.00%	210,728.55 ms 68.931 ms/sec 6020.816 ms/call 4.07%	2 0.00/sec 0.06/call 0.00%	92,300,251 blks 30.20K blks/sec 2.64M blks/call 4.51%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
10	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  projects.id FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	38 0.01/sec 1.00/call 0.00%	171,098.32 ms 55.968 ms/sec 4502.587 ms/call 3.31%	158 0.05/sec 4.16/call 0.00%	74,039,603 blks 24.22K blks/sec 1.95M blks/call 3.62%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
11	SELECT COUNT(*) FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) Full query	46 0.02/sec 1.00/call 0.00%	164,659.41 ms 53.862 ms/sec 3579.552 ms/call 3.18%	46 0.02/sec 1.00/call 0.00%	16,777,978 blks 5.49K blks/sec 364.74K blks/call 0.82%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
12	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  "projects"."namespace_id"  FROM "projects" WHERE (EXISTS ( SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	35 0.01/sec 1.00/call 0.00%	140,012.93 ms 45.800 ms/sec 4000.369 ms/call 2.70%	66 0.02/sec 1.89/call 0.00%	59,094,125 blks 19.34K blks/sec 1.69M blks/call 2.89%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
13	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "projects" WHERE (EXISTS  (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."archived" =  ? AND "projects"."pending_delete"  = ? LIMIT ?) subquery_for_count Full query	38 0.01/sec 1.00/call 0.00%	135,887.22 ms 44.450 ms/sec 3575.979 ms/call 2.63%	38 0.01/sec 1.00/call 0.00%	55,174,542 blks 18.05K blks/sec 1.46M blks/call 2.70%	4 blks 0.00 blks/sec 0.11 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.04 ms 0.000 ms/sec 0.001 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
14	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  "issues"."confidential" = ? AND  (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_level"  = ? AND EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)))) AND ("issues"."title"  ILIKE ? OR "issues"."description"  ILIKE ?) LIMIT ?) subquery_for_count Full query	19 0.01/sec 1.00/call 0.00%	106,930.50 ms 34.978 ms/sec 5627.921 ms/call 2.07%	19 0.01/sec 1.00/call 0.00%	1,627,822 blks 532.48 blks/sec 85.68K blks/call 0.08%	69,626 blks 22.78 blks/sec 3.67K blks/call 2.75%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	1,268.92 ms 0.415 ms/sec 66.786 ms/call 0.93%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
15	SELECT  "snippets".* FROM (SELECT  "snippets".* FROM "snippets" WHERE  (snippets.visibility_level IN ( ?,?,?) OR snippets.author_id =  ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? ORDER BY created_at DESC... Full query	19 0.01/sec 1.00/call 0.00%	90,067.75 ms 29.462 ms/sec 4740.408 ms/call 1.74%	399 0.13/sec 21.00/call 0.00%	7,245,740 blks 2.38K blks/sec 381.36K blks/call 0.35%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
16	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM (SELECT "snippets".*  FROM "snippets" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND "snippets"."project_id"  IS NULL UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE (snippets.visibility_level  IN (?,?,?) OR snippets.author_id  = ?) AND (projects.visibility_level  IN (?,?,?)) AND "project_features"."snippets_access_level"  IN (?, ?) UNION SELECT "snippets".*  FROM "snippets" INNER JOIN "projects"  ON "projects"."id" = "snippets"."project_id"  INNER JOIN "project_features" ON  "project_features"."project_id"  = "projects"."id" WHERE "project_features"."snippets_access_level"  IN (?, ?, ?) AND (EXISTS (SELECT  ? FROM "project_authorizations"  WHERE (project_id = snippets.project_id)  AND "project_authorizations"."user_id"  = ?))) snippets WHERE "snippets"."visibility_level"  = ? LIMIT ... Full query	19 0.01/sec 1.00/call 0.00%	89,030.17 ms 29.123 ms/sec 4685.798 ms/call 1.72%	19 0.01/sec 1.00/call 0.00%	7,239,274 blks 2.37K blks/sec 381.02K blks/call 0.35%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
17	SELECT "members".* FROM "members"  WHERE "members"."source_type" =  ? AND "members"."type" IN (?) AND  "members"."user_id" = ? AND "members"."requested_at"  IS NULL AND "members"."source_id"  IN (SELECT  projects.id FROM "projects"  WHERE (EXISTS (SELECT ? FROM "project_authorizations"  WHERE "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id)) OR projects.visibility_level  IN (?,?)) AND "projects"."visibility_level"  = ? AND "projects"."pending_delete"  = ? ORDER BY "projects"."last_activity_at"  DESC, "projects"."id" DESC LIMIT  ? OFFSET ?) Full query	20 0.01/sec 1.00/call 0.00%	83,211.88 ms 27.219 ms/sec 4160.594 ms/call 1.61%	7 0.00/sec 0.35/call 0.00%	34,908,693 blks 11.42K blks/sec 1.75M blks/call 1.71%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
18	SELECT  "deployments".* FROM "deployments"  WHERE "deployments"."environment_id"  = ? AND "deployments"."status"  = ? ORDER BY deployments.id DESC  LIMIT ? Full query	16,848 5.51/sec 1.00/call 0.12%	82,259.54 ms 26.908 ms/sec 4.882 ms/call 1.59%	14,159 4.63/sec 0.84/call 0.03%	28,131,034 blks 9.21K blks/sec 1.67K blks/call 1.37%	15,257 blks 4.99 blks/sec 0.91 blks/call 0.60%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	559.00 ms 0.183 ms/sec 0.033 ms/call 0.41%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
19	SELECT COUNT(*) FROM "deploy_keys_projects"  WHERE "deploy_keys_projects"."deploy_key_id"  = ? Full query	1,632 0.53/sec 1.00/call 0.01%	80,753.81 ms 26.415 ms/sec 49.481 ms/call 1.56%	1,632 0.53/sec 1.00/call 0.00%	5,969,856 blks 1.96K blks/sec 3.66K blks/call 0.29%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%
20	SELECT COUNT(*) FROM (SELECT  ?  AS one FROM "issues" INNER JOIN  "projects" ON "projects"."id" =  "issues"."project_id" LEFT JOIN  project_features ON projects.id  = project_features.project_id WHERE  (       issues.confidential IS  NOT TRUE       OR (issues.confidential  = ?         AND (issues.author_id  = ?           OR EXISTS (SELECT  ? FROM issue_assignees WHERE user_id  = ? AND issue_id = issues.id)          OR EXISTS (SELECT ? FROM  "project_authorizations" WHERE  "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = issues.project_id) AND (project_authorizations.access_level  >= ?))))) AND (EXISTS (SELECT ?  FROM "project_authorizations" WHERE  "project_authorizations"."user_id"  = ? AND (project_authorizations.project_id  = projects.id) AND (project_authorizations.access_level  >= ?)) OR projects.visibility_level  IN (?,?)) AND ("project_features"."issues_access_level"  IS NULL OR "project_features"."issues_access_level"  IN (?,?) OR ("project_features"."issues_access_l... Full query	12 0.00/sec 1.00/call 0.00%	72,963.58 ms 23.867 ms/sec 6080.298 ms/call 1.41%	12 0.00/sec 1.00/call 0.00%	1,197,576 blks 391.74 blks/sec 99.80K blks/call 0.06%	8 blks 0.00 blks/sec 0.67 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0 blks 0.00 blks/sec 0.00 blks/call 0.00%	0.10 ms 0.000 ms/sec 0.009 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 bytes 0.00 bytes/sec 0.00 bytes/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%	0.00 ms 0.000 ms/sec 0.000 ms/call 0.00%

Conclusions

Recommendations

All good, no recommendations here.

---

  Table of contents

L001 Table Sizes

Observations

Data collected: 2019-09-30 13:38:34 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.101)

The list is limited to 20 items. Total: 322.

#	Table	Rows	▼ Total size	Table size	Index(es) Size	TOAST Size
	===== TOTAL =====	~8B	3.99 TiB	2.01 TiB	1.21 TiB	794.43 GiB
1	merge_request_diff_files	~915M	1.28 TiB (32.03%)	664.07 GiB (32.31%)	23.68 GiB (1.93%)	618.78 GiB (77.89%)
2	ci_builds	~292M	708.74 GiB (17.37%)	291.64 GiB (14.19%)	305.64 GiB (24.86%)	111.47 GiB (14.03%)
3	merge_request_diff_commits	~2B	634.04 GiB (15.54%)	488.89 GiB (23.78%)	131.88 GiB (10.73%)	13.28 GiB (1.67%)
4	notes	~191M	245.66 GiB (6.02%)	122.37 GiB (5.95%)	115.37 GiB (9.38%)	7.92 GiB (1.00%)
5	ci_build_trace_sections	~2B	218.94 GiB (5.37%)	94.19 GiB (4.58%)	124.76 GiB (10.15%)
6	events	~439M	111.71 GiB (2.74%)	28.76 GiB (1.40%)	82.96 GiB (6.75%)	8.00 KiB (0.00%)
7	web_hook_logs	~43M	83.81 GiB (2.05%)	50.77 GiB (2.47%)	3.55 GiB (0.29%)	29.50 GiB (3.71%)
8	ci_job_artifacts	~245M	80.47 GiB (1.97%)	36.70 GiB (1.79%)	43.77 GiB (3.56%)	8.00 KiB (0.00%)
9	merge_requests	~31M	60.31 GiB (1.48%)	13.42 GiB (0.65%)	45.82 GiB (3.73%)	1.08 GiB (0.14%)
10	audit_events	~187M	58.39 GiB (1.43%)	42.53 GiB (2.07%)	15.86 GiB (1.29%)	6.27 MiB (0.00%)
11	ci_pipelines	~80M	58.24 GiB (1.43%)	16.48 GiB (0.80%)	41.74 GiB (3.39%)	33.58 MiB (0.00%)
12	issues	~20M	52.91 GiB (1.30%)	12.56 GiB (0.61%)	35.03 GiB (2.85%)	5.33 GiB (0.67%)
13	sent_notifications	~209M	49.58 GiB (1.22%)	30.81 GiB (1.50%)	18.77 GiB (1.53%)	8.00 KiB (0.00%)
14	projects	~11M	48.97 GiB (1.20%)	2.79 GiB (0.14%)	45.92 GiB (3.73%)	278.35 MiB (0.03%)
15	push_event_payloads	~332M	46.63 GiB (1.14%)	39.53 GiB (1.92%)	7.11 GiB (0.58%)	8.00 KiB (0.00%)
16	taggings	~117M	30.62 GiB (0.75%)	7.46 GiB (0.36%)	23.16 GiB (1.88%)	8.00 KiB (0.00%)
17	ci_stages	~143M	30.54 GiB (0.75%)	11.70 GiB (0.57%)	18.85 GiB (1.53%)	8.00 KiB (0.00%)
18	ci_builds_metadata	~225M	27.68 GiB (0.68%)	11.06 GiB (0.54%)	16.63 GiB (1.35%)	8.00 KiB (0.00%)
19	users	~5M	21.17 GiB (0.52%)	1.96 GiB (0.09%)	19.22 GiB (1.56%)	128.00 KiB (0.00%)
20	deployments	~31M	17.12 GiB (0.42%)	4.92 GiB (0.24%)	12.20 GiB (0.99%)	8.00 KiB (0.00%)

Conclusions

Recommendations

---

  Table of contents

L003 Integer (int2, int4) Out-of-range Risks in PKs

Observations

Data collected: 2019-09-30 13:38:37 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.101)

Table	PK	Type	Current max value	▼ Capacity used, %
ci_build_trace_sections	id	int4	1,353,474,139	63.03
events	id	int4	520,890,475	24.26
ci_job_artifacts	id	int4	318,448,365	14.83
ci_builds	id	int4	307,846,655	14.34
ci_builds_metadata	id	int4	234,989,880	10.94
notes	id	int4	223,917,139	10.43

Conclusions

• [P1] High risks of out-of-range errors for an integer column. The columns listed below, being part of a primary key, have high risks to reach 100% of the integer capacity (2^31-1, or 2147483647 for int4 columns, and 2^15-1, or 32767 for int2 columns; see the documentation. If it happens, INSERTs of new rows will not be possible (unless they use some non-incremental values, such as some negative values) and fixing it will require a long downtime. 6 such columns are found:
  • ci_builds: reached value 307846655, or 14.34% of int4 capacity
  • ci_builds_metadata: reached value 234989880, or 10.94% of int4 capacity
  • notes: reached value 223917139, or 10.43% of int4 capacity
  • ci_build_trace_sections: reached value 1353474139, or 63.03% of int4 capacity
  • events: reached value 520890475, or 24.26% of int4 capacity
  • etc.

Recommendations

• [P1] High risks of out-of-range errors for an integer column. Consider using int8 in all PK columns, always. To convert existing columns to int8, consider the following approaches:
  1. Blocking ALTER TABLE .. ALTER COLUMN: a straightforward solution requiring significant downtime (a maintenance window).
  2. "New column": create a new column, update it in batches (running not longer than a few seconds, to avoid blocking issues), and then switch to using it, redefining all the constraints. Notice, that to redefine a primary key constraint, ALTER TABLE .. ALTER COLUMN .. SET NOT NULL will be needed. It is a blocking operation in all Postgres versions up to 12 (where it might be lightweight if a proper CHECK constraint is defined first; such constraint can be defined in a non-blocking way). Since Postgres 11, it is possible to use a trick: when adding a column, use DEFAULT with NOT NULL, it will be a non-blocking operation. For all Postgres versions prior to 11, a specific downtime (maintenance window) will be needed anyway.
  3. "New table": create a new table with the same schema as the existing one, capture all ongoing changes to an additional "log" table, copy existing data from the old table to the new one, and switch. This method, as the previous one, is non-trivial and requires careful development and testing under load (consider using Nancy for database experiments developing this solution). This approach is non-blocking regardless of Postgres version, but it requires significantly more efforts to implement.

added Database Workload analysis postgres-checkup labels

Summary:

1. DB size reached 4 TiB.
2. ci_build_trace_sections.id exceeded 64% of int4 capacity.
3. Table and index bloat require attention (created https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/7931).
4. 64 unused indexes, the total size of which is 142.08 GiB.
5. Top-1 queries on replicas take only <20% of total_time, which is good, no obvious huge resource consumers But those queries are still suboptimal, optimization is needed.
6. On replicas: SELECT COUNT(*) FROM "projects" WHERE "projects"."namespace_id" IN where VERY slow, ~9 seconds on average, attention is needed /cc @gl-database
7. On the master: SELECT COUNT(*) FROM "projects" WHERE (EXISTS (SELECT ? FROM "project_authorizations" is also very slow, ~3 seconds on average.
8. On the master: the second (by total_time) query group, is also quite slow ~300ms, and it's quite frequent, ~20 times per second. Most likely, locking issues -- redesign is needed (relying on SELECT .. FOR UPDATE SKIP LOCKED or NOWAIT with retries). At some hours, this query exceeds 20% of overall total_time, popping up to the first place on the master.

mentioned in issue gitlab-org/gitlab#23924 (closed)

mentioned in issue gitlab-org/database-team/team-tasks#7 (closed)

I created gitlab-org/database-team/team-tasks#7 (closed) to look into (8).

Is there a way to look at the full report? I'm interested in the full list of unused indices.

mentioned in issue gitlab-org/quality/performance#66 (closed)

closed

added workflow-infraDone label