Skip to content

postgres-checkup report for gitlab_production: 2020-03-09 (auto-generated)

PostgreSQL Checkup. Project: 'gitlab_production'. Database: 'gitlabhq_production'

Epoch number: '20200309001'

NOTICE: while most reports describe the “current database”, some of them may contain cluster-wide information describing all databases in the cluster.

Last modified at: 2020-03-09 20:13:50 +0000

 

Table of contents

A002 Version Information
A003 Postgres Settings
A004 Cluster Information
A005 Extensions
A006 Postgres Setting Deviations
A007 Altered Settings
D004 pg_stat_statements and pg_stat_kcache Settings
F001 Autovacuum: Current Settings
F002 Autovacuum: Transaction ID Wraparound Check
F003 Autovacuum: Dead Tuples
F004 Autovacuum: Heap Bloat (Estimated)
F005 Autovacuum: Btree Index Bloat (Estimated)
F008 Autovacuum: Resource Usage
G001 Memory-related Settings
G002 Connections and Current Activity
G003 Timeouts, Locks, Deadlocks
H001 Invalid Indexes
H002 Unused Indexes
H003 Non-indexed Foreign Keys
H004 Redundant Indexes
K001 Globally Aggregated Query Metrics
K002 Workload Type ("The First Word" Analysis)
K003 Top-20 Queries by total_time
L001 Table Sizes
L003 Integer (int2, int4) Out-of-range Risks in PKs

Issues found

The empty lines represent reports for which Conclusions and Recommendations are not yet implemented.

Report P1 P2 P3
A002 Version Information 0 2 1
A003 Postgres Settings
A004 Cluster Information
A005 Extensions
A006 Postgres Setting Deviations
A007 Altered Settings
D004 pg_stat_statements and pg_stat_kcache Settings
F001 Autovacuum: Current Settings 0 0 0
F002 Autovacuum: Transaction ID Wraparound Check 0 0 0
F003 Autovacuum: Dead Tuples
F004 Autovacuum: Heap Bloat (Estimated) 1 0 0
F005 Autovacuum: Btree Index Bloat (Estimated) 1 0 0
F008 Autovacuum: Resource Usage
G001 Memory-related Settings
G002 Connections and Current Activity 0 0 0
G003 Timeouts, Locks, Deadlocks
H001 Invalid Indexes
H002 Unused Indexes 0 0 1
H003 Non-indexed Foreign Keys
H004 Redundant Indexes 0 0 1
K001 Globally Aggregated Query Metrics
K002 Workload Type ("The First Word" Analysis)
K003 Top-20 Queries by total_time 1 0 0
L001 Table Sizes
L003 Integer (int2, int4) Out-of-range Risks in PKs 1 0 0

  Table of contents

A002 Version Information

Observations

Data collected: 2020-03-09 20:07:24 +0000 UTC

Master (10.220.16.106)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica servers:

Replica (10.220.16.101)

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu (Ubuntu 9.6.15-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.102)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.103)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.104)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.105)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.107)

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu (Ubuntu 9.6.15-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.108)

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu (Ubuntu 9.6.15-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.109)

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu (Ubuntu 9.6.15-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Conclusions

  • [P2] Postgres minor version on the master and replica(s) differ. Nodes 10.220.16.102, 10.220.16.107, 10.220.16.109, 10.220.16.101, 10.220.16.103, 10.220.16.104, 10.220.16.105, 10.220.16.106, 10.220.16.108 use Postgres 9.6.14, 9.6.15.

  • Postgres major version being used is 9.6 and it is currently supported by PostgreSQL Global Development Group. End of life is scheduled 2021-11-11. It means that in case of bugs and security issues, updates (new minor versions) with fixes will be released and available for use. Read more: Versioning Policy.

  • [P2] The minor versions being used (9.6.14, 9.6.15) are not up-to-date (the newest version: 9.6.17). See the full list of changes between 9.6.14 and 9.6.17 and between 9.6.15 and 9.6.17.

Recommendations

  Table of contents

A003 Postgres Settings

Observations

Data collected: 2020-03-09 20:07:25 +0000 UTC

Master (10.220.16.106)

▼ Category Setting Value Unit Pretty value
Autovacuum autovacuum on
Autovacuum autovacuum_analyze_scale_factor 0.005
Autovacuum autovacuum_analyze_threshold 50
Autovacuum autovacuum_freeze_max_age 200000000
Autovacuum autovacuum_max_workers 6
Autovacuum autovacuum_multixact_freeze_max_age 400000000
Autovacuum autovacuum_naptime 30 s
Autovacuum autovacuum_vacuum_cost_delay 5 ms
Autovacuum autovacuum_vacuum_cost_limit 6000
Autovacuum autovacuum_vacuum_scale_factor 0.005
Autovacuum autovacuum_vacuum_threshold 50
Client Connection Defaults / Locale and Formatting DateStyle ISO, MDY
Client Connection Defaults / Locale and Formatting IntervalStyle postgres
Client Connection Defaults / Locale and Formatting TimeZone GMT
Client Connection Defaults / Locale and Formatting client_encoding UTF8
Client Connection Defaults / Locale and Formatting default_text_search_config pg_catalog.english
Client Connection Defaults / Locale and Formatting extra_float_digits 0
Client Connection Defaults / Locale and Formatting lc_collate C.UTF-8
Client Connection Defaults / Locale and Formatting lc_ctype C.UTF-8
Client Connection Defaults / Locale and Formatting lc_messages C
Client Connection Defaults / Locale and Formatting lc_monetary C
Client Connection Defaults / Locale and Formatting lc_numeric C
Client Connection Defaults / Locale and Formatting lc_time C
Client Connection Defaults / Locale and Formatting server_encoding UTF8
Client Connection Defaults / Locale and Formatting timezone_abbreviations Default
Client Connection Defaults / Other Defaults dynamic_library_path $libdir
Client Connection Defaults / Other Defaults gin_fuzzy_search_limit 0
Client Connection Defaults / Other Defaults tcp_keepalives_count 9
Client Connection Defaults / Other Defaults tcp_keepalives_idle 300 s
Client Connection Defaults / Other Defaults tcp_keepalives_interval 75 s
Client Connection Defaults / Shared Library Preloading local_preload_libraries
Client Connection Defaults / Shared Library Preloading session_preload_libraries
Client Connection Defaults / Shared Library Preloading shared_preload_libraries pg_stat_statements
Client Connection Defaults / Statement Behavior bytea_output hex
Client Connection Defaults / Statement Behavior check_function_bodies on
Client Connection Defaults / Statement Behavior client_min_messages notice
Client Connection Defaults / Statement Behavior default_tablespace
Client Connection Defaults / Statement Behavior default_transaction_deferrable off
Client Connection Defaults / Statement Behavior default_transaction_isolation read committed
Client Connection Defaults / Statement Behavior default_transaction_read_only off
Client Connection Defaults / Statement Behavior gin_pending_list_limit 4096 kB 4.00 MiB
Client Connection Defaults / Statement Behavior idle_in_transaction_session_timeout 30000 ms
Client Connection Defaults / Statement Behavior lock_timeout 0 ms
Client Connection Defaults / Statement Behavior search_path "$user", public
Client Connection Defaults / Statement Behavior session_replication_role origin
Client Connection Defaults / Statement Behavior statement_timeout 15000 ms
Client Connection Defaults / Statement Behavior temp_tablespaces
Client Connection Defaults / Statement Behavior transaction_deferrable off
Client Connection Defaults / Statement Behavior transaction_isolation read committed
Client Connection Defaults / Statement Behavior transaction_read_only off
Client Connection Defaults / Statement Behavior vacuum_freeze_min_age 50000000
Client Connection Defaults / Statement Behavior vacuum_freeze_table_age 150000000
Client Connection Defaults / Statement Behavior vacuum_multixact_freeze_min_age 5000000
Client Connection Defaults / Statement Behavior vacuum_multixact_freeze_table_age 150000000
Client Connection Defaults / Statement Behavior xmlbinary base64
Client Connection Defaults / Statement Behavior xmloption content
Connections and Authentication / Connection Settings bonjour off
Connections and Authentication / Connection Settings bonjour_name
Connections and Authentication / Connection Settings listen_addresses 0.0.0.0
Connections and Authentication / Connection Settings max_connections 300
Connections and Authentication / Connection Settings port 5432
Connections and Authentication / Connection Settings superuser_reserved_connections 3
Connections and Authentication / Connection Settings unix_socket_directories /var/opt/gitlab/postgresql
Connections and Authentication / Connection Settings unix_socket_group
Connections and Authentication / Connection Settings unix_socket_permissions 0777
Connections and Authentication / Security and Authentication authentication_timeout 60 s
Connections and Authentication / Security and Authentication db_user_namespace off
Connections and Authentication / Security and Authentication krb_caseins_users off
Connections and Authentication / Security and Authentication krb_server_keyfile FILE:/etc/postgresql-common/krb5.keytab
Connections and Authentication / Security and Authentication password_encryption on
Connections and Authentication / Security and Authentication row_security on
Connections and Authentication / Security and Authentication ssl on
Connections and Authentication / Security and Authentication ssl_ca_file
Connections and Authentication / Security and Authentication ssl_cert_file /var/opt/gitlab/postgresql/server.crt
Connections and Authentication / Security and Authentication ssl_ciphers HIGH:MEDIUM:+3DES:!aNULL:!SSLv3:!TLSv1
Connections and Authentication / Security and Authentication ssl_crl_file
Connections and Authentication / Security and Authentication ssl_ecdh_curve prime256v1
Connections and Authentication / Security and Authentication ssl_key_file /var/opt/gitlab/postgresql/server.key
Connections and Authentication / Security and Authentication ssl_prefer_server_ciphers on
Customized Options pg_stat_statements.max 5000
Customized Options pg_stat_statements.save on
Customized Options pg_stat_statements.track top
Customized Options pg_stat_statements.track_utility on
Developer Options allow_system_table_mods off
Developer Options ignore_checksum_failure off
Developer Options ignore_system_indexes off
Developer Options post_auth_delay 0 s
Developer Options pre_auth_delay 0 s
Developer Options trace_notify off
Developer Options trace_recovery_messages log
Developer Options trace_sort off
Developer Options zero_damaged_pages off
Error Handling data_sync_retry off
Error Handling exit_on_error off
Error Handling restart_after_crash on
File Locations config_file /var/opt/gitlab/postgresql/postgresql.conf
File Locations data_directory /var/opt/gitlab/postgresql/data
File Locations external_pid_file
File Locations hba_file /var/opt/gitlab/postgresql/data/pg_hba.conf
File Locations ident_file /var/opt/gitlab/postgresql/pg_ident.conf
Lock Management deadlock_timeout 5000 ms
Lock Management max_locks_per_transaction 128
Lock Management max_pred_locks_per_transaction 64
Preset Options block_size 8192
Preset Options data_checksums off
Preset Options debug_assertions off
Preset Options integer_datetimes on
Preset Options max_function_args 100
Preset Options max_identifier_length 63
Preset Options max_index_keys 32
Preset Options segment_size 131072 8kB 1.00 GiB
Preset Options server_version 9.6.14
Preset Options server_version_num 90614
Preset Options wal_block_size 8192
Preset Options wal_segment_size 2048 8kB 16.00 MiB
Process Title cluster_name pg-ha-cluster
Process Title update_process_title on
Query Tuning / Genetic Query Optimizer geqo on
Query Tuning / Genetic Query Optimizer geqo_effort 5
Query Tuning / Genetic Query Optimizer geqo_generations 0
Query Tuning / Genetic Query Optimizer geqo_pool_size 0
Query Tuning / Genetic Query Optimizer geqo_seed 0
Query Tuning / Genetic Query Optimizer geqo_selection_bias 2
Query Tuning / Genetic Query Optimizer geqo_threshold 12
Query Tuning / Other Planner Options constraint_exclusion partition
Query Tuning / Other Planner Options cursor_tuple_fraction 0.1
Query Tuning / Other Planner Options default_statistics_target 1000
Query Tuning / Other Planner Options force_parallel_mode off
Query Tuning / Other Planner Options from_collapse_limit 8
Query Tuning / Other Planner Options join_collapse_limit 8
Query Tuning / Planner Cost Constants cpu_index_tuple_cost 0.005
Query Tuning / Planner Cost Constants cpu_operator_cost 0.0025
Query Tuning / Planner Cost Constants cpu_tuple_cost 0.01
Query Tuning / Planner Cost Constants effective_cache_size 43352064 8kB 330.75 GiB
Query Tuning / Planner Cost Constants min_parallel_relation_size 1024 8kB 8.00 MiB
Query Tuning / Planner Cost Constants parallel_setup_cost 1000
Query Tuning / Planner Cost Constants parallel_tuple_cost 0.1
Query Tuning / Planner Cost Constants random_page_cost 1.5
Query Tuning / Planner Cost Constants seq_page_cost 4
Query Tuning / Planner Method Configuration enable_bitmapscan on
Query Tuning / Planner Method Configuration enable_hashagg on
Query Tuning / Planner Method Configuration enable_hashjoin on
Query Tuning / Planner Method Configuration enable_indexonlyscan on
Query Tuning / Planner Method Configuration enable_indexscan on
Query Tuning / Planner Method Configuration enable_material on
Query Tuning / Planner Method Configuration enable_mergejoin on
Query Tuning / Planner Method Configuration enable_nestloop on
Query Tuning / Planner Method Configuration enable_seqscan on
Query Tuning / Planner Method Configuration enable_sort on
Query Tuning / Planner Method Configuration enable_tidscan on
Replication track_commit_timestamp off
Replication / Master Server synchronous_standby_names
Replication / Master Server vacuum_defer_cleanup_age 0
Replication / Sending Servers max_replication_slots 32
Replication / Sending Servers max_wal_senders 32
Replication / Sending Servers wal_keep_segments 2048
Replication / Sending Servers wal_sender_timeout 60000 ms
Replication / Standby Servers hot_standby on
Replication / Standby Servers hot_standby_feedback on
Replication / Standby Servers max_standby_archive_delay 30000 ms
Replication / Standby Servers max_standby_streaming_delay 30000 ms
Replication / Standby Servers wal_receiver_status_interval 10 s
Replication / Standby Servers wal_receiver_timeout 60000 ms
Replication / Standby Servers wal_retrieve_retry_interval 5000 ms
Reporting and Logging / What to Log application_name checkup
Reporting and Logging / What to Log debug_pretty_print on
Reporting and Logging / What to Log debug_print_parse off
Reporting and Logging / What to Log debug_print_plan off
Reporting and Logging / What to Log debug_print_rewritten off
Reporting and Logging / What to Log log_autovacuum_min_duration 0 ms
Reporting and Logging / What to Log log_checkpoints on
Reporting and Logging / What to Log log_connections on
Reporting and Logging / What to Log log_disconnections on
Reporting and Logging / What to Log log_duration off
Reporting and Logging / What to Log log_error_verbosity default
Reporting and Logging / What to Log log_hostname off
Reporting and Logging / What to Log log_line_prefix %m [%p, %x]: [%l-1] user=%u, db=%d, app=%a, client=%h
Reporting and Logging / What to Log log_lock_waits on
Reporting and Logging / What to Log log_replication_commands off
Reporting and Logging / What to Log log_statement ddl
Reporting and Logging / What to Log log_temp_files 0 kB 0.00 bytes
Reporting and Logging / What to Log log_timezone GMT
Reporting and Logging / When to Log log_min_duration_statement 1000 ms
Reporting and Logging / When to Log log_min_error_statement error
Reporting and Logging / When to Log log_min_messages warning
Reporting and Logging / Where to Log event_source PostgreSQL
Reporting and Logging / Where to Log log_destination csvlog
Reporting and Logging / Where to Log log_directory /var/log/gitlab/postgresql
Reporting and Logging / Where to Log log_file_mode 0640
Reporting and Logging / Where to Log log_filename postgresql.log
Reporting and Logging / Where to Log log_rotation_age 0 min
Reporting and Logging / Where to Log log_rotation_size 0 kB 0.00 bytes
Reporting and Logging / Where to Log log_truncate_on_rotation off
Reporting and Logging / Where to Log logging_collector on
Reporting and Logging / Where to Log syslog_facility local0
Reporting and Logging / Where to Log syslog_ident postgres
Reporting and Logging / Where to Log syslog_sequence_numbers on
Reporting and Logging / Where to Log syslog_split_messages on
Resource Usage / Asynchronous Behavior backend_flush_after 0 8kB 0.00 bytes
Resource Usage / Asynchronous Behavior effective_io_concurrency 100
Resource Usage / Asynchronous Behavior max_parallel_workers_per_gather 0
Resource Usage / Asynchronous Behavior max_worker_processes 8
Resource Usage / Asynchronous Behavior old_snapshot_threshold -1 min
Resource Usage / Background Writer bgwriter_delay 200 ms
Resource Usage / Background Writer bgwriter_flush_after 64 8kB 512.00 KiB
Resource Usage / Background Writer bgwriter_lru_maxpages 100
Resource Usage / Background Writer bgwriter_lru_multiplier 2
Resource Usage / Cost-Based Vacuum Delay vacuum_cost_delay 0 ms
Resource Usage / Cost-Based Vacuum Delay vacuum_cost_limit 200
Resource Usage / Cost-Based Vacuum Delay vacuum_cost_page_dirty 20
Resource Usage / Cost-Based Vacuum Delay vacuum_cost_page_hit 1
Resource Usage / Cost-Based Vacuum Delay vacuum_cost_page_miss 10
Resource Usage / Disk temp_file_limit -1 kB
Resource Usage / Kernel Resources max_files_per_process 1000
Resource Usage / Memory autovacuum_work_mem -1 kB
Resource Usage / Memory dynamic_shared_memory_type posix
Resource Usage / Memory huge_pages try
Resource Usage / Memory maintenance_work_mem 2097152 kB 2.00 GiB
Resource Usage / Memory max_prepared_transactions 0
Resource Usage / Memory max_stack_depth 2048 kB 2.00 MiB
Resource Usage / Memory replacement_sort_tuples 150000
Resource Usage / Memory shared_buffers 14450688 8kB 110.25 GiB
Resource Usage / Memory temp_buffers 1024 8kB 8.00 MiB
Resource Usage / Memory track_activity_query_size 1024
Resource Usage / Memory work_mem 102400 kB 100.00 MiB
Statistics / Monitoring log_executor_stats off
Statistics / Monitoring log_parser_stats off
Statistics / Monitoring log_planner_stats off
Statistics / Monitoring log_statement_stats off
Statistics / Query and Index Statistics Collector stats_temp_directory pg_stat_tmp
Statistics / Query and Index Statistics Collector track_activities on
Statistics / Query and Index Statistics Collector track_counts on
Statistics / Query and Index Statistics Collector track_functions none
Statistics / Query and Index Statistics Collector track_io_timing on
Version and Platform Compatibility / Other Platforms and Clients transform_null_equals off
Version and Platform Compatibility / Previous PostgreSQL Versions array_nulls on
Version and Platform Compatibility / Previous PostgreSQL Versions backslash_quote safe_encoding
Version and Platform Compatibility / Previous PostgreSQL Versions default_with_oids off
Version and Platform Compatibility / Previous PostgreSQL Versions escape_string_warning on
Version and Platform Compatibility / Previous PostgreSQL Versions lo_compat_privileges off
Version and Platform Compatibility / Previous PostgreSQL Versions operator_precedence_warning off
Version and Platform Compatibility / Previous PostgreSQL Versions quote_all_identifiers off
Version and Platform Compatibility / Previous PostgreSQL Versions sql_inheritance on
Version and Platform Compatibility / Previous PostgreSQL Versions standard_conforming_strings on
Version and Platform Compatibility / Previous PostgreSQL Versions synchronize_seqscans on
Write-Ahead Log / Archiving archive_command /usr/bin/envdir /etc/wal-e.d/env /opt/wal-e/bin/wal-e wal-push %p
Write-Ahead Log / Archiving archive_mode on
Write-Ahead Log / Archiving archive_timeout 60 s
Write-Ahead Log / Checkpoints checkpoint_completion_target 0.7
Write-Ahead Log / Checkpoints checkpoint_flush_after 32 8kB 256.00 KiB
Write-Ahead Log / Checkpoints checkpoint_timeout 300 s
Write-Ahead Log / Checkpoints checkpoint_warning 30 s
Write-Ahead Log / Checkpoints max_wal_size 320 16MB 5.00 GiB
Write-Ahead Log / Checkpoints min_wal_size 5 16MB 80.00 MiB
Write-Ahead Log / Settings commit_delay 0
Write-Ahead Log / Settings commit_siblings 5
Write-Ahead Log / Settings fsync on
Write-Ahead Log / Settings full_page_writes on
Write-Ahead Log / Settings synchronous_commit on
Write-Ahead Log / Settings wal_buffers 2048 8kB 16.00 MiB
Write-Ahead Log / Settings wal_compression off
Write-Ahead Log / Settings wal_level replica
Write-Ahead Log / Settings wal_log_hints on
Write-Ahead Log / Settings wal_sync_method fdatasync
Write-Ahead Log / Settings wal_writer_delay 200 ms
Write-Ahead Log / Settings wal_writer_flush_after 128 8kB 1.00 MiB

  Table of contents

A004 Cluster Information

Observations

Data collected: 2020-03-09 20:07:26 +0000 UTC

▼ Indicator 10.220.16.106 10.220.16.101 10.220.16.102 10.220.16.103 10.220.16.104 10.220.16.105 10.220.16.107 10.220.16.108 10.220.16.109
Config file /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf
Role Master Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false)
Replicas async/streaming: 10.220.16.102, 10.220.16.104, 10.220.16.101, 10.220.16.110, 10.220.16.103, 10.220.16.105, 10.220.16.111, 10.220.16.108, 10.220.16.109, 10.220.16.112, 10.220.16.107
Started At 2019-10-28 16:07:17+00 2019-12-12 07:32:00+00 2019-12-12 07:32:03+00 2019-12-12 07:32:05+00 2019-12-12 07:32:03+00 2019-12-12 07:32:09+00 2020-03-09 16:55:48+00 2019-12-12 07:32:09+00 2019-12-12 07:32:10+00
Uptime 133 days 03:58:16 88 days 12:31:14 88 days 12:31:37 88 days 12:32:04 88 days 12:32:33 88 days 12:32:55 03:10:41 88 days 12:34:48 88 days 12:35:17
Checkpoints 87107 1646277 1670884 1658425 1656328 1665642 4055 1635585 1646341
Forced Checkpoints 82.2% 99.8% 99.8% 99.8% 99.8% 99.8% 100.0% 99.8% 99.8%
Checkpoint MB/sec 14.550354 17.724831 17.727677 17.725036 17.725520 17.729102 28.725824 17.722274 17.724211
Database Name gitlabhq_production gitlabhq_production gitlabhq_production gitlabhq_production gitlabhq_production gitlabhq_production gitlabhq_production gitlabhq_production gitlabhq_production
Database Size 5587 GB 5587 GB 5587 GB 5587 GB 5587 GB 5587 GB 5587 GB 5587 GB 5587 GB
Stats Since 2019-10-28 16:07:35+00 2019-12-12 07:32:12+00 2019-12-12 07:32:11+00 2019-12-12 07:32:13+00 2019-12-12 07:32:12+00 2019-12-12 07:32:20+00 2020-03-09 16:56:18+00 2019-12-12 07:32:21+00 2019-12-12 07:32:25+00
Stats Age 133 days 03:57:58 88 days 12:31:02 88 days 12:31:29 88 days 12:31:55 88 days 12:32:24 88 days 12:32:44 03:10:11 88 days 12:34:36 88 days 12:35:02
Cache Effectiveness 99.36% 99.69% 99.70% 99.70% 99.70% 99.70% 99.62% 99.69% 99.69%
Successful Commits 99.71% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00%
Conflicts 4 2 0 0 0 0 0 0 0
Temp Files: total size 751 GB 4446 GB 4491 GB 4390 GB 4354 GB 4467 GB 631 MB 4408 GB 4412 GB
Temp Files: total number of files 4967 30761 31155 29599 29852 31183 5 30888 30970
Temp Files: total number of files per day 37 345 350 332 335 350 5 347 347
Temp Files: avg file size 155 MB 148 MB 148 MB 152 MB 149 MB 147 MB 126 MB 146 MB 146 MB
Deadlocks 954 0 0 0 0 0 0 0 0
Deadlocks per day 7 0 0 0 0 0 0 0 0

Databases sizes

Database ▼ Size
gitlabhq_production 5.46 TiB
postgres 6.99 MiB
template1 6.94 MiB
template0 6.91 MiB

Conclusions

Recommendations

  Table of contents

A005 Extensions

Observations

Data collected: 2020-03-09 20:07:26 +0000 UTC

Master (10.220.16.106)

▼ Database Extension name Installed version Default version Is old
gitlabhq_production btree_gin 1.0 1.0
gitlabhq_production dblink 1.2 1.2
gitlabhq_production pageinspect 1.5 1.5
gitlabhq_production pg_buffercache 1.2 1.2
gitlabhq_production pg_repack 1.4.4 1.4.4
gitlabhq_production pg_stat_statements 1.4 1.4
gitlabhq_production pg_trgm 1.3 1.3
gitlabhq_production pgstattuple 1.4 1.4
gitlabhq_production plpgsql 1.0 1.0

Conclusions

Recommendations

  Table of contents

A006 Postgres Setting Deviations

Observations

Data collected: 2020-03-09 20:07:27 +0000 UTC

Settings (pg_settings) that Differ

▼ Setting 10.220.16.106 10.220.16.101 10.220.16.102 10.220.16.103 10.220.16.104 10.220.16.105 10.220.16.107 10.220.16.108 10.220.16.109
server_version 9.6.14 9.6.15 9.6.14 9.6.14 9.6.14 9.6.14 9.6.15 9.6.15 9.6.15
server_version_num 90614 90615 90614 90614 90614 90614 90615 90615 90615

Configs(pg_config) that differ

▼ Config 10.220.16.106 10.220.16.101 10.220.16.102 10.220.16.103 10.220.16.104 10.220.16.105 10.220.16.107 10.220.16.108 10.220.16.109
CONFIGURE '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.15-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.15-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.15-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.15-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
VERSION PostgreSQL 9.6.14 PostgreSQL 9.6.15 PostgreSQL 9.6.14 PostgreSQL 9.6.14 PostgreSQL 9.6.14 PostgreSQL 9.6.14 PostgreSQL 9.6.15 PostgreSQL 9.6.15 PostgreSQL 9.6.15

Conclusions

Recommendations

  Table of contents

A007 Altered Settings

Observations

Data collected: 2020-03-09 20:07:28 +0000 UTC

Master (10.220.16.106)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 9 autovacuum_naptime idle_in_transaction_session_timeout log_connections log_disconnections log_min_duration_statement maintenance_work_mem max_wal_size track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 44 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_rotation_age log_rotation_size log_statement log_temp_files logging_collector random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica servers:

Replica (10.220.16.101)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 7 autovacuum_naptime idle_in_transaction_session_timeout log_connections log_disconnections log_min_duration_statement track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 46 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.102)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.103)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.104)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.105)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.107)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.108)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.109)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Conclusions

Recommendations

  Table of contents

D004 pg_stat_statements and pg_stat_kcache Settings

Observations

Data collected: 2020-03-09 20:07:29 +0000 UTC

Master (10.220.16.106)

pg_stat_statements extension settings

Setting Value Unit Type Min value Max value
pg_stat_statements.max 5000 integer 100 2147483647
pg_stat_statements.save on bool
pg_stat_statements.track top enum
pg_stat_statements.track_utility on bool

Conclusions

Recommendations

  Table of contents

F001 Autovacuum: Current Settings

Observations

Data collected: 2020-03-09 20:07:30 +0000 UTC

Master (10.220.16.106)

▼ Setting name Value Unit Pretty value
autovacuum on
autovacuum_analyze_scale_factor 0.005
autovacuum_analyze_threshold 50
autovacuum_freeze_max_age 200000000
autovacuum_max_workers 6
autovacuum_multixact_freeze_max_age 400000000
autovacuum_naptime 30 s
autovacuum_vacuum_cost_delay 5 ms
autovacuum_vacuum_cost_limit 6000
autovacuum_vacuum_scale_factor 0.005
autovacuum_vacuum_threshold 50
autovacuum_work_mem -1 kB
maintenance_work_mem 2097152 kB 2.00 GiB
vacuum_cost_delay 0 ms
vacuum_cost_limit 200
vacuum_cost_page_dirty 20
vacuum_cost_page_hit 1
vacuum_cost_page_miss 10
vacuum_defer_cleanup_age 0
vacuum_freeze_min_age 50000000
vacuum_freeze_table_age 150000000
vacuum_multixact_freeze_min_age 5000000
vacuum_multixact_freeze_table_age 150000000

Tuned tables

▼ Namespace Relation Options
pg_toast pg_toast_33145 autovacuum_enabled=true
pg_toast pg_toast_33153 autovacuum_enabled=true
pg_toast pg_toast_33161 autovacuum_enabled=true
pg_toast pg_toast_33208 autovacuum_enabled=true
pg_toast pg_toast_33221 autovacuum_enabled=true
pg_toast pg_toast_33229 autovacuum_enabled=true
pg_toast pg_toast_33237 autovacuum_enabled=true
pg_toast pg_toast_33248 autovacuum_enabled=true
pg_toast pg_toast_33262 autovacuum_enabled=true
pg_toast pg_toast_33272 autovacuum_enabled=true
pg_toast pg_toast_33322 autovacuum_enabled=true
pg_toast pg_toast_33356 autovacuum_enabled=true
pg_toast pg_toast_33369 autovacuum_enabled=true
pg_toast pg_toast_33383 autovacuum_enabled=true
pg_toast pg_toast_33393 autovacuum_enabled=true
pg_toast pg_toast_33400 autovacuum_enabled=true
pg_toast pg_toast_33426 autovacuum_enabled=true
pg_toast pg_toast_33439 autovacuum_enabled=true
pg_toast pg_toast_33447 autovacuum_enabled=true
pg_toast pg_toast_33462 autovacuum_enabled=true
pg_toast pg_toast_33476 autovacuum_enabled=true
pg_toast pg_toast_33497 autovacuum_enabled=true
pg_toast pg_toast_33506 autovacuum_enabled=true
pg_toast pg_toast_33514 autovacuum_enabled=true
pg_toast pg_toast_33527 autovacuum_enabled=true
pg_toast pg_toast_33541 autovacuum_enabled=true
pg_toast pg_toast_33553 autovacuum_enabled=true
pg_toast pg_toast_33568 autovacuum_enabled=true
pg_toast pg_toast_33587 autovacuum_enabled=true
pg_toast pg_toast_33597 autovacuum_enabled=true
pg_toast pg_toast_33614 autovacuum_enabled=true
pg_toast pg_toast_33625 autovacuum_enabled=true
pg_toast pg_toast_33634 autovacuum_enabled=true
pg_toast pg_toast_33642 autovacuum_enabled=true
pg_toast pg_toast_33650 autovacuum_enabled=true
pg_toast pg_toast_33659 autovacuum_enabled=true
pg_toast pg_toast_33667 autovacuum_enabled=true
pg_toast pg_toast_33675 autovacuum_enabled=true
pg_toast pg_toast_33696 autovacuum_enabled=true
pg_toast pg_toast_33706 autovacuum_enabled=true
pg_toast pg_toast_33749 autovacuum_enabled=true
pg_toast pg_toast_33756 autovacuum_enabled=true
pg_toast pg_toast_33768 autovacuum_enabled=true
pg_toast pg_toast_33776 autovacuum_enabled=true
pg_toast pg_toast_33791 autovacuum_enabled=true
pg_toast pg_toast_33801 autovacuum_enabled=true
pg_toast pg_toast_33824 autovacuum_enabled=true
pg_toast pg_toast_33835 autovacuum_enabled=true
pg_toast pg_toast_33851 autovacuum_enabled=true
pg_toast pg_toast_33863 autovacuum_enabled=true
pg_toast pg_toast_33872 autovacuum_enabled=true
pg_toast pg_toast_33885 autovacuum_enabled=true
pg_toast pg_toast_33898 autovacuum_enabled=true
pg_toast pg_toast_33909 autovacuum_enabled=true
pg_toast pg_toast_33959 autovacuum_enabled=true
public abuse_reports autovacuum_enabled=true
public appearances autovacuum_enabled=true
public application_settings autovacuum_enabled=true
public approval_merge_request_rules_users autovacuum_enabled=on
public approvals autovacuum_enabled=true
public approver_groups autovacuum_enabled=true
public approvers autovacuum_enabled=on
public audit_events autovacuum_enabled=true
public award_emoji autovacuum_enabled=on
public boards autovacuum_enabled=true
public broadcast_messages autovacuum_enabled=true
public ci_builds autovacuum_enabled=true
public ci_pipelines autovacuum_enabled=true
public ci_runner_projects autovacuum_enabled=true
public ci_runners autovacuum_enabled=on
public ci_trigger_requests autovacuum_enabled=true
public ci_triggers autovacuum_enabled=true
public ci_variables autovacuum_enabled=true
public deploy_keys_projects autovacuum_enabled=true
public deployments autovacuum_enabled=on
public emails autovacuum_enabled=true
public environments autovacuum_enabled=true
public geo_nodes autovacuum_enabled=true
public historical_data autovacuum_enabled=true
public identities autovacuum_enabled=true
public index_statuses autovacuum_enabled=true
public issue_metrics autovacuum_enabled=true
public issues autovacuum_enabled=true
public keys autovacuum_enabled=true
public label_links autovacuum_enabled=true
public label_priorities autovacuum_enabled=true
public labels autovacuum_enabled=on
public ldap_group_links autovacuum_enabled=true
public lfs_objects autovacuum_enabled=true
public lfs_objects_projects autovacuum_enabled=true
public licenses autovacuum_enabled=true
public lists autovacuum_enabled=true
public members autovacuum_enabled=true
public merge_request_diffs autovacuum_enabled=true
public merge_request_metrics autovacuum_enabled=true
public merge_requests autovacuum_enabled=true
public merge_requests_closing_issues autovacuum_enabled=true
public milestones autovacuum_enabled=on
public namespaces autovacuum_enabled=on
public notes autovacuum_enabled=true
public notification_settings autovacuum_enabled=on
public oauth_access_grants autovacuum_enabled=true
public oauth_access_tokens autovacuum_enabled=true
public oauth_applications autovacuum_enabled=true
public pages_domains autovacuum_enabled=true
public path_locks autovacuum_enabled=true
public personal_access_tokens autovacuum_enabled=true
public project_authorizations autovacuum_enabled=true
public project_features autovacuum_enabled=true
public project_group_links autovacuum_enabled=true
public project_import_data autovacuum_enabled=on
public project_mirror_data autovacuum_vacuum_cost_limit=10000
autovacuum_enabled=on
public project_repository_states autovacuum_enabled=on
public projects autovacuum_vacuum_cost_limit=7000
autovacuum_enabled=on
public protected_branch_merge_access_levels autovacuum_enabled=true
public protected_branch_push_access_levels autovacuum_enabled=true
public protected_branches autovacuum_enabled=true
public push_rules autovacuum_enabled=true
public releases autovacuum_enabled=on
public remote_mirrors autovacuum_enabled=true
public sent_notifications autovacuum_enabled=true
public services autovacuum_enabled=on
public snippets autovacuum_enabled=on
public spam_logs autovacuum_enabled=true
public subscriptions autovacuum_enabled=true
public taggings autovacuum_enabled=true
public tags autovacuum_enabled=true
public todos autovacuum_enabled=true
public trending_projects autovacuum_enabled=true
public u2f_registrations autovacuum_enabled=true
public untracked_files_for_uploads autovacuum_enabled=on
public user_agent_details autovacuum_enabled=true
public users autovacuum_enabled=on
public users_star_projects autovacuum_enabled=true
public web_hooks autovacuum_enabled=true

Replicas settings

Setting 10.220.16.101 10.220.16.102 10.220.16.103 10.220.16.104 10.220.16.105 10.220.16.107 10.220.16.108 10.220.16.109
hot_standby_feedback on on on on on on on on

Conclusions

Recommendations

All good, no recommendations here.

  Table of contents

F002 Autovacuum: Transaction ID Wraparound Check

Observations

  • Data collected: 2020-03-09 20:05:36 +0000 UTC
  • Current database: gitlabhq_production

Master (10.220.16.106)

Databases

# Database ▼ Age Capacity used, % Warning datfrozenxid
1 gitlabhq_production 198941357 10.2 2563802698
2 postgres 189698526 9.73 2573045529
3 template0 142893885 7.33 2619850170
4 template1 142866793 7.33 2619877262

Tables in the observed database

The list is limited to 10 items. Total: 323.

# Relation Age ▼ Capacity used, % Warning rel_relfrozenxid toast_relfrozenxid
1 design_management_versions 198941357 10.2 2563802698 2616331869
2 vulnerability_occurrence_pipelines 198941357 10.2 2563802698 0
3 operations_feature_flag_scopes 195036437 10 2567707618 2620668984
4 project_pages_metadata 193669484 9.93 2569074571 0
5 serverless_domain_cluster 188504018 9.67 2711775161 2574240037
6 resource_milestone_events 188504018 9.67 2574240037 2574240037
7 merge_request_diff_commits 187505237 9.62 2618095270 2575238818
8 vulnerability_feedback 186831240 9.58 2575912815 2616331869
9 user_details 185058667 9.49 2577685388 0
10 plan_limits 185134366 9.49 2577609689 0

* This table has specific autovacuum settings. See 'F001 Autovacuum: Current settings'

Conclusions

Recommendations

All good, no recommendations here.

  Table of contents

F003 Autovacuum: Dead Tuples

Observations

Data collected: 2020-03-09 20:05:37 +0000 UTC
Current database: gitlabhq_production
Stats reset: 4 mons 12 days 03:58:00 ago (2019-10-28 16:07:35 +0000 UTC)

Master (10.220.16.106)

The list is limited to 20 items. Total: 218.

# Relation reltype Since last autovacuum Since last vacuum Autovacuum Count Vacuum Count n_tup_ins n_tup_upd n_tup_del pg_class.reltuples n_live_tup n_dead_tup ▼Dead Tuples Ratio, %
1 ci_pipeline_schedules r 00:00:08.808841 67535 0 10330 14401916 4512 35466 28086 584 2.04
2 namespace_aggregation_schedules r 00:00:09.881263 470864 0 55810974 0 55285881 13915 13914 155 1.1
3 clusters_applications_runners r 06:55:04.50352 926 0 5579 30572 3539 10217 10220 97 0.94
4 pg_toast.pg_toast_2619 t 00:00:08.555862 204750 0 30437669 0 30436138 21290 21193 194 0.91
5 board_labels r 1 day 14:30:23.02964 293 0 8539 0 6343 11101 11112 99 0.88
6 epics r 01:18:18.29924 3984 0 13681 292705 5487 20324 20324 146 0.71
7 pg_toast.pg_toast_33587* t 05:52:33.250325 228 0 23061 0 18871 12055 12113 80 0.66
8 clusters_applications_helm r 07:18:57.820799 596 0 10039 37447 6101 19909 19919 133 0.66
9 pg_toast.pg_toast_21968240 t 01:43:56.331921 1766 0 123193 0 115780 29192 29202 192 0.65
10 pg_toast.pg_toast_33776* t 00:15:10.619452 15228 0 1342238 0 1340502 19128 19128 123 0.64
11 board_group_recent_visits r 02:18:03.695662 4237 0 13516 601018 3112 34117 34122 216 0.63
12 clusters r 08:31:03.790379 157 0 16773 54442 10327 33395 33398 200 0.6
13 pg_toast.pg_toast_54153184 t 2 days 13:09:48.924237 73 0 30117 0 18277 47110 47344 282 0.59
14 protected_tag_create_access_levels r 1 day 09:59:18.294754 47 0 13329 244 7554 25432 25434 143 0.56
15 u2f_registrations* r 03:32:37.200862 717 0 5696 44192 2170 19452 19452 104 0.53
16 clusters_kubernetes_namespaces r 19:10:36.11696 76 0 25936 3182 14287 45191 45104 240 0.53
17 cluster_providers_gcp r 1 day 18:48:05.002422 94 0 5646 9512 3422 12719 12733 65 0.51
18 epic_issues r 1 day 13:10:04.159292 58 0 31972 9107 5856 62021 62058 310 0.5
19 internal_ids r 00:36:43.606616 7669 0 828635 90354076 180104 3312595 3312679 15958 0.48
20 external_pull_requests r 03:41:08.144032 2149 0 60022 189011 18041 63170 63201 304 0.48

* This table has specific autovacuum settings. See 'F001 Autovacuum: Current settings'

Conclusions

Recommendations

  Table of contents

F004 Autovacuum: Heap Bloat (Estimated)

⚠ This report is based on estimations. The errors in bloat estimates may be significant (in some cases, up to 15% and even more). Use it only as an indicator of potential issues.

Observations

Data collected: 2020-03-09 20:05:37 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

The list is limited to 20 items. Total: 324.

# Table Real Size ▼ Estimated bloat Est. bloat, bytes Est. bloat factor Est. bloat level, % Live Data Size Last vacuum Fillfactor
  ===== TOTAL ===== 3.79 TiB 90.99 GiB 97,695,154,176 1.02 2.35 ~3.70 TiB
1 merge_request_diff_files 1.65 TiB 45.68 GiB 49,048,305,664 1.03 2.72 ~1.60 TiB 2020-03-09 15:15:13 (auto) 100
2 ci_builds* 577.35 GiB 12.56 GiB 13,477,019,648 1.02 2.17 ~564.80 GiB 2020-03-09 12:43:17 (auto) 100
3 notes* 181.61 GiB 6.51 GiB 6,982,352,896 1.04 3.58 ~175.11 GiB 2020-03-09 10:01:42 (auto) 100
4 merge_request_diff_commits 632.45 GiB 4.59 GiB 4,918,935,552 1.01 0.72 ~627.87 GiB 2020-03-07 19:50:37 (auto) 100
5 audit_events* 61.29 GiB 2.73 GiB 2,930,425,856 1.05 4.45 ~58.56 GiB 2020-03-07 19:21:10 (auto) 100
6 ci_pipelines* 24.20 GiB 2.21 GiB 2,369,339,392 1.10 9.12 ~21.99 GiB 2020-03-09 18:11:28 (auto) 100
7 project_mirror_data* 2.14 GiB 1.91 GiB 2,049,572,864 9.33 89.29 ~234.57 MiB 2020-03-09 20:05:37 (auto) 100
8 ci_job_artifacts 57.52 GiB 1.81 GiB 1,935,007,744 1.03 3.13 ~55.72 GiB 2020-03-09 14:08:33 (auto) 100
9 merge_requests* 21.01 GiB 1.18 GiB 1,266,466,816 1.06 5.62 ~19.83 GiB 2020-03-09 19:06:18 (auto) 100
10 services* 1.70 GiB 1.12 GiB 1,193,598,976 2.91 65.67 ~595.02 MiB 2020-03-07 08:21:12 (auto) 100
11 ci_stages 17.71 GiB 1.08 GiB 1,152,917,504 1.06 6.06 ~16.64 GiB 2020-03-08 00:14:12 (auto) 100
12 merge_request_diffs* 14.83 GiB 1.00 GiB 1,070,497,792 1.07 6.73 ~13.83 GiB 2020-03-07 08:37:19 (auto) 100
13 sent_notifications* 42.00 GiB 0.97 GiB 1,031,782,400 1.02 2.29 ~41.03 GiB 2020-03-07 21:00:08 (auto) 100
14 resource_label_events 16.59 GiB 0.96 GiB 1,022,189,568 1.06 5.74 ~15.64 GiB 2020-03-09 18:21:35 (auto) 100
15 deployments* 8.44 GiB 797.08 MiB 835,788,800 1.10 9.23 ~7.66 GiB 2020-03-07 20:32:56 (auto) 100
16 ci_build_trace_sections 155.79 GiB 711.19 MiB 745,734,144 1.00 0.45 ~155.10 GiB 2020-03-07 19:46:30 (auto) 100
17 projects* 3.68 GiB 425.44 MiB 446,103,552 1.13 11.30 ~3.27 GiB 2020-03-09 19:41:40 (auto) 100
18 todos* 4.05 GiB 376.37 MiB 394,649,600 1.10 9.09 ~3.68 GiB 2020-03-09 04:25:22 (auto) 100
19 users* 2.41 GiB 370.47 MiB 388,464,640 1.18 15.06 ~2.05 GiB 2020-03-09 19:52:41 (auto) 100
20 lfs_objects* 4.42 GiB 337.83 MiB 354,238,464 1.08 7.48 ~4.09 GiB 2020-03-09 17:28:06 (auto) 100

* This table has specific autovacuum settings. See 'F001 Autovacuum: Current settings'

Conclusions

  • The estimated table (heap) bloat in this DB is low, just ~2.35% (~90.99 GiB). No action is needed now. Keep watching it though.

  • [P1] The following 2 tables have significant size (>1 MiB) and bloat estimate > 90.00%:

    • ci_build_trace_chunks: size 4.34 MiB, can be reduced 61.67 times, by ~4.27 MiB (~98.38%)
    • appearances: size 1.58 MiB, can be reduced 100.50 times, by ~1.56 MiB (~99.00%)

  • [P2] There are 13 tables with size > 1 MiB and table bloat estimate >= 40.00% and < 90.00%:

    • project_mirror_data: size 2.14 GiB, can be reduced 9.33 times, by ~1.91 GiB (~89.29%)
    • services: size 1.70 GiB, can be reduced 2.91 times, by ~1.12 GiB (~65.67%)
    • personal_access_tokens: size 624.11 MiB, can be reduced 1.71 times, by ~259.97 MiB (~41.65%)
    • lists: size 465.29 MiB, can be reduced 1.88 times, by ~217.81 MiB (~46.81%)
    • ci_runners: size 371.92 MiB, can be reduced 2.33 times, by ~212.22 MiB (~57.06%)
    • etc.

Recommendations

  • [P1] Reduce and prevent the high level of table bloat:

    • to prevent a high level of bloat in the future, tune autovacuum: consider more aggressive autovacuum settings (see F001);
    • eliminate or reduce the current table bloat using one of the approaches listed below.

  • The following tables have size > 1 MiB and table bloat estimate > 40.00%. Use this list to reduce the bloat applying one of the approaches described below. Here are these tables: project_mirror_data, services, personal_access_tokens, lists, ci_runners, vulnerabilities, issue_links, deploy_tokens, project_feature_usages, ci_pipeline_schedules, oauth_applications, ci_build_trace_chunks, appearances, lfs_file_locks, import_failures.

  • If you want to get exact bloat numbers, clone the database, get table sizes, then apply database-wide VACUUM FULL (it eliminate all the bloat), and get new table sizes. Then compare old and new numbers.

  • To reduce the table bloat, consider one of the following approaches:

  • Read more on this topic:

  Table of contents

F005 Autovacuum: Btree Index Bloat (Estimated)

⚠ This report is based on estimations. The errors in bloat estimates may be significant (in some cases, up to 15% and even more). Use it only as an indicator of potential issues.

Observations

Data collected: 2020-03-09 20:05:38 +0000 UTC
Current database: gitlabhq_production
ℹ This report considers only btree indexes.

Master (10.220.16.106)

The list is limited to 20 items. Total: 1150.

# Index (Table) Index Size Table Size ▼ Estimated bloat Est. bloat, bytes Est. bloat factor Est. bloat level, % Live Data Size Fillfactor
  ===== TOTAL ===== 1.63 TiB 9.24 TiB 353.96 GiB 380,059,893,760 1.27 21.34 1.28 TiB
947 index_ci_builds_on_commit_id_and_type_and_name_and_ref (ci_builds*) 51.12 GiB 446.72 GiB 1.00 0.00 ~51.12 GiB 90
2 index_merge_request_diff_commits_on_sha (merge_request_diff_commits) 114.36 GiB 618.35 GiB 19.59 GiB 21,034,082,304 1.21 17.13 ~94.77 GiB 90
3 index_ci_builds_on_token_encrypted (ci_builds*) 31.90 GiB 446.72 GiB 15.51 GiB 16,649,453,568 1.95 48.62 ~16.39 GiB 90
4 index_ci_builds_on_auto_canceled_by_id (ci_builds*) 16.74 GiB 446.72 GiB 11.23 GiB 12,051,881,984 3.04 67.06 ~5.52 GiB 90
5 index_ci_build_trace_sections_on_section_name_id (ci_build_trace_sections) 56.06 GiB 156.48 GiB 10.52 GiB 11,289,583,616 1.23 18.76 ~45.55 GiB 90
6 index_ci_builds_on_token (ci_builds*) 18.65 GiB 446.72 GiB 9.66 GiB 10,371,588,096 2.07 51.80 ~8.99 GiB 90
7 index_ci_build_trace_sections_on_project_id (ci_build_trace_sections) 55.09 GiB 156.48 GiB 9.55 GiB 10,247,618,560 1.21 17.33 ~45.55 GiB 90
8 index_projects_on_repository_storage (projects*) 8.87 GiB 3.42 GiB 8.50 GiB 9,124,265,984 24.13 95.86 ~376.14 MiB 90
9 index_sent_notifications_on_reply_key (sent_notifications*) 25.43 GiB 42.04 GiB 8.40 GiB 9,014,591,488 1.49 33.02 ~17.03 GiB 90
140 index_ci_builds_on_status_and_type_and_runner_id (ci_builds*) 24.42 GiB 446.72 GiB 303.31 MiB 318,038,016 1.01 1.21 ~24.12 GiB 90
11 index_ci_builds_on_queued_at (ci_builds*) 16.91 GiB 446.72 GiB 7.78 GiB 8,344,207,360 1.85 45.97 ~9.14 GiB 90
12 index_ci_job_artifacts_on_expire_at_and_job_id (ci_job_artifacts) 15.81 GiB 57.54 GiB 7.63 GiB 8,185,225,216 1.93 48.24 ~8.18 GiB 90
13 index_ci_builds_on_protected (ci_builds*) 16.66 GiB 446.72 GiB 7.55 GiB 8,105,672,704 1.83 45.32 ~9.11 GiB 90
14 index_projects_on_pending_delete (projects*) 7.54 GiB 3.42 GiB 7.27 GiB 7,804,780,544 28.64 96.51 ~269.26 MiB 90
15 index_ci_builds_on_runner_id (ci_builds*) 16.04 GiB 446.72 GiB 7.00 GiB 7,511,318,528 1.77 43.64 ~9.04 GiB 90
16 index_projects_on_star_count (projects*) 7.16 GiB 3.42 GiB 6.90 GiB 7,399,260,160 27.21 96.32 ~269.26 MiB 90
17 index_projects_on_mirror_user_id (projects*) 6.58 GiB 3.42 GiB 6.32 GiB 6,786,007,040 25.56 96.09 ~263.50 MiB 90
18 index_events_on_target_type_and_target_id (events) 18.17 GiB 39.07 GiB 6.23 GiB 6,683,181,056 1.52 34.26 ~11.95 GiB 90
19 index_ci_builds_on_updated_at (ci_builds*) 14.40 GiB 446.72 GiB 5.24 GiB 5,617,893,376 1.57 36.36 ~9.16 GiB 90
20 index_notes_on_discussion_id (notes*) 19.26 GiB 171.27 GiB 5.15 GiB 5,527,142,400 1.36 26.73 ~14.11 GiB 90

* This table has specific autovacuum settings. See 'F001 Autovacuum: Current settings'

Conclusions

  • [P1] Total index (btree only) bloat estimation is ~353.96 GiB, it is 21.34% of the overall size of all indexes and 6.34% of the DB size. Removing the index bloat will reduce the total DB size down to ~5.12 TiB. Free disk space will be increased by ~353.96 GiB. Total size of indexes is 1.27 times bigger than it could be. Notice that this is only an estimation, sometimes it may be significantly off.

  • [P1] The following 39 indexes have significant size (>1 MiB) and bloat estimate > 90.00%:

    • index_projects_on_repository_storage: size 8.87 GiB, can be reduced 24.13 times, by ~8.54 GiB (~95.86%)
    • index_projects_on_pending_delete: size 7.54 GiB, can be reduced 28.64 times, by ~7.30 GiB (~96.51%)
    • index_projects_on_star_count: size 7.16 GiB, can be reduced 27.21 times, by ~6.92 GiB (~96.32%)
    • index_projects_on_mirror_user_id: size 6.58 GiB, can be reduced 25.56 times, by ~6.35 GiB (~96.09%)
    • index_projects_on_runners_token: size 2.32 GiB, can be reduced 14.33 times, by ~2.18 GiB (~93.02%)
    • etc.

  • [P2] There are 213 indexes with size > 1 MiB and index bloat estimate >= 40.00% and < 90.00%:

    • index_ci_builds_on_token_encrypted: size 31.90 GiB, can be reduced 1.95 times, by ~17.18 GiB (~48.62%)
    • index_ci_builds_on_auto_canceled_by_id: size 16.74 GiB, can be reduced 3.04 times, by ~11.78 GiB (~67.06%)
    • index_ci_builds_on_token: size 18.65 GiB, can be reduced 2.07 times, by ~10.55 GiB (~51.80%)
    • index_ci_builds_on_queued_at: size 16.91 GiB, can be reduced 1.85 times, by ~8.69 GiB (~45.97%)
    • index_ci_job_artifacts_on_expire_at_and_job_id: size 15.81 GiB, can be reduced 1.93 times, by ~8.45 GiB (~48.24%)
    • etc.

Recommendations

  • [P1] Reduce and prevent a high level of index bloat:

    • to prevent a high level of bloat in the future, tune autovacuum: consider more aggressive autovacuum settings (see F001);
    • eliminate or reduce the current index bloat using one of the approaches listed below.

  • The following indexes have size > 1 MiB and index bloat estimate > 40.00%. Use this list to reduce the bloat applying one of the approaches described below. Here are these indexes: index_ci_builds_on_token_encrypted, index_ci_builds_on_auto_canceled_by_id, index_ci_builds_on_token, index_projects_on_repository_storage, index_ci_builds_on_queued_at, index_ci_job_artifacts_on_expire_at_and_job_id, index_ci_builds_on_protected, index_projects_on_pending_delete, index_ci_builds_on_runner_id, index_projects_on_star_count, index_projects_on_mirror_user_id, index_notes_on_commit_id, index_merge_requests_on_milestone_id, index_merge_requests_on_assignee_id, index_ci_pipelines_on_auto_canceled_by_id, index_notes_on_review_id, index_merge_requests_on_target_branch, index_ci_pipelines_on_pipeline_schedule_id, index_merge_requests_on_head_pipeline_id, index_merge_requests_on_latest_merge_request_diff_id, index_projects_on_runners_token, index_projects_on_mirror_last_successful_update_at, index_projects_on_last_repository_check_failed, index_projects_on_last_activity_at_and_id, index_projects_api_last_activity_at_id_desc, index_merge_requests_on_title, index_projects_on_last_repository_updated_at, index_merge_requests_on_state_and_merge_status, index_merge_requests_on_source_branch, index_merge_requests_on_created_at, merge_requests_pkey, idx_issues_on_state_id, index_issues_on_state, index_issues_on_confidential, index_issues_on_closed_by_id, index_issues_on_milestone_id, index_users_on_managing_group_id, index_users_on_static_object_token, index_users_on_group_view, users_reset_password_token_key, index_users_on_ghost, index_users_on_bot_type, index_users_on_state_and_internal, index_users_on_state_and_internal_ee, index_users_on_state, index_users_on_admin, index_merge_request_metrics_on_first_deployed_to_production_at, index_merge_request_metrics_on_latest_closed_by_id, index_users_on_accepted_term_id, index_merge_requests_on_author_id, index_issues_on_updated_at, index_merge_request_metrics, index_todos_on_commit_id, index_todos_on_group_id, index_resource_label_events_on_merge_request_id, index_merge_request_metrics_on_merged_at, index_resource_label_events_on_epic_id, issues_pkey, index_merge_requests_on_lock_version, index_issues_on_relative_position, index_projects_on_runners_token_encrypted, index_issues_on_author_id, index_projects_on_created_at_and_id, index_keys_on_last_used_at, idx_merge_requests_on_state_id_and_merge_status, index_internal_ids_on_namespace_id, index_issues_on_lock_version, index_on_projects_lower_path, users_confirmation_token_key, index_on_projects_path, index_namespaces_on_parent_id_and_id, project_settings_pkey, projects_pkey, index_users_on_incoming_email_token, index_on_namespaces_lower_name, index_namespaces_on_file_template_project_id, index_members_on_requested_at, index_members_on_expires_at, index_protected_branch_push_access_levels_on_group_id, index_protected_branch_push_access_levels_on_user_id, index_keys_on_fingerprint_sha256, index_protected_branch_merge_access_levels_on_user_id, index_protected_branch_merge_access_levels_on_group_id, index_ci_builds_project_id_and_status_for_live_jobs_partial2, index_users_on_feed_token, index_namespaces_on_owner_id, index_namespaces_on_created_at, namespaces_pkey, index_gpg_signatures_on_gpg_key_subkey_id, index_projects_api_vis20_last_activity_at, index_namespaces_on_shared_and_extra_runners_minutes_limit, index_uploads_on_checksum, index_namespaces_on_runners_token_encrypted, index_on_users_name_lower, index_merge_requests_on_updated_by_id, keys_pkey, index_namespaces_on_require_two_factor_authentication, index_namespaces_on_ldap_sync_last_successful_update_at, index_namespaces_on_ldap_sync_last_update_at, index_namespaces_on_runners_token, index_namespaces_on_plan_id, index_ci_pipelines_on_merge_request_id, index_ci_builds_on_artifacts_expire_at, idx_merge_requests_on_source_project_and_branch_state_opened, idx_merge_requests_on_target_project_id_and_iid_opened, index_projects_on_id_partial_for_visibility, index_remote_mirrors_on_last_successful_update_at, index_keys_on_user_id, index_web_hooks_on_type, index_merge_requests_on_merge_user_id, index_container_expiration_policies_on_next_run_at_and_enabled, index_namespaces_on_type_partial, ci_variables_pkey, index_vulnerability_occurrences_on_scanner_id, index_timelogs_on_merge_request_id, index_vulnerability_occurrences_on_primary_identifier_id, index_index_statuses_on_project_id, index_boards_on_milestone_id, web_hooks_pkey, index_boards_on_group_id, releases_pkey, index_releases_on_author_id, ci_runner_projects_pkey, index_statuses_pkey, index_merge_request_metrics_on_latest_closed_at, index_milestones_on_group_id, index_ci_runner_projects_on_runner_id, namespace_aggregation_schedules_pkey, index_namespace_aggregation_schedules_on_namespace_id, project_group_links_pkey, namespace_statistics_pkey, project_import_data_pkey, index_ci_refs_on_last_updated_by_pipeline_id, deploy_keys_projects_pkey, index_deploy_keys_projects_on_project_id, index_merge_requests_on_id_and_merge_jid, idx_merge_requests_on_id_and_merge_jid, index_snippets_on_project_id_and_visibility_level, protected_branch_unprotect_access_levels_pkey, index_protected_branch_unprotect_access, index_project_import_data_on_project_id, index_protected_branch_unprotect_access_levels_on_user_id, index_snippets_on_visibility_level_and_secret, index_for_resource_group, ci_runner_namespaces_pkey, index_snippets_on_updated_at, import_export_uploads_pkey, index_import_export_uploads_on_project_id, index_snippets_on_created_at, snippets_pkey, pages_domains_pkey, index_ci_runner_namespaces_on_runner_id_and_namespace_id, index_snippets_on_author_id, index_pages_domains_on_verified_at_and_enabled_until, index_ci_pipelines_on_external_pull_request_id, index_pages_domains_on_project_id_and_enabled_until, index_import_export_uploads_on_updated_at, index_pages_domains_on_verified_at, index_projects_on_mirror_and_mirror_trigger_builds_both_true, index_issues_on_moved_to_id, index_users_on_public_email, index_pages_domains_on_remove_at, index_emails_on_confirmation_token, index_pages_domains_on_project_id, approval_project_rules_pkey, ci_group_variables_pkey, remote_mirrors_pkey, index_epics_on_closed_by_id, index_epics_on_assignee_id, index_projects_on_pool_repository_id, index_clusters_on_enabled, index_clusters_kubernetes_namespaces_on_cluster_project_id, index_remote_mirrors_on_project_id, index_epics_on_parent_id, index_label_priorities_on_priority, label_priorities_pkey, index_approval_project_rules_on_project_id, index_pages_domains_need_auto_ssl_renewal, index_pages_domains_on_wildcard, index_badges_on_group_id, clusters_pkey, index_namespaces_on_trial_ends_on, index_epics_on_end_date, index_epics_on_start_date, index_users_on_unconfirmed_email, clusters_kubernetes_namespaces_pkey, index_ci_resources_on_build_id, index_pages_domains_on_usage, draft_notes_pkey, index_clusters_on_user_id, index_ci_triggers_on_project_id, index_epics_on_group_id, index_clusters_kubernetes_namespaces_on_environment_id, index_clusters_kubernetes_namespaces_on_cluster_id, index_protected_tag_create_access_levels_on_user_id, index_draft_notes_on_discussion_id, index_epics_on_start_date_sourcing_milestone_id, index_epics_on_due_date_sourcing_milestone_id, index_pages_domains_on_scope, epics_pkey, index_board_group_recent_visits_on_group_id.

  • And the following indexes also have size > 1 MiB and index bloat estimate > 40.00%. However, they belong to the highly bloated tables (see F004), so if you plan to process those tables you may not need to use this additional list. Here are these indexes: index_project_mirror_data_on_status, index_ci_runners_on_contacted_at, index_ci_runners_on_is_shared, index_ci_runners_on_locked, index_ci_runners_on_runner_type, index_ci_runners_on_token, index_services_on_type, services_pkey, index_services_on_project_id, index_services_on_template, index_project_mirror_data_on_last_update_at_and_retry_count, index_mirror_data_on_next_execution_and_retry_count, index_project_mirror_data_on_last_successful_update_at, index_personal_access_tokens_on_token_digest, personal_access_tokens_pkey, index_personal_access_tokens_on_user_id, index_vulnerabilities_on_last_edited_by_id, index_vulnerabilities_on_updated_by_id, index_vulnerabilities_on_milestone_id, index_vulnerabilities_on_closed_by_id, index_vulnerabilities_on_epic_id, index_vulnerabilities_on_start_date_sourcing_milestone_id, index_vulnerabilities_on_resolved_by_id, index_vulnerabilities_on_due_date_sourcing_milestone_id, index_vulnerabilities_on_author_id, index_vulnerabilities_on_project_id, index_vulnerabilities_on_confirmed_by_id, ci_runners_pkey, vulnerabilities_pkey, index_lists_on_milestone_id, index_lists_on_user_id, index_ci_pipeline_schedules_on_next_run_at_and_active, ci_build_trace_chunks_pkey, issue_links_pkey, index_deploy_tokens_on_token_encrypted, idx_proj_feat_usg_on_jira_dvcs_server_last_sync_at_and_proj_id, index_issue_links_on_target_id, index_issue_links_on_source_id, index_ci_build_trace_chunks_on_build_id_and_chunk_index, index_project_feature_usages_on_project_id, project_feature_usages_pkey, index_issue_links_on_source_id_and_target_id, index_deploy_tokens_on_token, deploy_tokens_pkey, index_oauth_applications_on_uid, ci_pipeline_schedules_pkey, idx_proj_feat_usg_on_jira_dvcs_cloud_last_sync_at_and_proj_id, index_ci_pipeline_schedules_on_project_id, index_ci_pipeline_schedules_on_owner_id, oauth_applications_pkey, index_oauth_applications_on_owner_id_and_owner_type.

  • If you want to get exact bloat numbers, clone the database, get index sizes, then apply database-wide VACUUM FULL (it eliminates all the bloat), and get new table sizes. Then compare old and new numbers.

  • To reduce the index bloat, consider one of the following approaches:

    • VACUUM FULL (⚠ requires downtime / maintenance window),
    • REINDEX (REINDEX INDEX, REINDEX TABLE; ⚠ requires downtime / maintenance window),
    • recreating indexes online using CREATE INDEX CONCURRENTLY, DROP INDEX CONCURRENTLY and renaming (not trivial for indexes supporting PK, FK) // REINDEX CONCURRENTLY is available in Postgres 12+,
    • one of the tools reducing the bloat online, without interrupting the operations:

  • Read more on this topic:

  Table of contents

F008 Autovacuum: Resource Usage

Observations

Data collected: 2020-03-09 20:05:51 +0000 UTC

Settings

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_vacuum_cost_delay 5 ms
autovacuum_vacuum_cost_limit 6000
autovacuum_work_mem -1 kB
log_autovacuum_min_duration 0 ms
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
vacuum_cost_limit 200
work_mem 102400 kB 100.00 MiB

CPU

Cpu count you can see in report A001

RAM

Ram amount you can see in report A001

Max workers memory: 12 GiB

DISK

⚠ Warning: collection of current impact on disks is not yet implemented. Please refer to Postgres logs and see current read and write IO bandwidth caused by autovacuum.

Conclusions

Recommendations

  Table of contents

G001 Memory-related Settings

Observations

Data collected: 2020-03-09 20:07:30 +0000 UTC

Master (10.220.16.106)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica servers:

Replica (10.220.16.101)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.102)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.103)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.104)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.105)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.107)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.108)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.109)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Conclusions

Recommendations

  Table of contents

G002 Connections and Current Activity

Observations

Data collected: 2020-03-09 20:07:31 +0000 UTC

Master (10.220.16.106)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 232 2 2 0 0
2 gitlab ALL databases active 30 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 169 1 1 0 0
5 pgbouncer ALL databases idle 3 0 0 0 0
6 gitlab-superuser ALL databases idle 2 1 1 0 0
7 gitlab-monitor ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab ALL databases idle in transaction 25 0 0 0 0
10 gitlab gitlabhq_production active 30 0 0 0 0
11 gitlab-checkup gitlabhq_production active 1 0 0 0 0
12 gitlab gitlabhq_production idle 169 1 1 0 0
13 pgbouncer gitlabhq_production idle 3 0 0 0 0
14 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
15 postgres_exporter gitlabhq_production idle 1 0 0 0 0
16 gitlab-superuser gitlabhq_production idle 1 1 1 0 0
17 gitlab gitlabhq_production idle in transaction 25 0 0 0 0
18 gitlab-superuser postgres idle 1 0 0 0 0

Replica servers:

Replica (10.220.16.101)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 43 0 0 0 0
2 gitlab ALL databases active 5 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 29 0 0 0 0
5 pgbouncer ALL databases idle 4 0 0 0 0
6 gitlab-monitor ALL databases idle 2 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab gitlabhq_production active 5 0 0 0 0
10 gitlab-checkup gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 29 0 0 0 0
12 pgbouncer gitlabhq_production idle 4 0 0 0 0
13 gitlab-monitor gitlabhq_production idle 2 0 0 0 0
14 postgres_exporter gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.102)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 44 0 0 0 0
2 gitlab ALL databases active 9 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 27 0 0 0 0
5 pgbouncer ALL databases idle 4 0 0 0 0
6 gitlab-monitor ALL databases idle 1 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab gitlabhq_production active 9 0 0 0 0
10 gitlab-checkup gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 27 0 0 0 0
12 pgbouncer gitlabhq_production idle 4 0 0 0 0
13 postgres_exporter gitlabhq_production idle 1 0 0 0 0
14 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.103)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 45 1 0 0 0
2 gitlab-checkup ALL databases active 1 0 0 0 0
3 gitlab ALL databases idle 38 1 0 0 0
4 pgbouncer ALL databases idle 3 0 0 0 0
5 gitlab-monitor ALL databases idle 1 0 0 0 0
6 postgres_exporter ALL databases idle 1 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 gitlab-checkup gitlabhq_production active 1 0 0 0 0
9 gitlab gitlabhq_production idle 38 1 0 0 0
10 pgbouncer gitlabhq_production idle 3 0 0 0 0
11 postgres_exporter gitlabhq_production idle 1 0 0 0 0
12 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
13 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.104)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 40 0 0 0 0
2 gitlab ALL databases active 2 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 31 0 0 0 0
5 pgbouncer ALL databases idle 3 0 0 0 0
6 gitlab-monitor ALL databases idle 1 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab gitlabhq_production active 2 0 0 0 0
10 gitlab-checkup gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 31 0 0 0 0
12 pgbouncer gitlabhq_production idle 3 0 0 0 0
13 postgres_exporter gitlabhq_production idle 1 0 0 0 0
14 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.105)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 40 1 0 0 0
2 gitlab ALL databases active 4 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 29 1 0 0 0
5 pgbouncer ALL databases idle 3 0 0 0 0
6 gitlab-monitor ALL databases idle 1 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab gitlabhq_production active 4 0 0 0 0
10 gitlab-checkup gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 29 1 0 0 0
12 pgbouncer gitlabhq_production idle 3 0 0 0 0
13 postgres_exporter gitlabhq_production idle 1 0 0 0 0
14 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.107)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 48 0 0 0 0
2 gitlab ALL databases active 3 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 36 0 0 0 0
5 pgbouncer ALL databases idle 4 0 0 0 0
6 gitlab-monitor ALL databases idle 2 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab gitlabhq_production active 3 0 0 0 0
10 gitlab-checkup gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 36 0 0 0 0
12 pgbouncer gitlabhq_production idle 4 0 0 0 0
13 gitlab-monitor gitlabhq_production idle 2 0 0 0 0
14 postgres_exporter gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.108)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 105 0 0 0 0
2 gitlab ALL databases active 5 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 69 0 0 0 0
5 pgbouncer ALL databases idle 26 0 0 0 0
6 gitlab-monitor ALL databases idle 2 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab gitlabhq_production active 5 0 0 0 0
10 gitlab-checkup gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 69 0 0 0 0
12 pgbouncer gitlabhq_production idle 26 0 0 0 0
13 gitlab-monitor gitlabhq_production idle 2 0 0 0 0
14 postgres_exporter gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.109)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 44 0 0 0 0
2 gitlab ALL databases active 2 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 34 0 0 0 0
5 pgbouncer ALL databases idle 4 0 0 0 0
6 gitlab-monitor ALL databases idle 1 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab gitlabhq_production active 2 0 0 0 0
10 gitlab-checkup gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 34 0 0 0 0
12 pgbouncer gitlabhq_production idle 4 0 0 0 0
13 postgres_exporter gitlabhq_production idle 1 0 0 0 0
14 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Conclusions

Recommendations

All good, no recommendations here.

  Table of contents

G003 Timeouts, Locks, Deadlocks

Observations

Data collected: 2020-03-09 20:07:31 +0000 UTC

Master (10.220.16.106)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

# Database Conflicts ▼ Deadlocks Stats reset at Stat reset
1 gitlabhq_production 4 954 2019-10-28T16:07:34.673503+00:00 133 days 03:58:19
2 postgres 0 0 2019-10-28T16:07:29.861973+00:00 133 days 03:58:23

Replica servers:

Replica (10.220.16.101)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:12.080501+00:00 88 days 12:31:06
gitlabhq_production 2 0 2019-12-12T07:32:12.114073+00:00 88 days 12:31:06

Replica (10.220.16.102)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:11.427924+00:00 88 days 12:31:34
gitlabhq_production 0 0 2019-12-12T07:32:11.45222+00:00 88 days 12:31:34

Replica (10.220.16.103)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:12.581377+00:00 88 days 12:32:01
gitlabhq_production 0 0 2019-12-12T07:32:12.982689+00:00 88 days 12:32:00

Replica (10.220.16.104)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:11.156846+00:00 88 days 12:32:30
gitlabhq_production 0 0 2019-12-12T07:32:12.131981+00:00 88 days 12:32:29

Replica (10.220.16.105)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:17.615779+00:00 88 days 12:32:52
gitlabhq_production 0 0 2019-12-12T07:32:20.485667+00:00 88 days 12:32:49

Replica (10.220.16.107)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2020-03-09T16:56:17.566669+00:00 03:10:16
gitlabhq_production 0 0 2020-03-09T16:56:17.564697+00:00 03:10:16

Replica (10.220.16.108)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:20.760583+00:00 88 days 12:34:41
gitlabhq_production 0 0 2019-12-12T07:32:20.698715+00:00 88 days 12:34:41

Replica (10.220.16.109)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:21.689386+00:00 88 days 12:35:10
gitlabhq_production 0 0 2019-12-12T07:32:24.58281+00:00 88 days 12:35:07

Conclusions

Recommendations

  Table of contents

H001 Invalid Indexes

Observations

Data collected: 2020-03-09 20:05:53 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

Invalid indexes not found

Conclusions

Recommendations

  Table of contents

H002 Unused Indexes

Observations

Data collected: 2020-03-09 20:07:32 +0000 UTC
Current database: gitlabhq_production

Stats reset: 4 mons 12 days 03:58:00 ago (2019-10-28 16:07:35 +0000 UTC)

Never Used Indexes

The list is limited to 20 items. Total: 69.

# Table Index 10.220.16.106 usage 10.220.16.101 usage 10.220.16.102 usage 10.220.16.103 usage 10.220.16.104 usage 10.220.16.105 usage 10.220.16.107 usage 10.220.16.108 usage 10.220.16.109 usage ▼ Index size Table size Supports FK
  =====TOTAL===== 72.08 GiB 2.07 TiB
1 ci_builds index_ci_builds_on_queued_at 0 0 0 0 0 0 0 16.95 GiB 446.59 GiB
2 ci_builds index_ci_builds_on_protected 0 0 0 0 0 0 0 0 0 16.70 GiB 446.59 GiB
3 ci_job_artifacts index_ci_job_artifacts_on_file_store 0 0 0 0 0 0 0 0 11.44 GiB 57.52 GiB
4 audit_events analytics_index_audit_events_on_created_at_and_author_id 0 0 7.56 GiB 61.43 GiB
5 projects index_projects_on_runners_token 0 0 0 0 0 0 0 0 0 2.32 GiB 3.41 GiB Yes
6 projects index_projects_on_last_repository_check_failed 0 0 0 0 0 0 0 0 0 2.32 GiB 3.41 GiB Yes
7 merge_requests index_merge_requests_on_state_and_merge_status 0 0 0 0 0 0 0 0 2.08 GiB 19.28 GiB
8 merge_request_diffs index_merge_request_diffs_on_merge_request_id_and_id_partial 0 0 0 0 0 0 0 0 0 2.00 GiB 14.83 GiB Yes
9 issues index_issues_on_state 0 0 0 0 0 0 0 0 1.42 GiB 16.20 GiB Yes
10 merge_requests index_merge_requests_on_source_project_and_branch_state_opened 0 0 0 0 0 0 0 0 0 949.19 MiB 19.28 GiB
11 projects idx_project_repository_check_partial 0 0 0 0 0 0 0 0 0 889.95 MiB 3.41 GiB
12 users index_users_on_accepted_term_id 0 0 0 0 0 0 0 0 0 803.68 MiB 2.41 GiB Yes
13 projects index_on_projects_lower_path 0 0 0 0 0 0 0 0 0 658.97 MiB 3.41 GiB Yes
14 ci_runners index_ci_runners_on_contacted_at 0 0 480.24 MiB 371.92 MiB
15 ci_runners index_ci_runners_on_is_shared 0 0 0 0 0 0 0 0 0 473.25 MiB 371.92 MiB
16 ci_runners index_ci_runners_on_locked 0 0 0 0 0 0 0 0 0 457.95 MiB 371.92 MiB
17 merge_requests index_merge_requests_on_target_project_id_and_iid_opened 0 0 0 0 0 0 0 451.06 MiB 19.28 GiB Yes
18 ci_runners index_ci_runners_on_token 0 0 0 0 0 0 0 0 0 433.33 MiB 371.92 MiB
19 keys index_keys_on_last_used_at 0 0 0 0 0 0 413.81 MiB 1.98 GiB
20 merge_requests idx_merge_requests_on_state_id_and_merge_status 0 0 0 0 0 0 0 0 0 336.56 MiB 19.28 GiB

Rarely Used Indexes

The list is limited to 20 items. Total: 305.

# Table Index 10.220.16.106 usage 10.220.16.101 usage 10.220.16.102 usage 10.220.16.103 usage 10.220.16.104 usage 10.220.16.105 usage 10.220.16.107 usage 10.220.16.108 usage 10.220.16.109 usage ▼ Index size Table size Comment Supports FK
  =====TOTAL===== 911.90 GiB 11.76 TiB
1 merge_request_diff_commits index_merge_request_diff_commits_on_sha scans: 142,891,123/hour, writes: 582,445,770/hour scans: 445,909/hour, writes: 0/hour scans: 445,810/hour, writes: 0/hour scans: 444,491/hour, writes: 0/hour scans: 446,807/hour, writes: 0/hour scans: 444,646/hour, writes: 0/hour scans: 999/hour, writes: 0/hour scans: 445,416/hour, writes: 0/hour scans: 445,591/hour, writes: 0/hour 114.56 GiB 618.18 GiB Low Scans, High Writes
2 ci_build_trace_sections index_ci_build_trace_sections_on_section_name_id scans: 475,856/hour, writes: 866,483,544/hour 56.30 GiB 156.44 GiB Low Scans, High Writes
3 ci_build_trace_sections index_ci_build_trace_sections_on_project_id scans: 624,305/hour, writes: 866,483,544/hour 55.33 GiB 156.44 GiB Low Scans, High Writes
4 ci_builds index_ci_builds_on_status_and_type_and_runner_id scans: 320,646,420/hour, writes: 509,864,271/hour scans: 34,945,183/hour, writes: 0/hour scans: 34,950,146/hour, writes: 0/hour scans: 34,934,683/hour, writes: 0/hour scans: 34,946,514/hour, writes: 0/hour scans: 34,950,178/hour, writes: 0/hour scans: 107,455/hour, writes: 0/hour scans: 34,945,601/hour, writes: 0/hour scans: 34,923,987/hour, writes: 0/hour 24.45 GiB 446.59 GiB Low Scans, High Writes
5 events index_events_on_project_id_and_created_at scans: 42,943/hour, writes: 154,611,716/hour scans: 971,911/hour, writes: 0/hour scans: 1,006,942/hour, writes: 0/hour scans: 985,292/hour, writes: 0/hour scans: 909,376/hour, writes: 0/hour scans: 917,154/hour, writes: 0/hour scans: 505/hour, writes: 0/hour scans: 878,135/hour, writes: 0/hour scans: 920,629/hour, writes: 0/hour 23.78 GiB 39.06 GiB Low Scans, High Writes
6 ci_builds ci_builds_gitlab_monitor_metrics scans: 136,586/hour, writes: 509,864,271/hour scans: 12,174/hour, writes: 0/hour scans: 12,461/hour, writes: 0/hour scans: 12,228/hour, writes: 0/hour scans: 12,303/hour, writes: 0/hour scans: 12,343/hour, writes: 0/hour scans: 12,355/hour, writes: 0/hour scans: 12,120/hour, writes: 0/hour 20.90 GiB 446.59 GiB Low Scans, High Writes
7 notes index_notes_on_discussion_id scans: 53,110,674/hour, writes: 113,656,636/hour scans: 418,532/hour, writes: 0/hour scans: 419,799/hour, writes: 0/hour scans: 418,524/hour, writes: 0/hour scans: 420,751/hour, writes: 0/hour scans: 418,485/hour, writes: 0/hour scans: 999/hour, writes: 0/hour scans: 420,520/hour, writes: 0/hour scans: 420,011/hour, writes: 0/hour 19.32 GiB 171.22 GiB Low Scans, High Writes
8 events analytics_index_events_on_created_at_and_author_id scans: 351,320/hour, writes: 154,611,716/hour scans: 1,937,038/hour, writes: 0/hour scans: 1,935,059/hour, writes: 0/hour scans: 1,932,902/hour, writes: 0/hour scans: 1,935,700/hour, writes: 0/hour scans: 1,939,349/hour, writes: 0/hour scans: 4,563/hour, writes: 0/hour scans: 1,937,636/hour, writes: 0/hour scans: 1,934,903/hour, writes: 0/hour 17.49 GiB 39.06 GiB Low Scans, High Writes
9 ci_builds index_ci_builds_on_auto_canceled_by_id scans: 2,887,765/hour, writes: 509,864,271/hour 16.78 GiB 446.59 GiB Low Scans, High Writes Yes
10 ci_builds tmp_build_stage_position_index scans: 1,041/hour, writes: 509,864,271/hour 16.72 GiB 446.59 GiB Low Scans, High Writes
11 ci_builds index_ci_builds_on_runner_id scans: 76,952,009/hour, writes: 509,864,271/hour scans: 9,242,754/hour, writes: 0/hour scans: 9,243,578/hour, writes: 0/hour scans: 9,243,444/hour, writes: 0/hour scans: 9,241,706/hour, writes: 0/hour scans: 9,244,873/hour, writes: 0/hour scans: 19,684/hour, writes: 0/hour scans: 9,242,094/hour, writes: 0/hour scans: 9,241,326/hour, writes: 0/hour 16.07 GiB 446.59 GiB Low Scans, High Writes Yes
12 ci_job_artifacts index_ci_job_artifacts_on_expire_at_and_job_id scans: 336,419/hour, writes: 355,722,793/hour 15.84 GiB 57.52 GiB Low Scans, High Writes
13 events index_events_on_author_id_and_project_id scans: 21,401,029/hour, writes: 154,611,716/hour 15.68 GiB 39.06 GiB Low Scans, High Writes
14 events index_events_on_project_id_and_id scans: 25,921,142/hour, writes: 154,611,716/hour 15.65 GiB 39.06 GiB Low Scans, High Writes
15 ci_builds index_ci_builds_on_project_id_and_name_and_ref scans: 587/hour, writes: 509,864,271/hour scans: 9/hour, writes: 0/hour 15.37 GiB 446.59 GiB Low Scans, High Writes
16 events index_events_on_action scans: 22/hour, writes: 154,611,716/hour scans: 2,130/hour, writes: 0/hour scans: 2,052/hour, writes: 0/hour scans: 2,164/hour, writes: 0/hour scans: 2,112/hour, writes: 0/hour scans: 2,168/hour, writes: 0/hour scans: 2/hour, writes: 0/hour scans: 2,126/hour, writes: 0/hour scans: 2,024/hour, writes: 0/hour 14.44 GiB 39.06 GiB Low Scans, High Writes
17 ci_builds index_ci_builds_on_updated_at scans: 20,950/hour, writes: 509,864,271/hour 14.43 GiB 446.59 GiB Low Scans, High Writes
18 ci_builds index_ci_builds_on_user_id scans: 25,436/hour, writes: 509,864,271/hour scans: 20/hour, writes: 0/hour 12.96 GiB 446.59 GiB Low Scans, High Writes Yes
19 ci_job_artifacts index_ci_job_artifacts_on_project_id scans: 627,815/hour, writes: 355,722,793/hour scans: 1,727/hour, writes: 0/hour scans: 1,623/hour, writes: 0/hour scans: 1,688/hour, writes: 0/hour scans: 1,764/hour, writes: 0/hour scans: 1,899/hour, writes: 0/hour scans: 4/hour, writes: 0/hour scans: 1,735/hour, writes: 0/hour scans: 1,759/hour, writes: 0/hour 10.66 GiB 57.52 GiB Low Scans, High Writes
20 ci_builds_metadata index_ci_builds_metadata_on_project_id scans: 624,298/hour, writes: 229,820,618/hour 10.07 GiB 19.24 GiB Low Scans, High Writes

Conclusions

  • [P3] 69 unused index(es) have been found.

Recommendations

  • [P3] Use the database migration provided below to drop the unused indexes. Keep in mind, that under load, it is recommended to use DROP INDEX CONCURRENTLY (and CREATE INDEX CONCURRENTLY if reverting is needed) to avoid blocking issues.
  • Be careful dropping the indexes. If you have multiple setups of your software, the analysis of just a single setup might be not enough. Some indexes might be used (and therefore, needed) only on a limited number of setups. Also, in some cases, developers prepare indexes for new features in advance – in such cases, dropping those indexes is not a good idea.
  • If there are some doubts, consider a more careful approach. Before actualdropping, indexes disable listed in this report. For this, use queries like `UPDATE pg_index SET indisvalid = false WHERE indexrelid::regclass = (select oid from pg_class where relname = 'u_users_email');. Indexes will continue to get updates. In case of some performance degradations, re-enable the corresponding indexes, setting indisvalid to true. If everything looks fine, after a significant period of observations, proceed with DROP INDEX CONCURRENTLY.
  • "DO" database migrations
DROP INDEX CONCURRENTLY analytics_index_audit_events_on_created_at_and_author_id; -- 7740 MB, Never Used Indexes, table audit_events  
DROP INDEX CONCURRENTLY index_ci_builds_on_artifacts_expire_at; -- 82 MB, Never Used Indexes, table ci_builds  
DROP INDEX CONCURRENTLY index_ci_builds_on_project_id_for_successfull_pages_deploy; -- 105 MB, Never Used Indexes, table ci_builds  
DROP INDEX CONCURRENTLY index_ci_builds_on_protected; -- 17 GB, Never Used Indexes, table ci_builds  
DROP INDEX CONCURRENTLY index_ci_builds_on_queued_at; -- 17 GB, Never Used Indexes, table ci_builds  
DROP INDEX CONCURRENTLY index_ci_builds_metadata_on_build_id_and_has_exposed_artifacts; -- 2592 kB, Never Used Indexes, table ci_builds_metadata  
DROP INDEX CONCURRENTLY index_ci_job_artifacts_on_file_store; -- 11 GB, Never Used Indexes, table ci_job_artifacts  
DROP INDEX CONCURRENTLY index_ci_pipeline_schedules_on_next_run_at_and_active; -- 32 MB, Never Used Indexes, table ci_pipeline_schedules  
DROP INDEX CONCURRENTLY index_ci_runners_on_contacted_at; -- 480 MB, Never Used Indexes, table ci_runners  
DROP INDEX CONCURRENTLY index_ci_runners_on_is_shared; -- 473 MB, Never Used Indexes, table ci_runners  
DROP INDEX CONCURRENTLY index_ci_runners_on_locked; -- 458 MB, Never Used Indexes, table ci_runners  
DROP INDEX CONCURRENTLY index_ci_runners_on_token; -- 433 MB, Never Used Indexes, table ci_runners  
DROP INDEX CONCURRENTLY index_cluster_providers_gcp_on_cloud_run; -- 328 kB, Never Used Indexes, table cluster_providers_gcp  
DROP INDEX CONCURRENTLY index_deploy_tokens_on_token_and_expires_at_and_id; -- 16 MB, Never Used Indexes, table deploy_tokens  
DROP INDEX CONCURRENTLY idx_deployment_clusters_on_cluster_id_and_kubernetes_namespace; -- 33 MB, Never Used Indexes, table deployment_clusters  
DROP INDEX CONCURRENTLY partial_index_deployments_for_legacy_successful_deployments; -- 16 kB, Never Used Indexes, table deployments  
DROP INDEX CONCURRENTLY index_design_management_designs_versions_on_event; -- 160 kB, Never Used Indexes, table design_management_designs_versions  
DROP INDEX CONCURRENTLY index_elasticsearch_indexed_namespaces_on_created_at; -- 16 kB, Never Used Indexes, table elasticsearch_indexed_namespaces  
DROP INDEX CONCURRENTLY index_gitlab_subscription_histories_on_gitlab_subscription_id; -- 1456 kB, Never Used Indexes, table gitlab_subscription_histories  
DROP INDEX CONCURRENTLY index_grafana_integrations_on_enabled; -- 16 kB, Never Used Indexes, table grafana_integrations  
DROP INDEX CONCURRENTLY index_import_failures_on_correlation_id_value; -- 280 kB, Never Used Indexes, table import_failures  
DROP INDEX CONCURRENTLY index_issues_on_state; -- 1444 MB, Never Used Indexes, table issues  
DROP INDEX CONCURRENTLY issue_id_issues_self_managed_rometheus_alert_events_index; -- 16 kB, Never Used Indexes, table issues_self_managed_prometheus_alert_events  
DROP INDEX CONCURRENTLY index_keys_on_last_used_at; -- 414 MB, Never Used Indexes, table keys  
DROP INDEX CONCURRENTLY index_label_priorities_on_priority; -- 3880 kB, Never Used Indexes, table label_priorities  
DROP INDEX CONCURRENTLY index_labels_on_group_id_and_title; -- 8192 bytes, Never Used Indexes, table labels  
DROP INDEX CONCURRENTLY index_labels_on_project_id_and_title; -- 8192 bytes, Never Used Indexes, table labels  
DROP INDEX CONCURRENTLY index_merge_request_diffs_on_merge_request_id_and_id_partial; -- 2039 MB, Never Used Indexes, table merge_request_diffs  
DROP INDEX CONCURRENTLY index_merge_request_metrics_on_latest_closed_at; -- 53 MB, Never Used Indexes, table merge_request_metrics  
DROP INDEX CONCURRENTLY idx_merge_requests_on_state_id_and_merge_status; -- 337 MB, Never Used Indexes, table merge_requests  
DROP INDEX CONCURRENTLY index_merge_requests_on_id_and_merge_jid; -- 11 MB, Never Used Indexes, table merge_requests  
DROP INDEX CONCURRENTLY index_merge_requests_on_source_project_and_branch_state_opened; -- 949 MB, Never Used Indexes, table merge_requests  
DROP INDEX CONCURRENTLY index_merge_requests_on_state_and_merge_status; -- 2123 MB, Never Used Indexes, table merge_requests  
DROP INDEX CONCURRENTLY index_merge_requests_on_target_project_id_and_iid_opened; -- 451 MB, Never Used Indexes, table merge_requests  
DROP INDEX CONCURRENTLY index_namespaces_on_created_at; -- 295 MB, Never Used Indexes, table namespaces  
DROP INDEX CONCURRENTLY index_namespaces_on_ldap_sync_last_successful_update_at; -- 257 MB, Never Used Indexes, table namespaces  
DROP INDEX CONCURRENTLY index_namespaces_on_ldap_sync_last_update_at; -- 257 MB, Never Used Indexes, table namespaces  
DROP INDEX CONCURRENTLY index_namespaces_on_plan_id; -- 257 MB, Never Used Indexes, table namespaces  
DROP INDEX CONCURRENTLY index_namespaces_on_require_two_factor_authentication; -- 257 MB, Never Used Indexes, table namespaces  
DROP INDEX CONCURRENTLY index_packages_dependency_links_on_dependency_id; -- 6952 kB, Never Used Indexes, table packages_dependency_links  
DROP INDEX CONCURRENTLY index_pages_domains_need_auto_ssl_renewal; -- 1784 kB, Never Used Indexes, table pages_domains  
DROP INDEX CONCURRENTLY index_pages_domains_on_verified_at_and_enabled_until; -- 6080 kB, Never Used Indexes, table pages_domains  
DROP INDEX CONCURRENTLY index_project_custom_attributes_on_key_and_value; -- 16 kB, Never Used Indexes, table project_custom_attributes  
DROP INDEX CONCURRENTLY index_project_mirror_data_on_last_successful_update_at; -- 210 MB, Never Used Indexes, table project_mirror_data  
DROP INDEX CONCURRENTLY index_project_repositories_on_shard_id; -- 335 MB, Never Used Indexes, table project_repositories  
DROP INDEX CONCURRENTLY idx_repository_states_on_last_repository_verification_ran_at; -- 227 MB, Never Used Indexes, table project_repository_states  
DROP INDEX CONCURRENTLY idx_repository_states_on_last_wiki_verification_ran_at; -- 243 MB, Never Used Indexes, table project_repository_states  
DROP INDEX CONCURRENTLY idx_repository_states_on_repository_failure_partial; -- 16 kB, Never Used Indexes, table project_repository_states  
DROP INDEX CONCURRENTLY idx_repository_states_on_wiki_failure_partial; -- 8192 bytes, Never Used Indexes, table project_repository_states  
DROP INDEX CONCURRENTLY idx_repository_states_outdated_checksums; -- 23 MB, Never Used Indexes, table project_repository_states  
DROP INDEX CONCURRENTLY idx_project_repository_check_partial; -- 890 MB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_on_projects_lower_path; -- 659 MB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_projects_api_vis20_name; -- 89 MB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_projects_api_vis20_path; -- 88 MB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_projects_api_vis20_updated_at; -- 84 MB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_projects_on_last_repository_check_at; -- 16 kB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_projects_on_last_repository_check_failed; -- 2366 MB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_projects_on_mirror_and_mirror_trigger_builds_both_true; -- 5680 kB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_projects_on_runners_token; -- 2373 MB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_prometheus_metrics_on_group; -- 40 kB, Never Used Indexes, table prometheus_metrics  
DROP INDEX CONCURRENTLY index_remote_mirrors_on_last_successful_update_at; -- 60 MB, Never Used Indexes, table remote_mirrors  
DROP INDEX CONCURRENTLY idx_security_scans_on_scan_type; -- 3728 kB, Never Used Indexes, table security_scans  
DROP INDEX CONCURRENTLY index_term_agreements_on_term_id; -- 114 MB, Never Used Indexes, table term_agreements  
DROP INDEX CONCURRENTLY index_timelogs_on_user_id; -- 46 MB, Never Used Indexes, table timelogs  
DROP INDEX CONCURRENTLY index_user_custom_attributes_on_key_and_value; -- 16 kB, Never Used Indexes, table user_custom_attributes  
DROP INDEX CONCURRENTLY index_users_on_accepted_term_id; -- 804 MB, Never Used Indexes, table users  
DROP INDEX CONCURRENTLY index_x509_certificates_on_x509_issuer_id; -- 16 kB, Never Used Indexes, table x509_certificates  
DROP INDEX CONCURRENTLY index_x509_commit_signatures_on_x509_certificate_id; -- 40 kB, Never Used Indexes, table x509_commit_signatures  
DROP INDEX CONCURRENTLY index_zoom_meetings_on_issue_status; -- 16 kB, Never Used Indexes, table zoom_meetings
  • "UNDO" database migrations
CREATE INDEX CONCURRENTLY analytics_index_audit_events_on_created_at_and_author_id ON public.audit_events USING btree (created_at, author_id); -- table audit_events  
CREATE INDEX CONCURRENTLY index_ci_builds_on_artifacts_expire_at ON public.ci_builds USING btree (artifacts_expire_at) WHERE (artifacts_file <> ''::text); -- table ci_builds  
CREATE INDEX CONCURRENTLY index_ci_builds_on_project_id_for_successfull_pages_deploy ON public.ci_builds USING btree (project_id) WHERE (((type)::text = 'GenericCommitStatus'::text) AND ((stage)::text = 'deploy'::text) AND ((name)::text = 'pages:deploy'::text) AND ((status)::text = 'success'::text)); -- table ci_builds  
CREATE INDEX CONCURRENTLY index_ci_builds_on_protected ON public.ci_builds USING btree (protected); -- table ci_builds  
CREATE INDEX CONCURRENTLY index_ci_builds_on_queued_at ON public.ci_builds USING btree (queued_at); -- table ci_builds  
CREATE INDEX CONCURRENTLY index_ci_builds_metadata_on_build_id_and_has_exposed_artifacts ON public.ci_builds_metadata USING btree (build_id) WHERE (has_exposed_artifacts IS TRUE); -- table ci_builds_metadata  
CREATE INDEX CONCURRENTLY index_ci_job_artifacts_on_file_store ON public.ci_job_artifacts USING btree (file_store); -- table ci_job_artifacts  
CREATE INDEX CONCURRENTLY index_ci_pipeline_schedules_on_next_run_at_and_active ON public.ci_pipeline_schedules USING btree (next_run_at, active); -- table ci_pipeline_schedules  
CREATE INDEX CONCURRENTLY index_ci_runners_on_contacted_at ON public.ci_runners USING btree (contacted_at); -- table ci_runners  
CREATE INDEX CONCURRENTLY index_ci_runners_on_is_shared ON public.ci_runners USING btree (is_shared); -- table ci_runners  
CREATE INDEX CONCURRENTLY index_ci_runners_on_locked ON public.ci_runners USING btree (locked); -- table ci_runners  
CREATE INDEX CONCURRENTLY index_ci_runners_on_token ON public.ci_runners USING btree (token); -- table ci_runners  
CREATE INDEX CONCURRENTLY index_cluster_providers_gcp_on_cloud_run ON public.cluster_providers_gcp USING btree (cloud_run); -- table cluster_providers_gcp  
CREATE INDEX CONCURRENTLY index_deploy_tokens_on_token_and_expires_at_and_id ON public.deploy_tokens USING btree (token, expires_at, id) WHERE (revoked IS FALSE); -- table deploy_tokens  
CREATE INDEX CONCURRENTLY idx_deployment_clusters_on_cluster_id_and_kubernetes_namespace ON public.deployment_clusters USING btree (cluster_id, kubernetes_namespace); -- table deployment_clusters  
CREATE INDEX CONCURRENTLY partial_index_deployments_for_legacy_successful_deployments ON public.deployments USING btree (id) WHERE ((finished_at IS NULL) AND (status = 2)); -- table deployments  
CREATE INDEX CONCURRENTLY index_design_management_designs_versions_on_event ON public.design_management_designs_versions USING btree (event); -- table design_management_designs_versions  
CREATE INDEX CONCURRENTLY index_elasticsearch_indexed_namespaces_on_created_at ON public.elasticsearch_indexed_namespaces USING btree (created_at); -- table elasticsearch_indexed_namespaces  
CREATE INDEX CONCURRENTLY index_gitlab_subscription_histories_on_gitlab_subscription_id ON public.gitlab_subscription_histories USING btree (gitlab_subscription_id); -- table gitlab_subscription_histories  
CREATE INDEX CONCURRENTLY index_grafana_integrations_on_enabled ON public.grafana_integrations USING btree (enabled) WHERE (enabled IS TRUE); -- table grafana_integrations  
CREATE INDEX CONCURRENTLY index_import_failures_on_correlation_id_value ON public.import_failures USING btree (correlation_id_value); -- table import_failures  
CREATE INDEX CONCURRENTLY index_issues_on_state ON public.issues USING btree (state); -- table issues  
CREATE INDEX CONCURRENTLY issue_id_issues_self_managed_rometheus_alert_events_index ON public.issues_self_managed_prometheus_alert_events USING btree (self_managed_prometheus_alert_event_id); -- table issues_self_managed_prometheus_alert_events  
CREATE INDEX CONCURRENTLY index_keys_on_last_used_at ON public.keys USING btree (last_used_at DESC NULLS LAST); -- table keys  
CREATE INDEX CONCURRENTLY index_label_priorities_on_priority ON public.label_priorities USING btree (priority); -- table label_priorities  
CREATE INDEX CONCURRENTLY index_labels_on_group_id_and_title ON public.labels USING btree (group_id, title) WHERE (project_id = NULL::integer); -- table labels  
CREATE INDEX CONCURRENTLY index_labels_on_project_id_and_title ON public.labels USING btree (project_id, title) WHERE (group_id = NULL::integer); -- table labels  
CREATE INDEX CONCURRENTLY index_merge_request_diffs_on_merge_request_id_and_id_partial ON public.merge_request_diffs USING btree (merge_request_id, id) WHERE ((NOT stored_externally) OR (stored_externally IS NULL)); -- table merge_request_diffs  
CREATE INDEX CONCURRENTLY index_merge_request_metrics_on_latest_closed_at ON public.merge_request_metrics USING btree (latest_closed_at) WHERE (latest_closed_at IS NOT NULL); -- table merge_request_metrics  
CREATE INDEX CONCURRENTLY idx_merge_requests_on_state_id_and_merge_status ON public.merge_requests USING btree (state_id, merge_status) WHERE ((state_id = 1) AND ((merge_status)::text = 'can_be_merged'::text)); -- table merge_requests  
CREATE INDEX CONCURRENTLY index_merge_requests_on_id_and_merge_jid ON public.merge_requests USING btree (id, merge_jid) WHERE ((merge_jid IS NOT NULL) AND ((state)::text = 'locked'::text)); -- table merge_requests  
CREATE INDEX CONCURRENTLY index_merge_requests_on_source_project_and_branch_state_opened ON public.merge_requests USING btree (source_project_id, source_branch) WHERE ((state)::text = 'opened'::text); -- table merge_requests  
CREATE INDEX CONCURRENTLY index_merge_requests_on_state_and_merge_status ON public.merge_requests USING btree (state, merge_status) WHERE (((state)::text = 'opened'::text) AND ((merge_status)::text = 'can_be_merged'::text)); -- table merge_requests  
CREATE INDEX CONCURRENTLY index_merge_requests_on_target_project_id_and_iid_opened ON public.merge_requests USING btree (target_project_id, iid) WHERE ((state)::text = 'opened'::text); -- table merge_requests  
CREATE INDEX CONCURRENTLY index_namespaces_on_created_at ON public.namespaces USING btree (created_at); -- table namespaces  
CREATE INDEX CONCURRENTLY index_namespaces_on_ldap_sync_last_successful_update_at ON public.namespaces USING btree (ldap_sync_last_successful_update_at); -- table namespaces  
CREATE INDEX CONCURRENTLY index_namespaces_on_ldap_sync_last_update_at ON public.namespaces USING btree (ldap_sync_last_update_at); -- table namespaces  
CREATE INDEX CONCURRENTLY index_namespaces_on_plan_id ON public.namespaces USING btree (plan_id); -- table namespaces  
CREATE INDEX CONCURRENTLY index_namespaces_on_require_two_factor_authentication ON public.namespaces USING btree (require_two_factor_authentication); -- table namespaces  
CREATE INDEX CONCURRENTLY index_packages_dependency_links_on_dependency_id ON public.packages_dependency_links USING btree (dependency_id); -- table packages_dependency_links  
CREATE INDEX CONCURRENTLY index_pages_domains_need_auto_ssl_renewal ON public.pages_domains USING btree (certificate_source, certificate_valid_not_after) WHERE (auto_ssl_enabled = true); -- table pages_domains  
CREATE INDEX CONCURRENTLY index_pages_domains_on_verified_at_and_enabled_until ON public.pages_domains USING btree (verified_at, enabled_until); -- table pages_domains  
CREATE INDEX CONCURRENTLY index_project_custom_attributes_on_key_and_value ON public.project_custom_attributes USING btree (key, value); -- table project_custom_attributes  
CREATE INDEX CONCURRENTLY index_project_mirror_data_on_last_successful_update_at ON public.project_mirror_data USING btree (last_successful_update_at); -- table project_mirror_data  
CREATE INDEX CONCURRENTLY index_project_repositories_on_shard_id ON public.project_repositories USING btree (shard_id); -- table project_repositories  
CREATE INDEX CONCURRENTLY idx_repository_states_on_last_repository_verification_ran_at ON public.project_repository_states USING btree (project_id, last_repository_verification_ran_at) WHERE ((repository_verification_checksum IS NOT NULL) AND (last_repository_verification_failure IS NULL)); -- table project_repository_states  
CREATE INDEX CONCURRENTLY idx_repository_states_on_last_wiki_verification_ran_at ON public.project_repository_states USING btree (project_id, last_wiki_verification_ran_at) WHERE ((wiki_verification_checksum IS NOT NULL) AND (last_wiki_verification_failure IS NULL)); -- table project_repository_states  
CREATE INDEX CONCURRENTLY idx_repository_states_on_repository_failure_partial ON public.project_repository_states USING btree (last_repository_verification_failure) WHERE (last_repository_verification_failure IS NOT NULL); -- table project_repository_states  
CREATE INDEX CONCURRENTLY idx_repository_states_on_wiki_failure_partial ON public.project_repository_states USING btree (last_wiki_verification_failure) WHERE (last_wiki_verification_failure IS NOT NULL); -- table project_repository_states  
CREATE INDEX CONCURRENTLY idx_repository_states_outdated_checksums ON public.project_repository_states USING btree (project_id) WHERE (((repository_verification_checksum IS NULL) AND (last_repository_verification_failure IS NULL)) OR ((wiki_verification_checksum IS NULL) AND (last_wiki_verification_failure IS NULL))); -- table project_repository_states  
CREATE INDEX CONCURRENTLY idx_project_repository_check_partial ON public.projects USING btree (repository_storage, created_at) WHERE (last_repository_check_at IS NULL); -- table projects  
CREATE INDEX CONCURRENTLY index_on_projects_lower_path ON public.projects USING btree (lower((path)::text)); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_api_vis20_name ON public.projects USING btree (name, id) WHERE (visibility_level = 20); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_api_vis20_path ON public.projects USING btree (path, id) WHERE (visibility_level = 20); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_api_vis20_updated_at ON public.projects USING btree (updated_at, id) WHERE (visibility_level = 20); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_on_last_repository_check_at ON public.projects USING btree (last_repository_check_at) WHERE (last_repository_check_at IS NOT NULL); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_on_last_repository_check_failed ON public.projects USING btree (last_repository_check_failed); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_on_mirror_and_mirror_trigger_builds_both_true ON public.projects USING btree (id) WHERE ((mirror IS TRUE) AND (mirror_trigger_builds IS TRUE)); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_on_runners_token ON public.projects USING btree (runners_token); -- table projects  
CREATE INDEX CONCURRENTLY index_prometheus_metrics_on_group ON public.prometheus_metrics USING btree ("group"); -- table prometheus_metrics  
CREATE INDEX CONCURRENTLY index_remote_mirrors_on_last_successful_update_at ON public.remote_mirrors USING btree (last_successful_update_at); -- table remote_mirrors  
CREATE INDEX CONCURRENTLY idx_security_scans_on_scan_type ON public.security_scans USING btree (scan_type); -- table security_scans  
CREATE INDEX CONCURRENTLY index_term_agreements_on_term_id ON public.term_agreements USING btree (term_id); -- table term_agreements  
CREATE INDEX CONCURRENTLY index_timelogs_on_user_id ON public.timelogs USING btree (user_id); -- table timelogs  
CREATE INDEX CONCURRENTLY index_user_custom_attributes_on_key_and_value ON public.user_custom_attributes USING btree (key, value); -- table user_custom_attributes  
CREATE INDEX CONCURRENTLY index_users_on_accepted_term_id ON public.users USING btree (accepted_term_id); -- table users  
CREATE INDEX CONCURRENTLY index_x509_certificates_on_x509_issuer_id ON public.x509_certificates USING btree (x509_issuer_id); -- table x509_certificates  
CREATE INDEX CONCURRENTLY index_x509_commit_signatures_on_x509_certificate_id ON public.x509_commit_signatures USING btree (x509_certificate_id); -- table x509_commit_signatures  
CREATE INDEX CONCURRENTLY index_zoom_meetings_on_issue_status ON public.zoom_meetings USING btree (issue_status); -- table zoom_meetings

  Table of contents

H003 Non-indexed Foreign Keys

Observations

Data collected: 2020-03-09 20:05:55 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

Num Schema name Table name FK name Issue Table mb writes Table scans Parent name Parent mb Parent writes Cols list Indexdef
1 public design_management_versions fk_c1440b4896 no index 0 6372 25282 users 2460 121241830 [author_id]
2 public ci_builds fk_6661f4f0e8 questionable index 457308 509883124 50 ci_resource_groups 1 11393 [resource_group_id] CREATE INDEX index_for_resource_group ON public.ci_builds USING btree (resource_group_id, id) WHERE (resource_group_id IS NOT NULL)
3 public ci_builds fk_87f4cefcda questionable index 457308 509883124 50 ci_pipelines 24742 134106762 [upstream_pipeline_id] CREATE INDEX index_ci_builds_on_upstream_pipeline_id ON public.ci_builds USING btree (upstream_pipeline_id) WHERE (upstream_pipeline_id IS NOT NULL)
4 public ci_builds fk_d3130c9a7f questionable index 457308 509883124 50 ci_pipelines 24742 134106762 [commit_id] CREATE INDEX index_ci_builds_on_commit_id_and_type_and_name_and_ref ON public.ci_builds USING btree (commit_id, type, name, ref)
5 public ci_builds fk_d3130c9a7f questionable index 457308 509883124 50 ci_pipelines 24742 134106762 [commit_id] CREATE INDEX index_ci_builds_on_commit_id_and_status_and_type ON public.ci_builds USING btree (commit_id, status, type)
6 public ci_builds fk_d3130c9a7f questionable index 457308 509883124 50 ci_pipelines 24742 134106762 [commit_id] CREATE INDEX index_ci_builds_on_commit_id_and_stage_idx_and_created_at ON public.ci_builds USING btree (commit_id, stage_idx, created_at)
7 public ci_builds fk_d3130c9a7f questionable index 457308 509883124 50 ci_pipelines 24742 134106762 [commit_id] CREATE INDEX index_ci_builds_on_commit_id_and_artifacts_expireatandidpartial ON public.ci_builds USING btree (commit_id, artifacts_expire_at, id) WHERE (((type)::text = 'Ci::Build'::text) AND ((retried = false) OR (retried IS NULL)) AND ((name)::text = ANY ((ARRAY['sast'::character varying, 'dependency_scanning'::character varying, 'sast:container'::character varying, 'container_scanning'::character varying, 'dast'::character varying])::text[])))
8 public ci_builds fk_d3130c9a7f questionable index 457308 509883124 50 ci_pipelines 24742 134106762 [commit_id] CREATE INDEX index_ci_builds_on_commit_id_and_type_and_ref ON public.ci_builds USING btree (commit_id, type, ref)
9 public events fk_61fbf6ca48 questionable index 39991 154611736 0 namespaces 916 11994103 [group_id] CREATE INDEX index_events_on_group_id_partial ON public.events USING btree (group_id) WHERE (group_id IS NOT NULL)
10 public ci_pipelines fk_190998ef09 questionable index 24742 134106762 14 external_pull_requests 12 397165 [external_pull_request_id] CREATE INDEX index_ci_pipelines_on_external_pull_request_id ON public.ci_pipelines USING btree (external_pull_request_id) WHERE (external_pull_request_id IS NOT NULL)
11 public ci_pipelines fk_a23be95014 questionable index 24742 134106762 14 merge_requests 19737 234406294 [merge_request_id] CREATE INDEX index_ci_pipelines_on_merge_request_id ON public.ci_pipelines USING btree (merge_request_id) WHERE (merge_request_id IS NOT NULL)
12 public merge_requests fk_641731faff questionable index 19737 234406294 64 users 2460 121241830 [updated_by_id] CREATE INDEX index_merge_requests_on_updated_by_id ON public.merge_requests USING btree (updated_by_id) WHERE (updated_by_id IS NOT NULL)
13 public merge_requests fk_ad525e1f87 questionable index 19737 234406294 64 users 2460 121241830 [merge_user_id] CREATE INDEX index_merge_requests_on_merge_user_id ON public.merge_requests USING btree (merge_user_id) WHERE (merge_user_id IS NOT NULL)
14 public issues fk_9c4516d665 questionable index 16585 53720375 21709 issues 16585 53720375 [duplicated_to_id] CREATE INDEX index_issues_on_duplicated_to_id ON public.issues USING btree (duplicated_to_id) WHERE (duplicated_to_id IS NOT NULL)
15 public issues fk_a194299be1 questionable index 16585 53720375 21709 issues 16585 53720375 [moved_to_id] CREATE INDEX index_issues_on_moved_to_id ON public.issues USING btree (moved_to_id) WHERE (moved_to_id IS NOT NULL)
16 public issues fk_df75a7c8b8 questionable index 16585 53720375 21709 epics 11 457317 [promoted_to_epic_id] CREATE INDEX index_issues_on_promoted_to_epic_id ON public.issues USING btree (promoted_to_epic_id) WHERE (promoted_to_epic_id IS NOT NULL)
17 public issues fk_ffed080f01 questionable index 16585 53720375 21709 users 2460 121241830 [updated_by_id] CREATE INDEX index_issues_on_updated_by_id ON public.issues USING btree (updated_by_id) WHERE (updated_by_id IS NOT NULL)
18 public system_note_metadata fk_fbd87415c9 questionable index 10744 45084177 0 description_versions 50 66113 [description_version_id] CREATE UNIQUE INDEX index_system_note_metadata_on_description_version_id ON public.system_note_metadata USING btree (description_version_id) WHERE (description_version_id IS NOT NULL)
19 public projects fk_0a31cca0b8 questionable index 3491 147427957 91880 users 2460 121241830 [marked_for_deletion_by_user_id] CREATE INDEX index_projects_on_marked_for_deletion_by_user_id ON public.projects USING btree (marked_for_deletion_by_user_id) WHERE (marked_for_deletion_by_user_id IS NOT NULL)
20 public projects fk_6e5c14658a questionable index 3491 147427957 91880 pool_repositories 3 61235 [pool_repository_id] CREATE INDEX index_projects_on_pool_repository_id ON public.projects USING btree (pool_repository_id) WHERE (pool_repository_id IS NOT NULL)

Conclusions

Recommendations

  Table of contents

H004 Redundant Indexes

Observations

Data collected: 2020-03-09 20:07:33 +0000 UTC
Current database: gitlabhq_production

Stats reset: 4 mons 12 days 03:58:00 ago (2019-10-28 16:07:35 +0000 UTC)

The list is limited to 20 items. Total: 48.

# Table Index Redundant to 10.220.16.106 usage 10.220.16.101 usage 10.220.16.102 usage 10.220.16.103 usage 10.220.16.104 usage 10.220.16.105 usage 10.220.16.107 usage 10.220.16.108 usage 10.220.16.109 usage ▼ Index size Table size Supports FK
  =====TOTAL===== 28.03 GiB 80.80 GiB
1 taggings index_taggings_on_taggable_id_and_taggable_type index_taggings_on_taggable_id_and_taggable_type_and_context
 0 7.22 GiB 11.92 GiB
2 ci_stages index_ci_stages_on_pipeline_id index_ci_stages_on_pipeline_id_and_name
index_ci_stages_on_pipeline_id_and_position
 0 5.65 GiB 17.71 GiB Yes
3 taggings index_taggings_on_tag_id taggings_idx
 0 4.68 GiB 11.92 GiB
4 projects index_projects_api_last_activity_at_id_desc index_projects_on_last_activity_at_and_id
 0 2.37 GiB 3.41 GiB
5 deployments index_deployments_on_project_id_and_status index_deployments_on_project_id_and_status_and_created_at
 0 1.53 GiB 8.47 GiB
6 projects index_projects_api_path_id_desc index_projects_on_path_and_id
 0 658.43 MiB 3.41 GiB
7 projects index_projects_api_name_id_desc index_projects_on_name_and_id
 0 655.79 MiB 3.41 GiB
8 projects index_on_projects_path index_projects_api_path_id_desc
index_projects_on_path_and_id
index_projects_on_runners_token
 0 651.83 MiB 3.41 GiB Yes
9 projects index_projects_api_updated_at_id_desc index_projects_on_updated_at_and_id
 0 646.18 MiB 3.41 GiB
10 projects index_projects_api_created_at_id_desc index_projects_on_created_at_and_id
 0 591.55 MiB 3.41 GiB
11 notification_settings index_notification_settings_on_user_id index_notifications_on_user_id_and_source_id_and_source_type
 0 548.86 MiB 1.30 GiB
12 issue_metrics index_issue_metrics index_issue_metrics_on_issue_id_and_timestamps
 0 541.68 MiB 1.21 GiB
13 users index_users_on_email_trigram index_users_on_username_trigram
 0 448.95 MiB 2.41 GiB
14 ci_runners index_ci_runners_on_token index_ci_runners_on_token_encrypted
 0 433.33 MiB 371.92 MiB
15 merge_request_assignees index_merge_request_assignees_on_merge_request_id index_merge_request_assignees_on_merge_request_id_and_user_id
 0 406.75 MiB 558.23 MiB
16 ci_pipelines_config index_ci_pipelines_config_on_pipeline_id ci_pipelines_config_pkey
 0 362.98 MiB 1.20 GiB
17 namespaces index_namespaces_on_path index_namespaces_on_runners_token
index_namespaces_on_runners_token_encrypted
 0 262.22 MiB 915.59 MiB Yes
18 personal_access_tokens index_personal_access_tokens_on_user_id index_pat_on_user_id_and_expires_at
 0 125.54 MiB 623.96 MiB
19 term_agreements index_term_agreements_on_user_id term_agreements_unique_index
 0 94.90 MiB 239.47 MiB
20 vulnerabilities index_vulnerabilities_on_milestone_id index_vulnerabilities_on_resolved_by_id
 0 73.40 MiB 355.90 MiB

Conclusions

  • [P3] 48 redundant index(es) have been found.

Recommendations

  • [P3] Use the database migration provided below to drop the redundant indexes. Keep in mind, that under load, it is recommended to use DROP INDEX CONCURRENTLY (and CREATE INDEX CONCURRENTLY if reverting is needed) to avoid blocking issues.
  • Be careful dropping the indexes. If you have multiple setups of your software, the analysis of just a single setup might be not enough. Some indexes might be used (and therefore, needed) only on a limited number of setups. Also, in some cases, developers prepare indexes for new features in advance – in such cases, dropping those indexes is not a good idea.
  • If there are some doubts, consider a more careful approach. Before actualdropping, indexes disable listed in this report. For this, use queries like `UPDATE pg_index SET indisvalid = false WHERE indexrelid::regclass = (select oid from pg_class where relname = 'u_users_email');. Indexes will continue to get updates. In case of some performance degradations, re-enable the corresponding indexes, setting indisvalid to true. If everything looks fine, after a significant period of observations, proceed with DROP INDEX CONCURRENTLY.
  • "DO" database migrations
DROP INDEX CONCURRENTLY index_analytics_ca_group_stages_on_group_id; -- 8192 bytes, index_analytics_ca_group_stages_on_group_id_and_name, table analytics_cycle_analytics_group_stages  
DROP INDEX CONCURRENTLY index_analytics_ca_project_stages_on_project_id; -- 8192 bytes, index_analytics_ca_project_stages_on_project_id_and_name, table analytics_cycle_analytics_project_stages  
DROP INDEX CONCURRENTLY index_approval_rules_code_owners_rule_type; -- 6240 kB, index_approval_rule_name_for_code_owners_rule_type, table approval_merge_request_rules  
DROP INDEX CONCURRENTLY index_aws_roles_on_user_id; -- 40 kB, aws_roles_pkey, table aws_roles  
DROP INDEX CONCURRENTLY index_board_group_recent_visits_on_user_id; -- 1056 kB, index_board_group_recent_visits_on_user_group_and_board, table board_group_recent_visits  
DROP INDEX CONCURRENTLY index_board_project_recent_visits_on_user_id; -- 3768 kB, index_board_project_recent_visits_on_user_project_and_board, table board_project_recent_visits  
DROP INDEX CONCURRENTLY index_ci_pipelines_config_on_pipeline_id; -- 363 MB, ci_pipelines_config_pkey, table ci_pipelines_config  
DROP INDEX CONCURRENTLY index_ci_runners_on_token; -- 433 MB, index_ci_runners_on_token_encrypted, table ci_runners  
DROP INDEX CONCURRENTLY index_ci_stages_on_pipeline_id; -- 5784 MB, index_ci_stages_on_pipeline_id_and_name, index_ci_stages_on_pipeline_id_and_position, table ci_stages  
DROP INDEX CONCURRENTLY index_clusters_kubernetes_namespaces_on_cluster_id; -- 1784 kB, index_kubernetes_namespaces_on_cluster_project_environment_id, kubernetes_namespaces_cluster_and_namespace, table clusters_kubernetes_namespaces  
DROP INDEX CONCURRENTLY index_container_repositories_on_project_id; -- 21 MB, index_container_repositories_on_project_id_and_name, table container_repositories  
DROP INDEX CONCURRENTLY index_deployments_on_project_id_and_status; -- 1564 MB, index_deployments_on_project_id_and_status_and_created_at, table deployments  
DROP INDEX CONCURRENTLY index_design_management_designs_versions_on_design_id; -- 144 kB, design_management_designs_versions_uniqueness, table design_management_designs_versions  
DROP INDEX CONCURRENTLY index_geo_node_namespace_links_on_geo_node_id; -- 8192 bytes, index_geo_node_namespace_links_on_geo_node_id_and_namespace_id, table geo_node_namespace_links  
DROP INDEX CONCURRENTLY index_issue_links_on_source_id; -- 24 MB, index_issue_links_on_source_id_and_target_id, table issue_links  
DROP INDEX CONCURRENTLY index_issue_metrics; -- 542 MB, index_issue_metrics_on_issue_id_and_timestamps, table issue_metrics  
DROP INDEX CONCURRENTLY idx_jira_connect_subscriptions_on_installation_id; -- 16 kB, idx_jira_connect_subscriptions_on_installation_id_namespace_id, table jira_connect_subscriptions  
DROP INDEX CONCURRENTLY index_list_user_preferences_on_user_id; -- 7096 kB, index_list_user_preferences_on_user_id_and_list_id, table list_user_preferences  
DROP INDEX CONCURRENTLY index_merge_request_assignees_on_merge_request_id; -- 407 MB, index_merge_request_assignees_on_merge_request_id_and_user_id, table merge_request_assignees  
DROP INDEX CONCURRENTLY idx_mr_cc_diff_files_on_mr_cc_id; -- 8192 bytes, idx_mr_cc_diff_files_on_mr_cc_id_and_sha, table merge_request_context_commit_diff_files  
DROP INDEX CONCURRENTLY index_merge_request_context_commits_on_merge_request_id; -- 8192 bytes, index_mr_context_commits_on_merge_request_id_and_sha, table merge_request_context_commits  
DROP INDEX CONCURRENTLY index_namespace_aggregation_schedules_on_namespace_id; -- 15 MB, namespace_aggregation_schedules_pkey, table namespace_aggregation_schedules  
DROP INDEX CONCURRENTLY index_namespace_root_storage_statistics_on_namespace_id; -- 34 MB, namespace_root_storage_statistics_pkey, table namespace_root_storage_statistics  
DROP INDEX CONCURRENTLY index_namespaces_on_path; -- 262 MB, index_namespaces_on_runners_token, index_namespaces_on_runners_token_encrypted, table namespaces  
DROP INDEX CONCURRENTLY index_notification_settings_on_user_id; -- 549 MB, index_notifications_on_user_id_and_source_id_and_source_type, table notification_settings  
DROP INDEX CONCURRENTLY index_packages_tags_on_package_id; -- 56 kB, index_packages_tags_on_package_id_and_updated_at, table packages_tags  
DROP INDEX CONCURRENTLY index_pages_domains_on_project_id; -- 3712 kB, index_pages_domains_on_project_id_and_enabled_until, table pages_domains  
DROP INDEX CONCURRENTLY index_pages_domains_on_verified_at; -- 4696 kB, index_pages_domains_on_verified_at_and_enabled_until, table pages_domains  
DROP INDEX CONCURRENTLY index_personal_access_tokens_on_user_id; -- 126 MB, index_pat_on_user_id_and_expires_at, table personal_access_tokens  
DROP INDEX CONCURRENTLY index_project_feature_usages_on_project_id; -- 11 MB, project_feature_usages_pkey, table project_feature_usages  
DROP INDEX CONCURRENTLY index_on_projects_path; -- 652 MB, index_projects_api_path_id_desc, index_projects_on_path_and_id, index_projects_on_runners_token, table projects  
DROP INDEX CONCURRENTLY index_projects_api_created_at_id_desc; -- 592 MB, index_projects_on_created_at_and_id, table projects  
DROP INDEX CONCURRENTLY index_projects_api_last_activity_at_id_desc; -- 2418 MB, index_projects_on_last_activity_at_and_id, table projects  
DROP INDEX CONCURRENTLY index_projects_api_name_id_desc; -- 656 MB, index_projects_on_name_and_id, table projects  
DROP INDEX CONCURRENTLY index_projects_api_path_id_desc; -- 658 MB, index_projects_on_path_and_id, table projects  
DROP INDEX CONCURRENTLY index_projects_api_updated_at_id_desc; -- 646 MB, index_projects_on_updated_at_and_id, table projects  
DROP INDEX CONCURRENTLY index_protected_environments_on_project_id; -- 208 kB, index_protected_environments_on_project_id_and_name, table protected_environments  
DROP INDEX CONCURRENTLY index_protected_tags_on_project_id; -- 728 kB, index_protected_tags_on_project_id_and_name, table protected_tags  
DROP INDEX CONCURRENTLY index_status_page_settings_on_project_id; -- 8192 bytes, status_page_settings_pkey, table status_page_settings  
DROP INDEX CONCURRENTLY index_taggings_on_tag_id; -- 4791 MB, taggings_idx, table taggings  
DROP INDEX CONCURRENTLY index_taggings_on_taggable_id_and_taggable_type; -- 7392 MB, index_taggings_on_taggable_id_and_taggable_type_and_context, table taggings  
DROP INDEX CONCURRENTLY index_term_agreements_on_user_id; -- 95 MB, term_agreements_unique_index, table term_agreements  
DROP INDEX CONCURRENTLY index_user_callouts_on_user_id; -- 35 MB, index_user_callouts_on_user_id_and_feature_name, table user_callouts  
DROP INDEX CONCURRENTLY index_user_details_on_user_id; -- 8192 bytes, user_details_pkey, table user_details  
DROP INDEX CONCURRENTLY index_user_statuses_on_user_id; -- 2936 kB, user_statuses_pkey, table user_statuses  
DROP INDEX CONCURRENTLY index_users_on_email_trigram; -- 449 MB, index_users_on_username_trigram, table users  
DROP INDEX CONCURRENTLY index_vulnerabilities_on_epic_id; -- 73 MB, index_vulnerabilities_on_confirmed_by_id, index_vulnerabilities_on_dismissed_by_id, table vulnerabilities  
DROP INDEX CONCURRENTLY index_vulnerabilities_on_milestone_id; -- 73 MB, index_vulnerabilities_on_resolved_by_id, table vulnerabilities
  • "UNDO" database migrations
CREATE INDEX CONCURRENTLY index_analytics_ca_group_stages_on_group_id ON public.analytics_cycle_analytics_group_stages USING btree (group_id); -- table analytics_cycle_analytics_group_stages  
CREATE INDEX CONCURRENTLY index_analytics_ca_project_stages_on_project_id ON public.analytics_cycle_analytics_project_stages USING btree (project_id); -- table analytics_cycle_analytics_project_stages  
CREATE INDEX CONCURRENTLY index_approval_rules_code_owners_rule_type ON public.approval_merge_request_rules USING btree (merge_request_id, rule_type) WHERE (rule_type = 2); -- table approval_merge_request_rules  
CREATE UNIQUE INDEX index_aws_roles_on_user_id ON public.aws_roles USING btree (user_id); -- table aws_roles  
CREATE INDEX CONCURRENTLY index_board_group_recent_visits_on_user_id ON public.board_group_recent_visits USING btree (user_id); -- table board_group_recent_visits  
CREATE INDEX CONCURRENTLY index_board_project_recent_visits_on_user_id ON public.board_project_recent_visits USING btree (user_id); -- table board_project_recent_visits  
CREATE INDEX CONCURRENTLY index_ci_pipelines_config_on_pipeline_id ON public.ci_pipelines_config USING btree (pipeline_id); -- table ci_pipelines_config  
CREATE INDEX CONCURRENTLY index_ci_runners_on_token ON public.ci_runners USING btree (token); -- table ci_runners  
CREATE INDEX CONCURRENTLY index_ci_stages_on_pipeline_id ON public.ci_stages USING btree (pipeline_id); -- table ci_stages  
CREATE INDEX CONCURRENTLY index_clusters_kubernetes_namespaces_on_cluster_id ON public.clusters_kubernetes_namespaces USING btree (cluster_id); -- table clusters_kubernetes_namespaces  
CREATE INDEX CONCURRENTLY index_container_repositories_on_project_id ON public.container_repositories USING btree (project_id); -- table container_repositories  
CREATE INDEX CONCURRENTLY index_deployments_on_project_id_and_status ON public.deployments USING btree (project_id, status); -- table deployments  
CREATE INDEX CONCURRENTLY index_design_management_designs_versions_on_design_id ON public.design_management_designs_versions USING btree (design_id); -- table design_management_designs_versions  
CREATE INDEX CONCURRENTLY index_geo_node_namespace_links_on_geo_node_id ON public.geo_node_namespace_links USING btree (geo_node_id); -- table geo_node_namespace_links  
CREATE INDEX CONCURRENTLY index_issue_links_on_source_id ON public.issue_links USING btree (source_id); -- table issue_links  
CREATE INDEX CONCURRENTLY index_issue_metrics ON public.issue_metrics USING btree (issue_id); -- table issue_metrics  
CREATE INDEX CONCURRENTLY idx_jira_connect_subscriptions_on_installation_id ON public.jira_connect_subscriptions USING btree (jira_connect_installation_id); -- table jira_connect_subscriptions  
CREATE INDEX CONCURRENTLY index_list_user_preferences_on_user_id ON public.list_user_preferences USING btree (user_id); -- table list_user_preferences  
CREATE INDEX CONCURRENTLY index_merge_request_assignees_on_merge_request_id ON public.merge_request_assignees USING btree (merge_request_id); -- table merge_request_assignees  
CREATE INDEX CONCURRENTLY idx_mr_cc_diff_files_on_mr_cc_id ON public.merge_request_context_commit_diff_files USING btree (merge_request_context_commit_id); -- table merge_request_context_commit_diff_files  
CREATE INDEX CONCURRENTLY index_merge_request_context_commits_on_merge_request_id ON public.merge_request_context_commits USING btree (merge_request_id); -- table merge_request_context_commits  
CREATE UNIQUE INDEX index_namespace_aggregation_schedules_on_namespace_id ON public.namespace_aggregation_schedules USING btree (namespace_id); -- table namespace_aggregation_schedules  
CREATE UNIQUE INDEX index_namespace_root_storage_statistics_on_namespace_id ON public.namespace_root_storage_statistics USING btree (namespace_id); -- table namespace_root_storage_statistics  
CREATE INDEX CONCURRENTLY index_namespaces_on_path ON public.namespaces USING btree (path); -- table namespaces  
CREATE INDEX CONCURRENTLY index_notification_settings_on_user_id ON public.notification_settings USING btree (user_id); -- table notification_settings  
CREATE INDEX CONCURRENTLY index_packages_tags_on_package_id ON public.packages_tags USING btree (package_id); -- table packages_tags  
CREATE INDEX CONCURRENTLY index_pages_domains_on_project_id ON public.pages_domains USING btree (project_id); -- table pages_domains  
CREATE INDEX CONCURRENTLY index_pages_domains_on_verified_at ON public.pages_domains USING btree (verified_at); -- table pages_domains  
CREATE INDEX CONCURRENTLY index_personal_access_tokens_on_user_id ON public.personal_access_tokens USING btree (user_id); -- table personal_access_tokens  
CREATE INDEX CONCURRENTLY index_project_feature_usages_on_project_id ON public.project_feature_usages USING btree (project_id); -- table project_feature_usages  
CREATE INDEX CONCURRENTLY index_on_projects_path ON public.projects USING btree (path); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_api_created_at_id_desc ON public.projects USING btree (created_at, id DESC); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_api_last_activity_at_id_desc ON public.projects USING btree (last_activity_at, id DESC); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_api_name_id_desc ON public.projects USING btree (name, id DESC); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_api_path_id_desc ON public.projects USING btree (path, id DESC); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_api_updated_at_id_desc ON public.projects USING btree (updated_at, id DESC); -- table projects  
CREATE INDEX CONCURRENTLY index_protected_environments_on_project_id ON public.protected_environments USING btree (project_id); -- table protected_environments  
CREATE INDEX CONCURRENTLY index_protected_tags_on_project_id ON public.protected_tags USING btree (project_id); -- table protected_tags  
CREATE INDEX CONCURRENTLY index_status_page_settings_on_project_id ON public.status_page_settings USING btree (project_id); -- table status_page_settings  
CREATE INDEX CONCURRENTLY index_taggings_on_tag_id ON public.taggings USING btree (tag_id); -- table taggings  
CREATE INDEX CONCURRENTLY index_taggings_on_taggable_id_and_taggable_type ON public.taggings USING btree (taggable_id, taggable_type); -- table taggings  
CREATE INDEX CONCURRENTLY index_term_agreements_on_user_id ON public.term_agreements USING btree (user_id); -- table term_agreements  
CREATE INDEX CONCURRENTLY index_user_callouts_on_user_id ON public.user_callouts USING btree (user_id); -- table user_callouts  
CREATE UNIQUE INDEX index_user_details_on_user_id ON public.user_details USING btree (user_id); -- table user_details  
CREATE INDEX CONCURRENTLY index_user_statuses_on_user_id ON public.user_statuses USING btree (user_id); -- table user_statuses  
CREATE INDEX CONCURRENTLY index_users_on_email_trigram ON public.users USING gin (email gin_trgm_ops); -- table users  
CREATE INDEX CONCURRENTLY index_vulnerabilities_on_epic_id ON public.vulnerabilities USING btree (epic_id); -- table vulnerabilities  
CREATE INDEX CONCURRENTLY index_vulnerabilities_on_milestone_id ON public.vulnerabilities USING btree (milestone_id); -- table vulnerabilities

  Table of contents

K001 Globally Aggregated Query Metrics

Observations

Data collected: 2020-03-09 20:07:34 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

Start: 2020-03-09T19:33:07.398492+00:00
End: 2020-03-09T20:05:57.175736+00:00
Period seconds: 1969.77725
Period age: 00:32:49.777244

Error (calls): 0.00 (0.00%)
Error (total time): 0.00 (0.00%)

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
30,773,310
15.63K/sec
1.00/call
100.00%		 5,729,668.57 ms
2908.790 ms/sec
0.186 ms/call
100.00%		 242,454,418
123.09K/sec
7.88/call
100.00%		 1,936,365,456 blks
983.04K blks/sec
62.92 blks/call
100.00%		 1,890,174 blks
959.59 blks/sec
0.06 blks/call
100.00%		 2,284,498 blks
1.16K blks/sec
0.07 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 492,094.55 ms
249.822 ms/sec
0.016 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica servers:

Replica (10.220.16.101)

Start: 2020-03-09T19:33:02.038241+00:00
End: 2020-03-09T20:03:20.749474+00:00
Period seconds: 1818.71123
Period age: 00:30:18.711233

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
6,940,595
3.82K/sec
1.00/call
100.00%		 1,719,345.17 ms
945.365 ms/sec
0.248 ms/call
100.00%		 12,920,197
7.11K/sec
1.86/call
100.00%		 956,874,351 blks
526.13K blks/sec
137.87 blks/call
100.00%		 1,320,355 blks
725.98 blks/sec
0.19 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 68,197.49 ms
37.498 ms/sec
0.010 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.102)

Start: 2020-03-09T19:33:03.110816+00:00
End: 2020-03-09T20:03:47.794977+00:00
Period seconds: 1844.68416
Period age: 00:30:44.684161

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
7,156,025
3.88K/sec
1.00/call
100.00%		 1,704,388.02 ms
923.946 ms/sec
0.238 ms/call
100.00%		 13,470,703
7.31K/sec
1.88/call
100.00%		 935,015,753 blks
506.88K blks/sec
130.66 blks/call
100.00%		 1,499,972 blks
813.13 blks/sec
0.21 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 89,616.86 ms
48.581 ms/sec
0.013 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.103)

Start: 2020-03-09T19:33:04.1841+00:00
End: 2020-03-09T20:04:15.802677+00:00
Period seconds: 1871.61858
Period age: 00:31:11.618577

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
7,310,654
3.91K/sec
1.00/call
100.00%		 1,859,386.03 ms
993.464 ms/sec
0.254 ms/call
100.00%		 13,480,476
7.21K/sec
1.84/call
100.00%		 998,555,055 blks
533.53K blks/sec
136.59 blks/call
100.00%		 1,681,510 blks
898.43 blks/sec
0.23 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 102,309.61 ms
54.664 ms/sec
0.014 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.104)

Start: 2020-03-09T19:33:05.250185+00:00
End: 2020-03-09T20:04:43.366888+00:00
Period seconds: 1898.1167
Period age: 00:31:38.116703

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
7,390,906
3.90K/sec
1.00/call
100.00%		 1,791,489.55 ms
943.825 ms/sec
0.242 ms/call
100.00%		 13,665,903
7.20K/sec
1.85/call
100.00%		 974,886,828 blks
513.61K blks/sec
131.90 blks/call
100.00%		 1,505,673 blks
793.25 blks/sec
0.20 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 81,421.46 ms
42.896 ms/sec
0.011 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.105)

Start: 2020-03-09T19:33:06.332445+00:00
End: 2020-03-09T20:05:12.23579+00:00
Period seconds: 1925.90335
Period age: 00:32:05.903345

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
7,428,745
3.86K/sec
1.00/call
100.00%		 1,781,431.62 ms
924.985 ms/sec
0.240 ms/call
100.00%		 13,772,873
7.16K/sec
1.85/call
100.00%		 1,028,656,220 blks
534.12K blks/sec
138.47 blks/call
100.00%		 1,680,266 blks
872.46 blks/sec
0.23 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 92,000.94 ms
47.770 ms/sec
0.012 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.107)

Start: 2020-03-09T19:33:08.460544+00:00
End: 2020-03-09T20:06:36.371713+00:00
Period seconds: 2007.91117
Period age: 00:33:27.911169

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
7,323,953
3.65K/sec
1.00/call
100.00%		 2,112,573.66 ms
1052.125 ms/sec
0.288 ms/call
100.00%		 12,381,374
6.17K/sec
1.69/call
100.00%		 1,227,660,387 blks
611.42K blks/sec
167.62 blks/call
100.00%		 2,235,844 blks
1.12K blks/sec
0.31 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 402,557.78 ms
200.486 ms/sec
0.055 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.108)

Start: 2020-03-09T19:33:09.531386+00:00
End: 2020-03-09T20:07:04.795741+00:00
Period seconds: 2035.26435
Period age: 00:33:55.264355

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
7,905,338
3.89K/sec
1.00/call
100.00%		 2,033,770.00 ms
999.266 ms/sec
0.257 ms/call
100.00%		 14,478,218
7.12K/sec
1.83/call
100.00%		 1,097,346,999 blks
539.17K blks/sec
138.81 blks/call
100.00%		 1,679,451 blks
825.18 blks/sec
0.21 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 122,510.54 ms
60.194 ms/sec
0.015 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.109)

Start: 2020-03-09T19:33:10.608089+00:00
End: 2020-03-09T20:07:35.177649+00:00
Period seconds: 2064.56956
Period age: 00:34:24.56956

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
8,040,350
3.90K/sec
1.00/call
100.00%		 2,137,402.66 ms
1035.278 ms/sec
0.266 ms/call
100.00%		 14,941,521
7.24K/sec
1.86/call
100.00%		 1,072,475,487 blks
519.47K blks/sec
133.39 blks/call
100.00%		 1,876,300 blks
908.81 blks/sec
0.23 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 212,542.91 ms
102.948 ms/sec
0.026 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Conclusions

Recommendations

  Table of contents

K002 Workload Type ("The First Word" Analysis)

Observations

Data collected: 2020-03-09 20:07:34 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

Start: 2020-03-09T19:33:07.398492+00:00
End: 2020-03-09T20:05:57.175736+00:00
Period seconds: 1969.77725
Period age: 00:32:49.777244

Error (calls): 0.00 (0.00%)
Error (total time): 0.00 (0.00%)

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 22,662,163
11.51K/sec
1.00/call
73.64%		 3,978,311.24 ms
2019.676 ms/sec
0.176 ms/call
69.43%		 142,963,062
72.58K/sec
6.31/call
58.96%		 1,464,934,785 blks
743.71K blks/sec
64.64 blks/call
75.65%		 1,061,275 blks
538.78 blks/sec
0.05 blks/call
56.15%		 57,666 blks
29.28 blks/sec
0.00 blks/call
2.52%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 305,100.73 ms
154.891 ms/sec
0.013 ms/call
62.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 with 193,694
98.33/sec
1.00/call
0.63%		 907,334.18 ms
460.628 ms/sec
4.684 ms/call
15.84%		 98,151,138
49.83K/sec
506.73/call
40.48%		 418,351,274 blks
212.39K blks/sec
2.16K blks/call
21.60%		 74,044 blks
37.59 blks/sec
0.38 blks/call
3.92%		 3,907 blks
1.98 blks/sec
0.02 blks/call
0.17%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 30,977.75 ms
15.727 ms/sec
0.160 ms/call
6.30%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3 update 994,984
505.13/sec
1.00/call
3.23%		 550,961.53 ms
279.708 ms/sec
0.554 ms/call
9.62%		 868,196
440.76/sec
0.87/call
0.36%		 38,966,699 blks
19.79K blks/sec
39.16 blks/call
2.01%		 561,056 blks
284.83 blks/sec
0.56 blks/call
29.68%		 1,687,642 blks
856.77 blks/sec
1.70 blks/call
73.87%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 57,286.55 ms
29.083 ms/sec
0.058 ms/call
11.64%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4 insert 304,165
154.42/sec
1.00/call
0.99%		 199,530.74 ms
101.296 ms/sec
0.656 ms/call
3.48%		 454,405
230.69/sec
1.49/call
0.19%		 14,022,570 blks
7.12K blks/sec
46.10 blks/call
0.72%		 192,462 blks
97.71 blks/sec
0.63 blks/call
10.18%		 530,540 blks
269.34 blks/sec
1.74 blks/call
23.22%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 98,638.35 ms
50.076 ms/sec
0.324 ms/call
20.04%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5 set 6,600,687
3.36K/sec
1.00/call
21.45%		 92,047.60 ms
46.730 ms/sec
0.014 ms/call
1.61%		 0
0.00/sec
0.00/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6 select ... for [no key] update 17,617
8.94/sec
1.00/call
0.06%		 1,483.27 ms
0.753 ms/sec
0.084 ms/call
0.03%		 17,617
8.94/sec
1.00/call
0.01%		 90,128 blks
45.76 blks/sec
5.12 blks/call
0.00%		 1,337 blks
0.68 blks/sec
0.08 blks/call
0.07%		 4,743 blks
2.41 blks/sec
0.27 blks/call
0.21%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 91.17 ms
0.046 ms/sec
0.005 ms/call
0.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica servers:

Replica (10.220.16.101)

Start: 2020-03-09T19:33:02.038241+00:00
End: 2020-03-09T20:03:20.749474+00:00
Period seconds: 1818.71123
Period age: 00:30:18.711233

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 6,934,309
3.82K/sec
1.00/call
99.91%		 1,715,657.78 ms
943.337 ms/sec
0.247 ms/call
99.79%		 12,913,911
7.11K/sec
1.86/call
99.95%		 956,839,160 blks
526.11K blks/sec
137.99 blks/call
100.00%		 1,313,129 blks
722.01 blks/sec
0.19 blks/call
99.45%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 66,386.29 ms
36.502 ms/sec
0.010 ms/call
97.34%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 with 6,286
3.46/sec
1.00/call
0.09%		 3,687.39 ms
2.027 ms/sec
0.587 ms/call
0.21%		 6,286
3.46/sec
1.00/call
0.05%		 35,191 blks
19.35 blks/sec
5.60 blks/call
0.00%		 7,226 blks
3.97 blks/sec
1.15 blks/call
0.55%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,811.19 ms
0.996 ms/sec
0.288 ms/call
2.66%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.102)

Start: 2020-03-09T19:33:03.110816+00:00
End: 2020-03-09T20:03:47.794977+00:00
Period seconds: 1844.68416
Period age: 00:30:44.684161

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 7,149,624
3.88K/sec
1.00/call
99.91%		 1,700,821.16 ms
922.012 ms/sec
0.238 ms/call
99.79%		 13,464,302
7.30K/sec
1.88/call
99.95%		 934,973,784 blks
506.85K blks/sec
130.77 blks/call
100.00%		 1,491,931 blks
808.77 blks/sec
0.21 blks/call
99.46%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 87,825.70 ms
47.610 ms/sec
0.012 ms/call
98.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 with 6,401
3.47/sec
1.00/call
0.09%		 3,566.86 ms
1.934 ms/sec
0.557 ms/call
0.21%		 6,401
3.47/sec
1.00/call
0.05%		 41,969 blks
22.75 blks/sec
6.56 blks/call
0.00%		 8,041 blks
4.36 blks/sec
1.26 blks/call
0.54%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,791.16 ms
0.971 ms/sec
0.280 ms/call
2.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.103)

Start: 2020-03-09T19:33:04.1841+00:00
End: 2020-03-09T20:04:15.802677+00:00
Period seconds: 1871.61858
Period age: 00:31:11.618577

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 7,304,861
3.91K/sec
1.00/call
99.92%		 1,857,907.82 ms
992.674 ms/sec
0.254 ms/call
99.92%		 13,474,683
7.20K/sec
1.84/call
99.96%		 998,549,262 blks
533.53K blks/sec
136.70 blks/call
100.00%		 1,681,510 blks
898.43 blks/sec
0.23 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 102,309.61 ms
54.664 ms/sec
0.014 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 with 5,793
3.10/sec
1.00/call
0.08%		 1,478.21 ms
0.790 ms/sec
0.255 ms/call
0.08%		 5,793
3.10/sec
1.00/call
0.04%		 5,793 blks
3.10 blks/sec
1.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.104)

Start: 2020-03-09T19:33:05.250185+00:00
End: 2020-03-09T20:04:43.366888+00:00
Period seconds: 1898.1167
Period age: 00:31:38.116703

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 7,385,031
3.90K/sec
1.00/call
99.92%		 1,790,083.66 ms
943.084 ms/sec
0.242 ms/call
99.92%		 13,660,028
7.20K/sec
1.85/call
99.96%		 974,880,953 blks
513.61K blks/sec
132.01 blks/call
100.00%		 1,505,673 blks
793.25 blks/sec
0.20 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 81,421.46 ms
42.896 ms/sec
0.011 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 with 5,875
3.10/sec
1.00/call
0.08%		 1,405.89 ms
0.741 ms/sec
0.239 ms/call
0.08%		 5,875
3.10/sec
1.00/call
0.04%		 5,875 blks
3.10 blks/sec
1.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.105)

Start: 2020-03-09T19:33:06.332445+00:00
End: 2020-03-09T20:05:12.23579+00:00
Period seconds: 1925.90335
Period age: 00:32:05.903345

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 7,422,782
3.86K/sec
1.00/call
99.92%		 1,780,067.82 ms
924.277 ms/sec
0.240 ms/call
99.92%		 13,766,910
7.15K/sec
1.85/call
99.96%		 1,028,650,257 blks
534.12K blks/sec
138.58 blks/call
100.00%		 1,680,266 blks
872.46 blks/sec
0.23 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 92,000.94 ms
47.770 ms/sec
0.012 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 with 5,963
3.10/sec
1.00/call
0.08%		 1,363.80 ms
0.708 ms/sec
0.229 ms/call
0.08%		 5,963
3.10/sec
1.00/call
0.04%		 5,963 blks
3.10 blks/sec
1.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.107)

Start: 2020-03-09T19:33:08.460544+00:00
End: 2020-03-09T20:06:36.371713+00:00
Period seconds: 2007.91117
Period age: 00:33:27.911169

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 6,866,574
3.42K/sec
1.00/call
93.76%		 2,065,082.46 ms
1028.473 ms/sec
0.301 ms/call
97.75%		 11,572,349
5.77K/sec
1.69/call
93.47%		 1,208,438,064 blks
601.84K blks/sec
175.99 blks/call
98.43%		 2,224,943 blks
1.11K blks/sec
0.32 blks/call
99.51%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 398,917.98 ms
198.673 ms/sec
0.058 ms/call
99.10%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 with 457,379
227.79/sec
1.00/call
6.24%		 47,491.20 ms
23.652 ms/sec
0.104 ms/call
2.25%		 809,025
402.92/sec
1.77/call
6.53%		 19,222,323 blks
9.58K blks/sec
42.03 blks/call
1.57%		 10,901 blks
5.43 blks/sec
0.02 blks/call
0.49%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3,639.80 ms
1.813 ms/sec
0.008 ms/call
0.90%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.108)

Start: 2020-03-09T19:33:09.531386+00:00
End: 2020-03-09T20:07:04.795741+00:00
Period seconds: 2035.26435
Period age: 00:33:55.264355

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 7,899,037
3.89K/sec
1.00/call
99.92%		 2,032,000.92 ms
998.397 ms/sec
0.257 ms/call
99.91%		 14,471,917
7.12K/sec
1.83/call
99.96%		 1,097,340,698 blks
539.17K blks/sec
138.92 blks/call
100.00%		 1,679,451 blks
825.18 blks/sec
0.21 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 122,510.54 ms
60.194 ms/sec
0.016 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 with 6,301
3.10/sec
1.00/call
0.08%		 1,769.08 ms
0.869 ms/sec
0.281 ms/call
0.09%		 6,301
3.10/sec
1.00/call
0.04%		 6,301 blks
3.10 blks/sec
1.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.109)

Start: 2020-03-09T19:33:10.608089+00:00
End: 2020-03-09T20:07:35.177649+00:00
Period seconds: 2064.56956
Period age: 00:34:24.56956

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 8,033,957
3.90K/sec
1.00/call
99.92%		 2,135,772.41 ms
1034.488 ms/sec
0.266 ms/call
99.92%		 14,935,128
7.24K/sec
1.86/call
99.96%		 1,072,469,094 blks
519.47K blks/sec
133.49 blks/call
100.00%		 1,876,300 blks
908.81 blks/sec
0.23 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 212,542.91 ms
102.948 ms/sec
0.026 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 with 6,393
3.10/sec
1.00/call
0.08%		 1,630.25 ms
0.790 ms/sec
0.255 ms/call
0.08%		 6,393
3.10/sec
1.00/call
0.04%		 6,393 blks
3.10 blks/sec
1.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Conclusions

Recommendations

  Table of contents

K003 Top-20 Queries by total_time

Observations

Data collected: 2020-03-09 20:07:34 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

Start: 2020-03-09T19:33:07.398492+00:00
End: 2020-03-09T20:05:57.175736+00:00
Period seconds: 1969.77725
Period age: 00:32:49.777244

Error (calls): 0.00 (0.00%)
Error (total time): 0.00 (0.00%)

The list is limited to 20 items.

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(1308830512)		 SELECT ? AS one FROM ((SELECT "projects".* 
FROM "projects" WHERE "projects"."namespace_id" 
IN (WITH RECURSIVE "base_and_descendants" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE "namespaces"."id" 
IN (SELECT "elasticsearch_indexed_namespaces"."namespace_id" 
FROM "elasticsearch_indexed_namespaces"))
UNION
(
SELECT "namespaces".* FROM "namespaces",
 "base_and_descendants" WHERE "namespaces"."parent_id" 
= "base_and_descendants"."id")) 
SELECT "id" FROM "base_and_descendants" 
AS "namespaces"))
UNION
(SELECT 
"projects".* FROM "projects" WHERE 
"projects"."id" IN (SELECT "elasticsearch_indexed_projects"."project_id" 
FROM "elasticsearch_indexed_projects"))) 
projects WHERE "projects"."id" 
= ? LIMIT ?
Full query 		181,602
92.19/sec
1.00/call
0.59%		 2,401,457.81 ms
1219.152 ms/sec
13.224 ms/call
41.91%		 10,415
5.29/sec
0.06/call
0.00%		 1,081,683,922 blks
549.15K blks/sec
5.96K blks/call
55.86%		 4 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,845 blks
0.94 blks/sec
0.01 blks/call
0.08%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.65 ms
0.001 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(777048128)		 WITH RECURSIVE "namespaces_cte" 
AS ((SELECT "namespaces"."id", 
"members"."access_level" FROM "namespaces" 
INNER JOIN "members" ON "namespaces"."id" 
= "members"."source_id" WHERE "members"."type" 
= ? AND "members"."source_type" 
= ? AND "namespaces"."type" = ? 
AND "members"."user_id" = ? AND 
"members"."requested_at" IS NULL)
UNION
(
SELECT "namespaces"."id", LEAST(
"members"."access_level", "group_group_links"."group_access") 
AS access_level FROM "namespaces" 
INNER JOIN "group_group_links" 
ON "group_group_links"."shared_group_id" 
= "namespaces"."id" INNER JOIN 
"members" ON "group_group_links"."shared_with_group_id" 
= "members"."source_id" AND "members"."source_type" 
= ? AND "members"."requested_at" 
IS NULL AND "members"."user_id" 
= ? WHERE "namespaces"."type" = 
?)
UNION
(SELECT "namespaces"."id",
 GREATEST("members"."access_level",
 "namespaces_cte"."access_level") 
AS access_level FROM "namespaces" 
INNER JOIN "namespaces_cte" ON 
"namespaces_cte"."id" = "namespaces"."parent_id" 
LEFT OUTER JO...
Full query 		18,821
9.55/sec
1.00/call
0.06%		 824,470.43 ms
418.560 ms/sec
43.806 ms/call
14.39%		 98,131,371
49.82K/sec
5.22K/call
40.47%		 412,860,293 blks
209.60K blks/sec
21.94K blks/call
21.32%		 5,675 blks
2.88 blks/sec
0.30 blks/call
0.30%		 715 blks
0.36 blks/sec
0.04 blks/call
0.03%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,670.17 ms
0.848 ms/sec
0.089 ms/call
0.34%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(1608875156)		 SELECT (CASE
          WHEN (SELECT 
count(*) FROM "ci_builds" WHERE 
"ci_builds"."stage_id" = ? AND 
("ci_builds"."retried" = ? OR "ci_builds"."retried" 
IS NULL) AND (allow_failure = ? 
OR status IN (?,?,?,?,?,?,?,?)))=(
SELECT count(*) FROM "ci_builds" 
WHERE "ci_builds"."stage_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL) 
AND (allow_failure = ? OR status 
IN (?,?,?,?,?,?,?,?)) AND ("ci_builds"."status" 
IN (?))) AND (SELECT count(*) > 
? FROM "ci_builds" WHERE "ci_builds"."stage_id" 
= ? AND ("ci_builds"."retried" 
= ? OR "ci_builds"."retried" IS 
NULL) AND "ci_builds"."allow_failure" 
= ? AND "ci_builds"."status" IN 
(?, ?)) THEN ?
          WHEN (
SELECT count(*) FROM "ci_builds" 
WHERE "ci_builds"."stage_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL) 
AND (allow_failure = ? OR status 
IN (?,?,?,?,?,?,?,?)))=(SELECT 
count(*) FROM "ci_builds" WHERE 
"ci_builds"."stage_id" = ? AND 
("ci_builds"."retried" = ? OR "ci_builds"."retried" 
IS NULL) AND (al...
Full query 		85,423
43.37/sec
1.00/call
0.28%		 158,673.19 ms
80.554 ms/sec
1.857 ms/call
2.77%		 2,522,897
1.29K/sec
29.53/call
1.04%		 94,744,779 blks
48.10K blks/sec
1.11K blks/call
4.89%		 1,054 blks
0.54 blks/sec
0.01 blks/call
0.06%		 412 blks
0.21 blks/sec
0.00 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 868.82 ms
0.441 ms/sec
0.010 ms/call
0.18%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(3258414343)		 UPDATE "ci_builds" SET "runner_id" 
= ?, "status" = ?, "started_at" 
= ?, "updated_at" = ?, "lock_version" 
= ? WHERE "ci_builds"."type" IN 
(?) AND "ci_builds"."lock_version" 
= ? AND "ci_builds"."id" = ?
Full query 		116,653
59.22/sec
1.00/call
0.38%		 135,831.75 ms
68.958 ms/sec
1.164 ms/call
2.37%		 24,095
12.23/sec
0.21/call
0.01%		 2,871,430 blks
1.46K blks/sec
24.62 blks/call
0.15%		 5,997 blks
3.04 blks/sec
0.05 blks/call
0.32%		 23,016 blks
11.68 blks/sec
0.20 blks/call
1.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2,053.37 ms
1.042 ms/sec
0.018 ms/call
0.42%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(3670659708)		 SELECT "project_authorizations"."project_id",
 "project_authorizations"."access_level" 
FROM "project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? /*application:web,controller:project_members,
action:update,correlation_id:VTSi7gJohi7,
line:/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:235:in 
`mon_synchronize'*/
Full query 		18,821
9.55/sec
1.00/call
0.06%		 120,280.73 ms
61.063 ms/sec
6.391 ms/call
2.10%		 98,115,179
49.82K/sec
5.22K/call
40.47%		 32,326,690 blks
16.42K blks/sec
1.72K blks/call
1.67%		 4,796 blks
2.43 blks/sec
0.25 blks/call
0.25%		 3,351 blks
1.70 blks/sec
0.18 blks/call
0.15%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2,805.66 ms
1.424 ms/sec
0.149 ms/call
0.57%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(3502758514)		 SELECT (CASE
          WHEN (SELECT 
count(*) FROM "ci_builds" WHERE 
"ci_builds"."commit_id" = ? AND 
("ci_builds"."retried" = ? OR "ci_builds"."retried" 
IS NULL) AND (allow_failure = ? 
OR status IN (?,?,?,?,?,?,?,?)))=(
SELECT count(*) FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL) 
AND (allow_failure = ? OR status 
IN (?,?,?,?,?,?,?,?)) AND ("ci_builds"."status" 
IN (?))) AND (SELECT count(*) > 
? FROM "ci_builds" WHERE "ci_builds"."commit_id" 
= ? AND ("ci_builds"."retried" 
= ? OR "ci_builds"."retried" IS 
NULL) AND "ci_builds"."allow_failure" 
= ? AND "ci_builds"."status" IN 
(?, ?)) THEN ?
          WHEN (
SELECT count(*) FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL) 
AND (allow_failure = ? OR status 
IN (?,?,?,?,?,?,?,?)))=(SELECT 
count(*) FROM "ci_builds" WHERE 
"ci_builds"."commit_id" = ? AND 
("ci_builds"."retried" = ? OR "ci_builds"."retried" 
IS NULL) AN...
Full query 		92,102
46.76/sec
1.00/call
0.30%		 83,039.90 ms
42.157 ms/sec
0.902 ms/call
1.45%		 3,410,782
1.74K/sec
37.03/call
1.41%		 53,548,320 blks
27.19K blks/sec
581.40 blks/call
2.77%		 712 blks
0.36 blks/sec
0.01 blks/call
0.04%		 375 blks
0.19 blks/sec
0.00 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 570.54 ms
0.290 ms/sec
0.006 ms/call
0.12%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(2555545475)		 SELECT  "projects".* FROM "projects" 
WHERE "projects"."id" = ? LIMIT 
?
Full query 		1,353,034
686.90/sec
1.00/call
4.40%		 79,425.23 ms
40.322 ms/sec
0.059 ms/call
1.39%		 1,352,646
686.70/sec
1.00/call
0.56%		 5,754,670 blks
2.93K blks/sec
4.25 blks/call
0.30%		 224 blks
0.11 blks/sec
0.00 blks/call
0.01%		 747 blks
0.38 blks/sec
0.00 blks/call
0.03%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 12.98 ms
0.007 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(3220074313)		 UPDATE "project_mirror_data" SET 
"status" = ?, "last_update_started_at" 
= ? WHERE "project_mirror_data"."id" 
= ?
Full query 		115,679
58.73/sec
1.00/call
0.38%		 77,545.57 ms
39.368 ms/sec
0.670 ms/call
1.35%		 115,679
58.73/sec
1.00/call
0.05%		 3,765,131 blks
1.92K blks/sec
32.55 blks/call
0.19%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3,673 blks
1.86 blks/sec
0.03 blks/call
0.16%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(1372870422)		 INSERT INTO "notes" ("note", "noteable_type",
 "author_id", "created_at", "updated_at",
 "project_id", "noteable_id", "system",
 "discussion_id", "note_html", 
"cached_markdown_version") VALUES 
(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 
?) RETURNING "id" /*application:sidekiq,
correlation_id:PwF5NIOp8C5,jid:58a2c5d39f16b1d7a9ecb18a,
job_class:ProcessCommitWorker,line:/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:235:in 
`mon_synchronize'*/
Full query 		13,626
6.92/sec
1.00/call
0.04%		 67,681.25 ms
34.360 ms/sec
4.967 ms/call
1.18%		 13,626
6.92/sec
1.00/call
0.01%		 1,397,066 blks
709.25 blks/sec
102.53 blks/call
0.07%		 96,105 blks
48.79 blks/sec
7.05 blks/call
5.08%		 181,471 blks
92.13 blks/sec
13.32 blks/call
7.94%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 52,262.60 ms
26.532 ms/sec
3.836 ms/call
10.62%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(1322179629)		 UPDATE "ci_builds" SET "status" 
= ?, "updated_at" = ?, "lock_version" 
= ? WHERE "ci_builds"."type" = 
? AND ("ci_builds"."lock_version" 
= ? OR "ci_builds"."lock_version" 
IS NULL) AND "ci_builds"."id" = 
? /*application:sidekiq,correlation_id:3qOzlUPdGk9,
jid:dc849010594180b391eb3979,job_class:PipelineProcessWorker,
line:/config/initializers/config_initializers_active_record_locking.rb:28:in 
`_update_row'*/
Full query 		20,821
10.57/sec
1.00/call
0.07%		 63,570.22 ms
32.273 ms/sec
3.053 ms/call
1.11%		 5,285
2.68/sec
0.25/call
0.00%		 575,864 blks
292.35 blks/sec
27.66 blks/call
0.03%		 1,786 blks
0.91 blks/sec
0.09 blks/call
0.09%		 7,351 blks
3.73 blks/sec
0.35 blks/call
0.32%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 558.55 ms
0.284 ms/sec
0.027 ms/call
0.11%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		1,154,978
586.35/sec
1.00/call
3.75%		 54,996.16 ms
27.920 ms/sec
0.048 ms/call
0.96%		 1,154,978
586.35/sec
1.00/call
0.48%		 5,767,824 blks
2.93K blks/sec
4.99 blks/call
0.30%		 10,919 blks
5.54 blks/sec
0.01 blks/call
0.58%		 198 blks
0.10 blks/sec
0.00 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3,644.80 ms
1.850 ms/sec
0.003 ms/call
0.74%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(2817433262)		 SELECT "notification_settings"."user_id" 
FROM "notification_settings" WHERE 
"notification_settings"."source_type" 
= ? AND "notification_settings"."source_id" 
= ? AND "notification_settings"."level" 
= ? /*application:sidekiq,correlation_id:Rwkax5J0h49,
jid:0fa510da01ca3ced147e38b8,job_class:NewMergeRequestWorker,
line:/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:235:in 
`mon_synchronize'*/
Full query 		13,737
6.97/sec
1.00/call
0.04%		 51,160.84 ms
25.973 ms/sec
3.724 ms/call
0.89%		 8,325,398
4.23K/sec
606.06/call
3.43%		 23,988,931 blks
12.18K blks/sec
1.75K blks/call
1.24%		 1,443 blks
0.73 blks/sec
0.11 blks/call
0.08%		 18 blks
0.01 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 878.59 ms
0.446 ms/sec
0.064 ms/call
0.18%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(3931118300)		 WITH "shas" AS (SELECT  "merge_request_diff_commits"."sha" 
FROM "merge_request_diff_commits" 
WHERE "merge_request_diff_commits"."merge_request_diff_id" 
IN (SELECT  "merge_request_diffs"."id" 
FROM "merge_request_diffs" WHERE 
"merge_request_diffs"."merge_request_id" 
= ? ORDER BY "merge_request_diffs"."id" 
DESC LIMIT ?) LIMIT ?) SELECT  
? AS one FROM ((SELECT "ci_pipelines".* 
FROM "ci_pipelines" INNER JOIN 
"shas" ON "shas"."sha" = "ci_pipelines"."source_sha" 
WHERE "ci_pipelines"."project_id" 
= ? AND ("ci_pipelines"."config_source" 
IN (?, ?, ?, ?) OR "ci_pipelines"."config_source" 
IS NULL) AND "ci_pipelines"."source" 
= ? AND "ci_pipelines"."merge_request_id" 
= ?)
UNION
(SELECT "ci_pipelines".* 
FROM "ci_pipelines" INNER JOIN 
"shas" ON encode("shas"."sha", 
?) = "ci_pipelines"."sha" WHERE 
"ci_pipelines"."project_id" = ? 
AND ("ci_pipelines"."config_source" 
IN (?, ?, ?, ?) OR "ci_pipelines"."config_source" 
IS NULL) AND "ci_pipelines"."source" 
= ? AND "ci_pipelines"."merge_request_id" 
= ?)
UNIO...
Full query 		156,401
79.40/sec
1.00/call
0.51%		 51,023.10 ms
25.903 ms/sec
0.326 ms/call
0.89%		 7,844
3.98/sec
0.05/call
0.00%		 4,097,597 blks
2.09K blks/sec
26.20 blks/call
0.21%		 33,897 blks
17.21 blks/sec
0.22 blks/call
1.79%		 2,022 blks
1.03 blks/sec
0.01 blks/call
0.09%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 18,844.83 ms
9.567 ms/sec
0.120 ms/call
3.83%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(402441325)		 SELECT "ci_pipelines".* FROM "ci_pipelines" 
WHERE "ci_pipelines"."project_id" 
= ? AND ("ci_pipelines"."config_source" 
IN (?, ?, ?, ?) OR "ci_pipelines"."config_source" 
IS NULL) AND "ci_pipelines"."ref" 
= ? AND "ci_pipelines"."id" != 
? AND "ci_pipelines"."sha" != ? 
AND ("ci_pipelines"."status" IN 
(?,?,?,?,?,?)) AND (NOT EXISTS 
(SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "ci_builds_metadata" 
ON "ci_builds_metadata"."build_id" 
= "ci_builds"."id" WHERE "ci_builds"."type" 
= ? AND (ci_builds.commit_id = 
ci_pipelines.id) AND ("ci_builds"."status" 
IN (?,?,?)) AND (ci_builds_metadata.id 
NOT IN (SELECT "ci_builds_metadata"."id" 
FROM "ci_builds_metadata" WHERE 
(ci_builds_metadata.build_id = 
ci_builds.id) AND "ci_builds_metadata"."interruptible" 
= ?)))) ORDER BY "ci_pipelines"."id" 
ASC LIMIT ? /*application:sidekiq,
correlation_id:EZ8HRE7Dkm2,jid:9643152829c14d3b030a1466,
job_class:PostReceive,line:/app/services/ci/create_pipeline_service.rb:96:in 
`block in cancel_pending_pipelines'*/
Full query 		7,488
3.80/sec
1.00/call
0.02%		 50,165.22 ms
25.467 ms/sec
6.699 ms/call
0.88%		 178
0.09/sec
0.02/call
0.00%		 1,338,182 blks
679.36 blks/sec
178.71 blks/call
0.07%		 252,399 blks
128.14 blks/sec
33.71 blks/call
13.35%		 4,176 blks
2.12 blks/sec
0.56 blks/call
0.18%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 39,770.23 ms
20.190 ms/sec
5.311 ms/call
8.08%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(503014889)		 SELECT  ? AS one FROM "ci_builds" 
INNER JOIN "ci_builds_metadata" 
ON "ci_builds_metadata"."build_id" 
= "ci_builds"."id" LEFT OUTER JOIN 
"ci_job_artifacts" ON "ci_job_artifacts"."job_id" 
= "ci_builds"."id" AND "ci_job_artifacts"."file_type" 
= ? WHERE "ci_builds"."type" IN 
(?) AND "ci_builds"."commit_id" 
= ? AND ("ci_builds"."retried" 
= ? OR "ci_builds"."retried" IS 
NULL) AND "ci_builds_metadata"."has_exposed_artifacts" 
= ? LIMIT ?
Full query 		222,327
112.87/sec
1.00/call
0.72%		 44,438.02 ms
22.560 ms/sec
0.200 ms/call
0.78%		 3,279
1.66/sec
0.01/call
0.00%		 16,328,514 blks
8.29K blks/sec
73.44 blks/call
0.84%		 7,455 blks
3.78 blks/sec
0.03 blks/call
0.39%		 1,192 blks
0.61 blks/sec
0.01 blks/call
0.05%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 4,951.74 ms
2.514 ms/sec
0.022 ms/call
1.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(2632052184)		 SELECT "clusters".* FROM "clusters" 
WHERE "clusters"."cluster_type" 
= ? AND "clusters"."enabled" = 
? AND "clusters"."environment_scope" 
= ? ORDER BY "clusters"."id" ASC 
LIMIT ? /*application:sidekiq,correlation_id:a304b32a54cc6f14c4dffef3ec39447c,
jid:072d6e73b4adb26f22d34e87,job_class:RunPipelineScheduleWorker,
line:/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:235:in 
`mon_synchronize'*/
Full query 		2,781
1.41/sec
1.00/call
0.01%		 42,640.67 ms
21.647 ms/sec
15.333 ms/call
0.74%		 0
0.00/sec
0.00/call
0.00%		 25,189,469 blks
12.79K blks/sec
9.06K blks/call
1.30%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(860089102)		 SELECT (CASE
          WHEN (SELECT 
count(*) FROM "ci_builds" WHERE 
"ci_builds"."type" IN (?, ?, ?) 
AND "ci_builds"."commit_id" = ? 
AND (stage_idx < ?) AND ("ci_builds"."retried" 
= ? OR "ci_builds"."retried" IS 
NULL) AND (allow_failure = ? OR 
status IN (?,?,?,?,?,?,?,?)))=(
SELECT count(*) FROM "ci_builds" 
WHERE "ci_builds"."type" IN (?,
 ?, ?) AND "ci_builds"."commit_id" 
= ? AND (stage_idx < ?) AND ("ci_builds"."retried" 
= ? OR "ci_builds"."retried" IS 
NULL) AND (allow_failure = ? OR 
status IN (?,?,?,?,?,?,?,?)) AND 
("ci_builds"."status" IN (?))) 
AND (SELECT count(*) > ? FROM "ci_builds" 
WHERE "ci_builds"."type" IN (?,
 ?, ?) AND "ci_builds"."commit_id" 
= ? AND (stage_idx < ?) AND ("ci_builds"."retried" 
= ? OR "ci_builds"."retried" IS 
NULL) AND "ci_builds"."allow_failure" 
= ? AND "ci_builds"."status" IN 
(?, ?)) THEN ?
          WHEN (
SELECT count(*) FROM "ci_builds" 
WHERE "ci_builds"."type" IN (?,
 ?, ?) AND "ci_builds"."commit_id" 
= ? AND (stage_idx < ?) AND ("ci_builds"."retried" 
= ? O...
Full query 		52,703
26.76/sec
1.00/call
0.17%		 42,425.36 ms
21.538 ms/sec
0.805 ms/call
0.74%		 2,088,064
1.07K/sec
39.62/call
0.86%		 26,845,924 blks
13.63K blks/sec
509.38 blks/call
1.39%		 2 blks
0.00 blks/sec
0.00 blks/call
0.00%		 292 blks
0.15 blks/sec
0.01 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.76 ms
0.001 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(2388959352)		 SELECT  ? AS one FROM "ci_builds" 
WHERE "ci_builds"."type" IN (?) 
AND "ci_builds"."commit_id" = ? 
AND ("ci_builds"."retried" = ? 
OR "ci_builds"."retried" IS NULL) 
AND (EXISTS (SELECT ? FROM "ci_job_artifacts" 
WHERE (ci_builds.id = ci_job_artifacts.job_id) 
AND "ci_job_artifacts"."file_type" 
= ?)) LIMIT ?
Full query 		232,213
117.89/sec
1.00/call
0.75%		 41,835.44 ms
21.239 ms/sec
0.180 ms/call
0.73%		 35,827
18.19/sec
0.15/call
0.01%		 10,183,581 blks
5.17K blks/sec
43.85 blks/call
0.53%		 17,750 blks
9.01 blks/sec
0.08 blks/call
0.94%		 275 blks
0.14 blks/sec
0.00 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 12,188.96 ms
6.188 ms/sec
0.052 ms/call
2.48%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(22434950)		 SELECT "projects".* FROM "projects" 
INNER JOIN project_mirror_data 
import_state ON import_state.project_id 
= projects.id WHERE "projects"."archived" 
= ? AND "projects"."mirror" = ? 
AND "import_state"."status" NOT 
IN (?, ?) AND (import_state.next_execution_timestamp 
<= ?) AND (import_state.retry_count 
<= ?) ORDER BY import_state.next_execution_timestamp 
LIMIT ?
Full query 		132
0.07/sec
1.00/call
0.00%		 41,550.03 ms
21.094 ms/sec
314.773 ms/call
0.73%		 50,683
25.73/sec
383.96/call
0.02%		 15,340,922 blks
7.79K blks/sec
116.22K blks/call
0.79%		 1 blks
0.00 blks/sec
0.01 blks/call
0.00%		 1,293 blks
0.66 blks/sec
9.80 blks/call
0.06%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.73 ms
0.000 ms/sec
0.006 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(445809111)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."type" = ? AND 
"ci_builds"."token_encrypted" = 
? LIMIT ? /*application:sidekiq,
correlation_id:DnS39pDvhQ9,jid:21a2aeccd6b4c66f7f1ba053,
job_class:PostReceive,line:/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:235:in 
`mon_synchronize'*/
Full query 		33,512
17.01/sec
1.00/call
0.11%		 40,728.83 ms
20.677 ms/sec
1.215 ms/call
0.71%		 0
0.00/sec
0.00/call
0.00%		 127,342 blks
64.65 blks/sec
3.80 blks/call
0.01%		 40,698 blks
20.66 blks/sec
1.21 blks/call
2.15%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 38,351.61 ms
19.470 ms/sec
1.144 ms/call
7.79%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica servers:

Replica (10.220.16.101)

Start: 2020-03-09T19:33:02.038241+00:00
End: 2020-03-09T20:03:20.749474+00:00
Period seconds: 1818.71123
Period age: 00:30:18.711233

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(3795242938)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= 'Ci::Build' AND ("ci_builds"."status" 
IN ('running')) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= 1) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = 'Ci::Build' 
AND ("ci_builds"."status" IN ('pending')) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= TRUE AND "projects"."pending_delete" 
= FALSE AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> 0) AND (projects.visibility_level=20 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (na...
Full query 		614
0.34/sec
1.00/call
0.01%		 510,059.03 ms
280.451 ms/sec
830.715 ms/call
29.67%		 2,905
1.60/sec
4.73/call
0.02%		 361,895,105 blks
198.99K blks/sec
589.41K blks/call
37.82%		 218 blks
0.12 blks/sec
0.36 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 7.86 ms
0.004 ms/sec
0.013 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(297297570)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		555
0.31/sec
1.00/call
0.01%		 461,174.79 ms
253.572 ms/sec
830.946 ms/call
26.82%		 2,757
1.52/sec
4.97/call
0.02%		 326,381,112 blks
179.46K blks/sec
588.08K blks/call
34.11%		 187 blks
0.10 blks/sec
0.34 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 12.05 ms
0.007 ms/sec
0.022 ms/call
0.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(874562767)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		197
0.11/sec
1.00/call
0.00%		 155,870.67 ms
85.704 ms/sec
791.222 ms/call
9.07%		 9,201
5.06/sec
46.71/call
0.07%		 112,745,509 blks
62.00K blks/sec
572.32K blks/call
11.78%		 12 blks
0.01 blks/sec
0.06 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.32 ms
0.000 ms/sec
0.002 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		4,014
2.21/sec
1.00/call
0.06%		 44,074.64 ms
24.234 ms/sec
10.980 ms/call
2.56%		 715
0.39/sec
0.18/call
0.01%		 17,569,029 blks
9.67K blks/sec
4.38K blks/call
1.84%		 54,516 blks
29.98 blks/sec
13.58 blks/call
4.13%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,543.92 ms
0.849 ms/sec
0.385 ms/call
2.26%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(3357552630)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		463,576
254.89/sec
1.00/call
6.68%		 31,642.88 ms
17.399 ms/sec
0.068 ms/call
1.84%		 454,371
249.83/sec
0.98/call
3.52%		 4,129,860 blks
2.28K blks/sec
8.91 blks/call
0.43%		 24,118 blks
13.26 blks/sec
0.05 blks/call
1.83%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 801.04 ms
0.440 ms/sec
0.002 ms/call
1.17%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		121
0.07/sec
1.00/call
0.00%		 29,570.81 ms
16.259 ms/sec
244.387 ms/call
1.72%		 121
0.07/sec
1.00/call
0.00%		 17,279,710 blks
9.51K blks/sec
142.81K blks/call
1.81%		 56 blks
0.03 blks/sec
0.46 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 5.69 ms
0.003 ms/sec
0.047 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(62153368)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL)
Full query 		77,202
42.45/sec
1.00/call
1.11%		 28,883.25 ms
15.881 ms/sec
0.374 ms/call
1.68%		 1,207,029
663.67/sec
15.63/call
9.34%		 2,062,535 blks
1.14K blks/sec
26.72 blks/call
0.22%		 67,024 blks
36.85 blks/sec
0.87 blks/call
5.08%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 8,546.04 ms
4.699 ms/sec
0.111 ms/call
12.53%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		907
0.50/sec
1.00/call
0.01%		 27,472.35 ms
15.105 ms/sec
30.289 ms/call
1.60%		 907
0.50/sec
1.00/call
0.01%		 8,175,970 blks
4.50K blks/sec
9.02K blks/call
0.85%		 70,295 blks
38.65 blks/sec
77.50 blks/call
5.32%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 610.95 ms
0.336 ms/sec
0.674 ms/call
0.90%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		121
0.07/sec
1.00/call
0.00%		 26,274.94 ms
14.447 ms/sec
217.148 ms/call
1.53%		 121
0.07/sec
1.00/call
0.00%		 16,674,239 blks
9.17K blks/sec
137.81K blks/call
1.74%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(2454174675)		 SELECT  "ci_triggers".* FROM "ci_triggers" 
WHERE "ci_triggers"."token" = ? 
LIMIT ?
Full query 		2,925
1.61/sec
1.00/call
0.04%		 24,475.92 ms
13.458 ms/sec
8.368 ms/call
1.42%		 26
0.01/sec
0.01/call
0.00%		 2,699,008 blks
1.49K blks/sec
922.74 blks/call
0.28%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		562,967
309.54/sec
1.00/call
8.11%		 16,910.78 ms
9.298 ms/sec
0.030 ms/call
0.98%		 562,951
309.53/sec
1.00/call
4.36%		 2,803,194 blks
1.55K blks/sec
4.98 blks/call
0.29%		 13,530 blks
7.44 blks/sec
0.02 blks/call
1.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 395.84 ms
0.218 ms/sec
0.001 ms/call
0.58%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		562,196
309.12/sec
1.00/call
8.10%		 15,099.72 ms
8.302 ms/sec
0.027 ms/call
0.88%		 556,148
305.79/sec
0.99/call
4.30%		 2,903,442 blks
1.60K blks/sec
5.16 blks/call
0.30%		 16,693 blks
9.18 blks/sec
0.03 blks/call
1.26%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 892.91 ms
0.491 ms/sec
0.002 ms/call
1.31%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(2747454572)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND (project_mirror_data.next_execution_timestamp 
- project_mirror_data.last_update_at) 
<= ?::interval
AND project_mirror_data.last_update_at 
< NOW() - ?::interval
;
Full query 		121
0.07/sec
1.00/call
0.00%		 14,630.70 ms
8.045 ms/sec
120.915 ms/call
0.85%		 121
0.07/sec
1.00/call
0.00%		 8,140,025 blks
4.48K blks/sec
67.28K blks/call
0.85%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(2555545475)		 SELECT  "projects".* FROM "projects" 
WHERE "projects"."id" = ? LIMIT 
?
Full query 		335,372
184.40/sec
1.00/call
4.83%		 14,229.33 ms
7.824 ms/sec
0.042 ms/call
0.83%		 335,370
184.40/sec
1.00/call
2.60%		 1,422,394 blks
782.09 blks/sec
4.24 blks/call
0.15%		 9 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2.06 ms
0.001 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(3842224781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND "ci_pipelines"."ref" = ? 
LIMIT ?) subquery_for_count
Full query 		19,703
10.83/sec
1.00/call
0.28%		 14,030.38 ms
7.714 ms/sec
0.712 ms/call
0.82%		 19,703
10.83/sec
1.00/call
0.15%		 3,308,301 blks
1.82K blks/sec
167.91 blks/call
0.35%		 19,411 blks
10.67 blks/sec
0.99 blks/call
1.47%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 407.70 ms
0.224 ms/sec
0.021 ms/call
0.60%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(3912557115)		 SELECT  "project_features".* FROM 
"project_features" WHERE "project_features"."project_id" 
= ? LIMIT ?
Full query 		412,360
226.73/sec
1.00/call
5.94%		 11,081.97 ms
6.093 ms/sec
0.027 ms/call
0.64%		 412,360
226.73/sec
1.00/call
3.19%		 1,638,422 blks
900.87 blks/sec
3.97 blks/call
0.17%		 11,751 blks
6.46 blks/sec
0.03 blks/call
0.89%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 412.11 ms
0.227 ms/sec
0.001 ms/call
0.60%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(2364908498)		 SELECT relnamespace::regnamespace 
as schemaname,
       relname as 
relname,
       pg_total_relation_size(
oid) bytes
  FROM pg_class
 WHERE 
relkind = ?;

Full query 		243
0.13/sec
1.00/call
0.00%		 9,650.44 ms
5.306 ms/sec
39.714 ms/call
0.56%		 93,312
51.31/sec
384.00/call
0.72%		 25,987 blks
14.29 blks/sec
106.94 blks/call
0.00%		 2 blks
0.00 blks/sec
0.01 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.04 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(1252209353)		 SELECT  "deployments".* FROM "deployments" 
WHERE "deployments"."environment_id" 
= ? AND "deployments"."status" 
= ? ORDER BY deployments.id DESC 
LIMIT ?
Full query 		6,678
3.67/sec
1.00/call
0.10%		 9,612.00 ms
5.285 ms/sec
1.439 ms/call
0.56%		 5,868
3.23/sec
0.88/call
0.05%		 2,910,814 blks
1.61K blks/sec
435.88 blks/call
0.30%		 18,685 blks
10.27 blks/sec
2.80 blks/call
1.42%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 333.48 ms
0.183 ms/sec
0.050 ms/call
0.49%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(180341781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND ("ci_pipelines"."status" 
IN (?,?,?)) ORDER BY "ci_pipelines"."id" 
DESC LIMIT ?) subquery_for_count
Full query 		951
0.52/sec
1.00/call
0.01%		 9,434.82 ms
5.188 ms/sec
9.921 ms/call
0.55%		 951
0.52/sec
1.00/call
0.01%		 487,483 blks
268.04 blks/sec
512.60 blks/call
0.05%		 274,181 blks
150.76 blks/sec
288.31 blks/call
20.77%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 5,792.64 ms
3.185 ms/sec
6.091 ms/call
8.49%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2632052184)		 SELECT  "clusters".* FROM "clusters" 
WHERE "clusters"."cluster_type" 
= ? AND "clusters"."enabled" = 
? AND "clusters"."environment_scope" 
= ? ORDER BY "clusters"."id" ASC 
LIMIT ?
Full query 		636
0.35/sec
1.00/call
0.01%		 9,004.80 ms
4.951 ms/sec
14.158 ms/call
0.52%		 0
0.00/sec
0.00/call
0.00%		 5,795,658 blks
3.19K blks/sec
9.12K blks/call
0.61%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.102)

Start: 2020-03-09T19:33:03.110816+00:00
End: 2020-03-09T20:03:47.794977+00:00
Period seconds: 1844.68416
Period age: 00:30:44.684161

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(297297570)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		557
0.30/sec
1.00/call
0.01%		 468,129.54 ms
253.772 ms/sec
840.448 ms/call
27.47%		 2,818
1.53/sec
5.06/call
0.02%		 328,551,292 blks
178.11K blks/sec
589.86K blks/call
35.14%		 178 blks
0.10 blks/sec
0.32 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 6.24 ms
0.003 ms/sec
0.011 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(3795242938)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		551
0.30/sec
1.00/call
0.01%		 452,844.18 ms
245.486 ms/sec
821.859 ms/call
26.57%		 2,850
1.54/sec
5.17/call
0.02%		 324,011,839 blks
175.65K blks/sec
588.05K blks/call
34.65%		 178 blks
0.10 blks/sec
0.32 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 10.14 ms
0.005 ms/sec
0.018 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(874562767)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= 'Ci::Build' AND ("ci_builds"."status" 
IN ('running')) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= 1) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = 'Ci::Build' 
AND ("ci_builds"."status" IN ('pending')) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= TRUE AND "projects"."pending_delete" 
= FALSE AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> 0) AND (projects.visibility_level=20 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (na...
Full query 		231
0.13/sec
1.00/call
0.00%		 189,340.21 ms
102.641 ms/sec
819.655 ms/call
11.11%		 9,300
5.04/sec
40.26/call
0.07%		 132,036,843 blks
71.58K blks/sec
571.59K blks/call
14.12%		 13 blks
0.01 blks/sec
0.06 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2.75 ms
0.001 ms/sec
0.012 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		4,043
2.19/sec
1.00/call
0.06%		 38,887.43 ms
21.081 ms/sec
9.618 ms/call
2.28%		 640
0.35/sec
0.16/call
0.00%		 15,129,682 blks
8.21K blks/sec
3.75K blks/call
1.62%		 59,280 blks
32.14 blks/sec
14.66 blks/call
3.95%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,451.69 ms
0.787 ms/sec
0.359 ms/call
1.62%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(62153368)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL)
Full query 		79,400
43.04/sec
1.00/call
1.11%		 36,371.23 ms
19.717 ms/sec
0.458 ms/call
2.13%		 1,271,389
689.22/sec
16.01/call
9.44%		 2,224,133 blks
1.21K blks/sec
28.01 blks/call
0.24%		 74,686 blks
40.49 blks/sec
0.94 blks/call
4.98%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 14,911.22 ms
8.083 ms/sec
0.188 ms/call
16.64%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(3357552630)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		470,106
254.84/sec
1.00/call
6.57%		 31,492.56 ms
17.072 ms/sec
0.067 ms/call
1.85%		 460,903
249.85/sec
0.98/call
3.42%		 4,187,590 blks
2.28K blks/sec
8.91 blks/call
0.45%		 25,747 blks
13.96 blks/sec
0.05 blks/call
1.72%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 849.36 ms
0.460 ms/sec
0.002 ms/call
0.95%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= true
AND project_mirror_data.retry_count 
<= 14
AND (projects.visibility_level 
= 20 OR plans.name IN ('early_adopter',
 'bronze', 'silver', 'gold'))
AND 
project_mirror_data.status NOT 
IN ('scheduled', 'started')
AND 
project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		123
0.07/sec
1.00/call
0.00%		 28,251.84 ms
15.315 ms/sec
229.690 ms/call
1.66%		 123
0.07/sec
1.00/call
0.00%		 17,531,513 blks
9.51K blks/sec
142.54K blks/call
1.87%		 42 blks
0.02 blks/sec
0.34 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 13.06 ms
0.007 ms/sec
0.106 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		123
0.07/sec
1.00/call
0.00%		 26,545.86 ms
14.390 ms/sec
215.820 ms/call
1.56%		 123
0.07/sec
1.00/call
0.00%		 16,917,293 blks
9.18K blks/sec
137.54K blks/call
1.81%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(2454174675)		 SELECT  "ci_triggers".* FROM "ci_triggers" 
WHERE "ci_triggers"."token" = ? 
LIMIT ?
Full query 		3,026
1.64/sec
1.00/call
0.04%		 25,955.30 ms
14.070 ms/sec
8.577 ms/call
1.52%		 31
0.02/sec
0.01/call
0.00%		 2,789,786 blks
1.52K blks/sec
921.94 blks/call
0.30%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		941
0.51/sec
1.00/call
0.01%		 23,848.01 ms
12.928 ms/sec
25.343 ms/call
1.40%		 941
0.51/sec
1.00/call
0.01%		 6,018,747 blks
3.27K blks/sec
6.40K blks/call
0.64%		 266,568 blks
144.51 blks/sec
283.28 blks/call
17.77%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2,221.43 ms
1.204 ms/sec
2.361 ms/call
2.48%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		569,075
308.49/sec
1.00/call
7.95%		 17,014.05 ms
9.223 ms/sec
0.030 ms/call
1.00%		 569,059
308.49/sec
1.00/call
4.22%		 2,832,673 blks
1.54K blks/sec
4.98 blks/call
0.30%		 14,947 blks
8.10 blks/sec
0.03 blks/call
1.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 371.13 ms
0.201 ms/sec
0.001 ms/call
0.41%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		573,238
310.75/sec
1.00/call
8.01%		 15,477.20 ms
8.390 ms/sec
0.027 ms/call
0.91%		 566,936
307.33/sec
0.99/call
4.21%		 2,959,373 blks
1.61K blks/sec
5.16 blks/call
0.32%		 17,206 blks
9.33 blks/sec
0.03 blks/call
1.15%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,182.20 ms
0.641 ms/sec
0.002 ms/call
1.32%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(2747454572)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= true
AND project_mirror_data.retry_count 
<= 14
AND (projects.visibility_level 
= 20 OR plans.name IN ('early_adopter',
 'bronze', 'silver', 'gold'))
AND 
project_mirror_data.status NOT 
IN ('scheduled', 'started')
AND 
(project_mirror_data.next_execution_timestamp 
- project_mirror_data.last_update_at) 
<= '30 minutes'::interval
AND project_mirror_data.last_update_at 
< NOW() - '30 minutes'::interval
;
Full query 		123
0.07/sec
1.00/call
0.00%		 14,586.25 ms
7.907 ms/sec
118.587 ms/call
0.86%		 123
0.07/sec
1.00/call
0.00%		 8,310,109 blks
4.51K blks/sec
67.57K blks/call
0.89%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(2555545475)		 SELECT  "projects".* FROM "projects" 
WHERE "projects"."id" = ? LIMIT 
?
Full query 		337,810
183.13/sec
1.00/call
4.72%		 14,436.28 ms
7.826 ms/sec
0.043 ms/call
0.85%		 337,807
183.12/sec
1.00/call
2.51%		 1,431,871 blks
776.21 blks/sec
4.24 blks/call
0.15%		 7 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.15 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(3842224781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND "ci_pipelines"."ref" = ? 
LIMIT ?) subquery_for_count
Full query 		20,123
10.91/sec
1.00/call
0.28%		 13,723.53 ms
7.440 ms/sec
0.682 ms/call
0.81%		 20,123
10.91/sec
1.00/call
0.15%		 3,242,856 blks
1.76K blks/sec
161.15 blks/call
0.35%		 17,536 blks
9.51 blks/sec
0.87 blks/call
1.17%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 451.05 ms
0.245 ms/sec
0.022 ms/call
0.50%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(2146878610)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_builds" WHERE "ci_builds"."type" 
IN (?) AND "ci_builds"."project_id" 
= ? LIMIT ?) subquery_for_count
Full query 		1,009
0.55/sec
1.00/call
0.01%		 11,446.48 ms
6.205 ms/sec
11.344 ms/call
0.67%		 1,009
0.55/sec
1.00/call
0.01%		 2,788,162 blks
1.52K blks/sec
2.77K blks/call
0.30%		 72,895 blks
39.52 blks/sec
72.24 blks/call
4.86%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2,684.12 ms
1.455 ms/sec
2.660 ms/call
3.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(3912557115)		 SELECT  "project_features".* FROM 
"project_features" WHERE "project_features"."project_id" 
= ? LIMIT ?
Full query 		418,525
226.88/sec
1.00/call
5.85%		 11,018.44 ms
5.973 ms/sec
0.026 ms/call
0.65%		 418,525
226.88/sec
1.00/call
3.11%		 1,663,442 blks
901.75 blks/sec
3.97 blks/call
0.18%		 11,507 blks
6.24 blks/sec
0.03 blks/call
0.77%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 362.54 ms
0.197 ms/sec
0.001 ms/call
0.40%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(180341781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND ("ci_pipelines"."status" 
IN (?,?,?)) ORDER BY "ci_pipelines"."id" 
DESC LIMIT ?) subquery_for_count
Full query 		944
0.51/sec
1.00/call
0.01%		 10,705.84 ms
5.804 ms/sec
11.341 ms/call
0.63%		 944
0.51/sec
1.00/call
0.01%		 519,384 blks
281.56 blks/sec
550.19 blks/call
0.06%		 269,953 blks
146.34 blks/sec
285.97 blks/call
18.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 6,818.85 ms
3.696 ms/sec
7.223 ms/call
7.61%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(2496446444)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_builds" WHERE "ci_builds"."type" 
IN (?) AND "ci_builds"."project_id" 
= ? AND ("ci_builds"."status" NOT 
IN (?)) ORDER BY ci_builds.id DESC 
LIMIT ?) subquery_for_count
Full query 		82
0.04/sec
1.00/call
0.00%		 10,235.63 ms
5.549 ms/sec
124.825 ms/call
0.60%		 82
0.04/sec
1.00/call
0.00%		 14,201 blks
7.70 blks/sec
173.18 blks/call
0.00%		 22,062 blks
11.96 blks/sec
269.05 blks/call
1.47%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 10,006.45 ms
5.424 ms/sec
122.030 ms/call
11.17%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(3345469221)		 SELECT DISTINCT "plans".* FROM "plans" 
INNER JOIN "gitlab_subscriptions" 
ON "gitlab_subscriptions"."hosted_plan_id" 
= "plans"."id" WHERE "plans"."name" 
IN (?, ?, ?, ?) AND "gitlab_subscriptions"."namespace_id" 
= ?
Full query 		288,657
156.48/sec
1.00/call
4.03%		 8,961.36 ms
4.858 ms/sec
0.031 ms/call
0.53%		 109,186
59.19/sec
0.38/call
0.81%		 1,728,500 blks
937.02 blks/sec
5.99 blks/call
0.18%		 4,148 blks
2.25 blks/sec
0.01 blks/call
0.28%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 177.31 ms
0.096 ms/sec
0.001 ms/call
0.20%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.103)

Start: 2020-03-09T19:33:04.1841+00:00
End: 2020-03-09T20:04:15.802677+00:00
Period seconds: 1871.61858
Period age: 00:31:11.618577

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(297297570)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		618
0.33/sec
1.00/call
0.01%		 518,626.35 ms
277.100 ms/sec
839.201 ms/call
27.89%		 2,964
1.58/sec
4.80/call
0.02%		 363,162,949 blks
194.04K blks/sec
587.65K blks/call
36.37%		 196 blks
0.10 blks/sec
0.32 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 17.78 ms
0.010 ms/sec
0.029 ms/call
0.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(3795242938)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		597
0.32/sec
1.00/call
0.01%		 512,178.63 ms
273.655 ms/sec
857.921 ms/call
27.55%		 2,934
1.57/sec
4.91/call
0.02%		 350,767,512 blks
187.42K blks/sec
587.56K blks/call
35.13%		 190 blks
0.10 blks/sec
0.32 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 15.98 ms
0.009 ms/sec
0.027 ms/call
0.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(874562767)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		230
0.12/sec
1.00/call
0.00%		 196,542.27 ms
105.012 ms/sec
854.532 ms/call
10.57%		 9,420
5.03/sec
40.96/call
0.07%		 129,771,246 blks
69.34K blks/sec
564.23K blks/call
13.00%		 13 blks
0.01 blks/sec
0.06 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 26.15 ms
0.014 ms/sec
0.114 ms/call
0.03%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		4,071
2.18/sec
1.00/call
0.06%		 39,709.40 ms
21.217 ms/sec
9.754 ms/call
2.14%		 691
0.37/sec
0.17/call
0.01%		 14,947,191 blks
7.99K blks/sec
3.68K blks/call
1.50%		 61,686 blks
32.96 blks/sec
15.15 blks/call
3.67%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3,585.99 ms
1.916 ms/sec
0.881 ms/call
3.51%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(62153368)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL)
Full query 		81,085
43.32/sec
1.00/call
1.11%		 34,848.00 ms
18.619 ms/sec
0.430 ms/call
1.87%		 1,267,998
677.49/sec
15.64/call
9.41%		 2,220,665 blks
1.19K blks/sec
27.39 blks/call
0.22%		 62,811 blks
33.56 blks/sec
0.77 blks/call
3.74%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 12,893.34 ms
6.889 ms/sec
0.159 ms/call
12.60%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(3357552630)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		477,405
255.08/sec
1.00/call
6.53%		 33,405.19 ms
17.848 ms/sec
0.070 ms/call
1.80%		 468,086
250.10/sec
0.98/call
3.47%		 4,254,260 blks
2.28K blks/sec
8.91 blks/call
0.43%		 25,090 blks
13.41 blks/sec
0.05 blks/call
1.49%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,219.89 ms
0.652 ms/sec
0.003 ms/call
1.19%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		967
0.52/sec
1.00/call
0.01%		 30,597.81 ms
16.348 ms/sec
31.642 ms/call
1.65%		 967
0.52/sec
1.00/call
0.01%		 5,480,051 blks
2.93K blks/sec
5.67K blks/call
0.55%		 445,284 blks
237.91 blks/sec
460.48 blks/call
26.48%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 7,233.02 ms
3.865 ms/sec
7.480 ms/call
7.07%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		125
0.07/sec
1.00/call
0.00%		 28,821.11 ms
15.399 ms/sec
230.569 ms/call
1.55%		 125
0.07/sec
1.00/call
0.00%		 17,820,934 blks
9.53K blks/sec
142.57K blks/call
1.78%		 35 blks
0.02 blks/sec
0.28 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 7.70 ms
0.004 ms/sec
0.062 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		125
0.07/sec
1.00/call
0.00%		 26,957.59 ms
14.403 ms/sec
215.661 ms/call
1.45%		 125
0.07/sec
1.00/call
0.00%		 17,191,043 blks
9.19K blks/sec
137.53K blks/call
1.72%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(2454174675)		 SELECT  "ci_triggers".* FROM "ci_triggers" 
WHERE "ci_triggers"."token" = ? 
LIMIT ?
Full query 		3,101
1.66/sec
1.00/call
0.04%		 26,458.41 ms
14.137 ms/sec
8.532 ms/call
1.42%		 24
0.01/sec
0.01/call
0.00%		 2,863,556 blks
1.53K blks/sec
923.43 blks/call
0.29%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		582,942
311.46/sec
1.00/call
7.97%		 18,218.75 ms
9.734 ms/sec
0.031 ms/call
0.98%		 582,942
311.46/sec
1.00/call
4.32%		 2,902,323 blks
1.56K blks/sec
4.98 blks/call
0.29%		 14,428 blks
7.71 blks/sec
0.02 blks/call
0.86%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 706.11 ms
0.377 ms/sec
0.001 ms/call
0.69%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		580,775
310.31/sec
1.00/call
7.94%		 16,823.24 ms
8.989 ms/sec
0.029 ms/call
0.90%		 574,593
307.00/sec
0.99/call
4.26%		 2,999,364 blks
1.61K blks/sec
5.16 blks/call
0.30%		 16,518 blks
8.83 blks/sec
0.03 blks/call
0.98%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,606.47 ms
0.858 ms/sec
0.003 ms/call
1.57%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(3842224781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND "ci_pipelines"."ref" = ? 
LIMIT ?) subquery_for_count
Full query 		20,504
10.96/sec
1.00/call
0.28%		 15,746.27 ms
8.413 ms/sec
0.768 ms/call
0.85%		 20,504
10.96/sec
1.00/call
0.15%		 3,445,240 blks
1.85K blks/sec
168.03 blks/call
0.35%		 29,694 blks
15.87 blks/sec
1.45 blks/call
1.77%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,143.63 ms
0.611 ms/sec
0.056 ms/call
1.12%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(180341781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND ("ci_pipelines"."status" 
IN (?,?,?)) ORDER BY "ci_pipelines"."id" 
DESC LIMIT ?) subquery_for_count
Full query 		976
0.52/sec
1.00/call
0.01%		 15,334.57 ms
8.193 ms/sec
15.712 ms/call
0.82%		 976
0.52/sec
1.00/call
0.01%		 663,992 blks
354.77 blks/sec
680.32 blks/call
0.07%		 246,813 blks
131.87 blks/sec
252.88 blks/call
14.68%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 10,805.59 ms
5.773 ms/sec
11.071 ms/call
10.56%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(2747454572)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND (project_mirror_data.next_execution_timestamp 
- project_mirror_data.last_update_at) 
<= ?::interval
AND project_mirror_data.last_update_at 
< NOW() - ?::interval
;
Full query 		125
0.07/sec
1.00/call
0.00%		 15,164.00 ms
8.102 ms/sec
121.312 ms/call
0.82%		 125
0.07/sec
1.00/call
0.00%		 8,458,879 blks
4.52K blks/sec
67.68K blks/call
0.85%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(2555545475)		 SELECT  "projects".* FROM "projects" 
WHERE "projects"."id" = ? LIMIT 
?
Full query 		346,736
185.26/sec
1.00/call
4.74%		 15,041.05 ms
8.036 ms/sec
0.043 ms/call
0.81%		 346,733
185.26/sec
1.00/call
2.57%		 1,466,306 blks
783.44 blks/sec
4.23 blks/call
0.15%		 23 blks
0.01 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3.05 ms
0.002 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(2146878610)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_builds" WHERE "ci_builds"."type" 
IN (?) AND "ci_builds"."project_id" 
= ? LIMIT ?) subquery_for_count
Full query 		1,022
0.55/sec
1.00/call
0.01%		 12,401.94 ms
6.626 ms/sec
12.135 ms/call
0.67%		 1,022
0.55/sec
1.00/call
0.01%		 2,690,217 blks
1.44K blks/sec
2.64K blks/call
0.27%		 70,492 blks
37.66 blks/sec
68.97 blks/call
4.19%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3,760.15 ms
2.009 ms/sec
3.679 ms/call
3.68%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(3912557115)		 SELECT  "project_features".* FROM 
"project_features" WHERE "project_features"."project_id" 
= ? LIMIT ?
Full query 		425,573
227.38/sec
1.00/call
5.82%		 11,934.25 ms
6.376 ms/sec
0.028 ms/call
0.64%		 425,573
227.38/sec
1.00/call
3.16%		 1,692,071 blks
904.07 blks/sec
3.98 blks/call
0.17%		 11,057 blks
5.91 blks/sec
0.03 blks/call
0.66%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 590.74 ms
0.316 ms/sec
0.001 ms/call
0.58%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(3345469221)		 SELECT DISTINCT "plans".* FROM "plans" 
INNER JOIN "gitlab_subscriptions" 
ON "gitlab_subscriptions"."hosted_plan_id" 
= "plans"."id" WHERE "plans"."name" 
IN (?, ?, ?, ?) AND "gitlab_subscriptions"."namespace_id" 
= ?
Full query 		294,059
157.11/sec
1.00/call
4.02%		 9,499.09 ms
5.075 ms/sec
0.032 ms/call
0.51%		 111,327
59.48/sec
0.38/call
0.83%		 1,761,207 blks
941.01 blks/sec
5.99 blks/call
0.18%		 4,007 blks
2.14 blks/sec
0.01 blks/call
0.24%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 232.27 ms
0.124 ms/sec
0.001 ms/call
0.23%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2632052184)		 SELECT  "clusters".* FROM "clusters" 
WHERE "clusters"."cluster_type" 
= ? AND "clusters"."enabled" = 
? AND "clusters"."environment_scope" 
= ? ORDER BY "clusters"."id" ASC 
LIMIT ?
Full query 		632
0.34/sec
1.00/call
0.01%		 8,619.29 ms
4.605 ms/sec
13.638 ms/call
0.46%		 0
0.00/sec
0.00/call
0.00%		 5,759,347 blks
3.08K blks/sec
9.12K blks/call
0.58%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.104)

Start: 2020-03-09T19:33:05.250185+00:00
End: 2020-03-09T20:04:43.366888+00:00
Period seconds: 1898.1167
Period age: 00:31:38.116703

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(297297570)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		605
0.32/sec
1.00/call
0.01%		 510,304.01 ms
268.848 ms/sec
843.478 ms/call
28.48%		 2,816
1.48/sec
4.65/call
0.02%		 357,179,318 blks
188.18K blks/sec
590.38K blks/call
36.64%		 193 blks
0.10 blks/sec
0.32 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 10.45 ms
0.006 ms/sec
0.017 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(3795242938)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= 'Ci::Build' AND ("ci_builds"."status" 
IN ('running')) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= 1) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = 'Ci::Build' 
AND ("ci_builds"."status" IN ('pending')) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= TRUE AND "projects"."pending_delete" 
= FALSE AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> 0) AND (projects.visibility_level=20 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (na...
Full query 		559
0.29/sec
1.00/call
0.01%		 474,611.42 ms
250.043 ms/sec
849.037 ms/call
26.49%		 3,042
1.60/sec
5.44/call
0.02%		 328,643,730 blks
173.15K blks/sec
587.92K blks/call
33.71%		 205 blks
0.11 blks/sec
0.37 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 11.80 ms
0.006 ms/sec
0.021 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(874562767)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		231
0.12/sec
1.00/call
0.00%		 188,502.63 ms
99.310 ms/sec
816.029 ms/call
10.52%		 9,469
4.99/sec
40.99/call
0.07%		 131,731,714 blks
69.41K blks/sec
570.27K blks/call
13.51%		 15 blks
0.01 blks/sec
0.06 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 7.89 ms
0.004 ms/sec
0.034 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		4,154
2.19/sec
1.00/call
0.06%		 35,509.88 ms
18.708 ms/sec
8.548 ms/call
1.98%		 722
0.38/sec
0.17/call
0.01%		 13,754,366 blks
7.25K blks/sec
3.32K blks/call
1.41%		 52,599 blks
27.71 blks/sec
12.66 blks/call
3.49%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,992.98 ms
1.050 ms/sec
0.480 ms/call
2.45%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(62153368)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL)
Full query 		81,448
42.91/sec
1.00/call
1.10%		 35,020.91 ms
18.450 ms/sec
0.430 ms/call
1.95%		 1,190,349
627.12/sec
14.61/call
8.71%		 2,081,480 blks
1.10K blks/sec
25.56 blks/call
0.21%		 67,115 blks
35.36 blks/sec
0.82 blks/call
4.46%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 14,005.56 ms
7.379 ms/sec
0.172 ms/call
17.20%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(3357552630)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		483,765
254.87/sec
1.00/call
6.55%		 33,593.84 ms
17.699 ms/sec
0.069 ms/call
1.88%		 474,097
249.77/sec
0.98/call
3.47%		 4,312,071 blks
2.28K blks/sec
8.91 blks/call
0.44%		 23,106 blks
12.17 blks/sec
0.05 blks/call
1.53%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 737.26 ms
0.388 ms/sec
0.002 ms/call
0.91%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		127
0.07/sec
1.00/call
0.00%		 30,326.49 ms
15.977 ms/sec
238.791 ms/call
1.69%		 127
0.07/sec
1.00/call
0.00%		 18,153,316 blks
9.57K blks/sec
142.94K blks/call
1.86%		 44 blks
0.02 blks/sec
0.35 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 13.29 ms
0.007 ms/sec
0.105 ms/call
0.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(2454174675)		 SELECT  "ci_triggers".* FROM "ci_triggers" 
WHERE "ci_triggers"."token" = ? 
LIMIT ?
Full query 		3,194
1.68/sec
1.00/call
0.04%		 27,904.12 ms
14.701 ms/sec
8.736 ms/call
1.56%		 27
0.01/sec
0.01/call
0.00%		 2,948,076 blks
1.56K blks/sec
923.00 blks/call
0.30%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		127
0.07/sec
1.00/call
0.00%		 27,744.91 ms
14.617 ms/sec
218.464 ms/call
1.55%		 127
0.07/sec
1.00/call
0.00%		 17,519,437 blks
9.23K blks/sec
137.95K blks/call
1.80%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		977
0.51/sec
1.00/call
0.01%		 25,625.53 ms
13.501 ms/sec
26.229 ms/call
1.43%		 977
0.51/sec
1.00/call
0.01%		 6,691,250 blks
3.53K blks/sec
6.85K blks/call
0.69%		 245,989 blks
129.60 blks/sec
251.78 blks/call
16.34%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2,275.39 ms
1.199 ms/sec
2.329 ms/call
2.79%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		587,736
309.64/sec
1.00/call
7.95%		 18,087.80 ms
9.529 ms/sec
0.031 ms/call
1.01%		 587,735
309.64/sec
1.00/call
4.30%		 2,928,320 blks
1.55K blks/sec
4.98 blks/call
0.30%		 12,551 blks
6.61 blks/sec
0.02 blks/call
0.83%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 271.50 ms
0.143 ms/sec
0.000 ms/call
0.33%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		589,256
310.44/sec
1.00/call
7.97%		 16,627.31 ms
8.760 ms/sec
0.028 ms/call
0.93%		 582,790
307.04/sec
0.99/call
4.26%		 3,042,915 blks
1.61K blks/sec
5.16 blks/call
0.31%		 17,026 blks
8.97 blks/sec
0.03 blks/call
1.13%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,147.87 ms
0.605 ms/sec
0.002 ms/call
1.41%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(2747454572)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND (project_mirror_data.next_execution_timestamp 
- project_mirror_data.last_update_at) 
<= ?::interval
AND project_mirror_data.last_update_at 
< NOW() - ?::interval
;
Full query 		127
0.07/sec
1.00/call
0.00%		 15,411.74 ms
8.119 ms/sec
121.352 ms/call
0.86%		 127
0.07/sec
1.00/call
0.00%		 8,601,218 blks
4.54K blks/sec
67.73K blks/call
0.88%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(2555545475)		 SELECT  "projects".* FROM "projects" 
WHERE "projects"."id" = ? LIMIT 
?
Full query 		347,743
183.20/sec
1.00/call
4.71%		 15,368.84 ms
8.097 ms/sec
0.044 ms/call
0.86%		 347,735
183.20/sec
1.00/call
2.54%		 1,474,615 blks
776.88 blks/sec
4.24 blks/call
0.15%		 12 blks
0.01 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.21 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(3842224781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND "ci_pipelines"."ref" = ? 
LIMIT ?) subquery_for_count
Full query 		20,694
10.90/sec
1.00/call
0.28%		 15,221.70 ms
8.019 ms/sec
0.736 ms/call
0.85%		 20,694
10.90/sec
1.00/call
0.15%		 3,506,088 blks
1.85K blks/sec
169.43 blks/call
0.36%		 39,337 blks
20.72 blks/sec
1.90 blks/call
2.61%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 646.02 ms
0.340 ms/sec
0.031 ms/call
0.79%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(3912557115)		 SELECT  "project_features".* FROM 
"project_features" WHERE "project_features"."project_id" 
= ? LIMIT ?
Full query 		431,265
227.21/sec
1.00/call
5.84%		 11,844.66 ms
6.240 ms/sec
0.027 ms/call
0.66%		 431,265
227.21/sec
1.00/call
3.16%		 1,714,825 blks
903.43 blks/sec
3.98 blks/call
0.18%		 11,095 blks
5.85 blks/sec
0.03 blks/call
0.74%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 342.31 ms
0.180 ms/sec
0.001 ms/call
0.42%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(2146878610)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_builds" WHERE "ci_builds"."type" 
IN (?) AND "ci_builds"."project_id" 
= ? LIMIT ?) subquery_for_count
Full query 		1,033
0.54/sec
1.00/call
0.01%		 10,774.04 ms
5.676 ms/sec
10.430 ms/call
0.60%		 1,033
0.54/sec
1.00/call
0.01%		 2,714,404 blks
1.44K blks/sec
2.63K blks/call
0.28%		 67,739 blks
35.69 blks/sec
65.58 blks/call
4.50%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,933.79 ms
1.019 ms/sec
1.872 ms/call
2.38%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(1252209353)		 SELECT  "deployments".* FROM "deployments" 
WHERE "deployments"."environment_id" 
= ? AND "deployments"."status" 
= ? ORDER BY deployments.id DESC 
LIMIT ?
Full query 		7,331
3.86/sec
1.00/call
0.10%		 10,350.59 ms
5.453 ms/sec
1.412 ms/call
0.58%		 6,419
3.38/sec
0.88/call
0.05%		 3,203,205 blks
1.69K blks/sec
436.94 blks/call
0.33%		 35,659 blks
18.79 blks/sec
4.86 blks/call
2.37%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 510.01 ms
0.269 ms/sec
0.070 ms/call
0.63%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(180341781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND ("ci_pipelines"."status" 
IN (?,?,?)) ORDER BY "ci_pipelines"."id" 
DESC LIMIT ?) subquery_for_count
Full query 		991
0.52/sec
1.00/call
0.01%		 10,050.07 ms
5.295 ms/sec
10.141 ms/call
0.56%		 991
0.52/sec
1.00/call
0.01%		 491,412 blks
258.89 blks/sec
495.87 blks/call
0.05%		 261,912 blks
137.99 blks/sec
264.29 blks/call
17.40%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 6,279.37 ms
3.308 ms/sec
6.336 ms/call
7.71%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(3345469221)		 SELECT DISTINCT "plans".* FROM "plans" 
INNER JOIN "gitlab_subscriptions" 
ON "gitlab_subscriptions"."hosted_plan_id" 
= "plans"."id" WHERE "plans"."name" 
IN (?, ?, ?, ?) AND "gitlab_subscriptions"."namespace_id" 
= ?
Full query 		298,408
157.21/sec
1.00/call
4.04%		 9,736.50 ms
5.130 ms/sec
0.033 ms/call
0.54%		 113,049
59.56/sec
0.38/call
0.83%		 1,786,982 blks
941.45 blks/sec
5.99 blks/call
0.18%		 4,103 blks
2.16 blks/sec
0.01 blks/call
0.27%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 174.15 ms
0.092 ms/sec
0.001 ms/call
0.21%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.105)

Start: 2020-03-09T19:33:06.332445+00:00
End: 2020-03-09T20:05:12.23579+00:00
Period seconds: 1925.90335
Period age: 00:32:05.903345

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(3795242938)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		627
0.33/sec
1.00/call
0.01%		 497,590.54 ms
258.367 ms/sec
793.605 ms/call
27.93%		 3,654
1.90/sec
5.83/call
0.03%		 369,166,422 blks
191.69K blks/sec
588.79K blks/call
35.89%		 213 blks
0.11 blks/sec
0.34 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 15.77 ms
0.008 ms/sec
0.025 ms/call
0.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(297297570)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		625
0.32/sec
1.00/call
0.01%		 494,519.39 ms
256.773 ms/sec
791.231 ms/call
27.76%		 3,510
1.82/sec
5.62/call
0.03%		 367,925,465 blks
191.05K blks/sec
588.69K blks/call
35.77%		 231 blks
0.12 blks/sec
0.37 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 19.41 ms
0.010 ms/sec
0.031 ms/call
0.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(874562767)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		238
0.12/sec
1.00/call
0.00%		 187,808.00 ms
97.517 ms/sec
789.109 ms/call
10.54%		 10,722
5.57/sec
45.05/call
0.08%		 135,441,409 blks
70.33K blks/sec
569.09K blks/call
13.17%		 19 blks
0.01 blks/sec
0.08 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 17.34 ms
0.009 ms/sec
0.073 ms/call
0.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		4,298
2.23/sec
1.00/call
0.06%		 36,742.44 ms
19.078 ms/sec
8.549 ms/call
2.06%		 704
0.37/sec
0.16/call
0.01%		 14,330,330 blks
7.45K blks/sec
3.34K blks/call
1.39%		 52,848 blks
27.44 blks/sec
12.30 blks/call
3.15%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2,102.68 ms
1.092 ms/sec
0.489 ms/call
2.29%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(62153368)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL)
Full query 		82,603
42.89/sec
1.00/call
1.11%		 34,047.57 ms
17.679 ms/sec
0.412 ms/call
1.91%		 1,248,418
648.22/sec
15.11/call
9.06%		 2,167,826 blks
1.13K blks/sec
26.24 blks/call
0.21%		 62,495 blks
32.45 blks/sec
0.76 blks/call
3.72%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 12,976.41 ms
6.738 ms/sec
0.157 ms/call
14.10%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(3357552630)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		491,383
255.14/sec
1.00/call
6.61%		 32,341.01 ms
16.793 ms/sec
0.066 ms/call
1.82%		 481,961
250.25/sec
0.98/call
3.50%		 4,384,875 blks
2.28K blks/sec
8.92 blks/call
0.43%		 20,276 blks
10.53 blks/sec
0.04 blks/call
1.21%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 805.31 ms
0.418 ms/sec
0.002 ms/call
0.88%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		1,033
0.54/sec
1.00/call
0.01%		 30,792.20 ms
15.988 ms/sec
29.809 ms/call
1.73%		 1,033
0.54/sec
1.00/call
0.01%		 7,044,914 blks
3.66K blks/sec
6.82K blks/call
0.68%		 482,432 blks
250.50 blks/sec
467.02 blks/call
28.71%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 4,909.23 ms
2.549 ms/sec
4.752 ms/call
5.34%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		128
0.07/sec
1.00/call
0.00%		 28,674.21 ms
14.889 ms/sec
224.017 ms/call
1.61%		 128
0.07/sec
1.00/call
0.00%		 18,295,436 blks
9.50K blks/sec
142.94K blks/call
1.78%		 42 blks
0.02 blks/sec
0.33 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 9.59 ms
0.005 ms/sec
0.075 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		128
0.07/sec
1.00/call
0.00%		 26,248.11 ms
13.629 ms/sec
205.063 ms/call
1.47%		 128
0.07/sec
1.00/call
0.00%		 17,655,629 blks
9.17K blks/sec
137.94K blks/call
1.72%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(2454174675)		 SELECT  "ci_triggers".* FROM "ci_triggers" 
WHERE "ci_triggers"."token" = ? 
LIMIT ?
Full query 		3,081
1.60/sec
1.00/call
0.04%		 26,179.06 ms
13.593 ms/sec
8.497 ms/call
1.47%		 28
0.01/sec
0.01/call
0.00%		 2,841,534 blks
1.48K blks/sec
922.28 blks/call
0.28%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = 3503181 
AND "routes"."source_type" = 'Project' 
LIMIT 1
Full query 		594,379
308.62/sec
1.00/call
8.00%		 17,303.01 ms
8.984 ms/sec
0.029 ms/call
0.97%		 594,364
308.62/sec
1.00/call
4.32%		 2,963,403 blks
1.54K blks/sec
4.99 blks/call
0.29%		 10,725 blks
5.57 blks/sec
0.02 blks/call
0.64%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 269.37 ms
0.140 ms/sec
0.000 ms/call
0.29%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		595,227
309.06/sec
1.00/call
8.01%		 16,048.72 ms
8.333 ms/sec
0.027 ms/call
0.90%		 588,874
305.77/sec
0.99/call
4.28%		 3,074,117 blks
1.60K blks/sec
5.16 blks/call
0.30%		 17,640 blks
9.16 blks/sec
0.03 blks/call
1.05%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,340.27 ms
0.696 ms/sec
0.002 ms/call
1.46%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(2555545475)		 SELECT  "projects".* FROM "projects" 
WHERE "projects"."id" = ? LIMIT 
?
Full query 		352,387
182.97/sec
1.00/call
4.74%		 14,677.05 ms
7.621 ms/sec
0.042 ms/call
0.82%		 352,381
182.97/sec
1.00/call
2.56%		 1,494,254 blks
775.87 blks/sec
4.24 blks/call
0.15%		 1 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.03 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(2747454572)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND (project_mirror_data.next_execution_timestamp 
- project_mirror_data.last_update_at) 
<= ?::interval
AND project_mirror_data.last_update_at 
< NOW() - ?::interval
;
Full query 		128
0.07/sec
1.00/call
0.00%		 14,643.89 ms
7.604 ms/sec
114.405 ms/call
0.82%		 128
0.07/sec
1.00/call
0.00%		 8,662,648 blks
4.50K blks/sec
67.68K blks/call
0.84%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(3842224781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND "ci_pipelines"."ref" = ? 
LIMIT ?) subquery_for_count
Full query 		20,920
10.86/sec
1.00/call
0.28%		 14,172.07 ms
7.359 ms/sec
0.677 ms/call
0.80%		 20,920
10.86/sec
1.00/call
0.15%		 3,254,028 blks
1.69K blks/sec
155.55 blks/call
0.32%		 33,647 blks
17.47 blks/sec
1.61 blks/call
2.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 716.04 ms
0.372 ms/sec
0.034 ms/call
0.78%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(2146878610)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_builds" WHERE "ci_builds"."type" 
IN (?) AND "ci_builds"."project_id" 
= ? LIMIT ?) subquery_for_count
Full query 		1,085
0.56/sec
1.00/call
0.01%		 13,117.98 ms
6.811 ms/sec
12.090 ms/call
0.74%		 1,085
0.56/sec
1.00/call
0.01%		 2,953,677 blks
1.54K blks/sec
2.73K blks/call
0.29%		 52,835 blks
27.43 blks/sec
48.70 blks/call
3.14%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3,872.74 ms
2.011 ms/sec
3.569 ms/call
4.21%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(2496446444)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_builds" WHERE "ci_builds"."type" 
IN (?) AND "ci_builds"."project_id" 
= ? AND ("ci_builds"."status" NOT 
IN (?)) ORDER BY ci_builds.id DESC 
LIMIT ?) subquery_for_count
Full query 		64
0.03/sec
1.00/call
0.00%		 12,254.88 ms
6.363 ms/sec
191.482 ms/call
0.69%		 64
0.03/sec
1.00/call
0.00%		 17,802 blks
9.24 blks/sec
278.16 blks/call
0.00%		 22,349 blks
11.60 blks/sec
349.20 blks/call
1.33%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 11,985.13 ms
6.223 ms/sec
187.268 ms/call
13.03%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(3912557115)		 SELECT  "project_features".* FROM 
"project_features" WHERE "project_features"."project_id" 
= ? LIMIT ?
Full query 		437,318
227.07/sec
1.00/call
5.89%		 11,577.59 ms
6.012 ms/sec
0.026 ms/call
0.65%		 437,318
227.07/sec
1.00/call
3.18%		 1,738,219 blks
902.55 blks/sec
3.97 blks/call
0.17%		 11,903 blks
6.18 blks/sec
0.03 blks/call
0.71%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 429.79 ms
0.223 ms/sec
0.001 ms/call
0.47%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(180341781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND ("ci_pipelines"."status" 
IN (?,?,?)) ORDER BY "ci_pipelines"."id" 
DESC LIMIT ?) subquery_for_count
Full query 		1,034
0.54/sec
1.00/call
0.01%		 10,661.74 ms
5.536 ms/sec
10.311 ms/call
0.60%		 1,034
0.54/sec
1.00/call
0.01%		 570,978 blks
296.47 blks/sec
552.20 blks/call
0.06%		 255,035 blks
132.42 blks/sec
246.65 blks/call
15.18%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 6,519.61 ms
3.385 ms/sec
6.305 ms/call
7.09%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2632052184)		 SELECT  "clusters".* FROM "clusters" 
WHERE "clusters"."cluster_type" 
= ? AND "clusters"."enabled" = 
? AND "clusters"."environment_scope" 
= ? ORDER BY "clusters"."id" ASC 
LIMIT ?
Full query 		671
0.35/sec
1.00/call
0.01%		 9,301.53 ms
4.830 ms/sec
13.862 ms/call
0.52%		 0
0.00/sec
0.00/call
0.00%		 6,114,741 blks
3.18K blks/sec
9.12K blks/call
0.59%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.107)

Start: 2020-03-09T19:33:08.460544+00:00
End: 2020-03-09T20:06:36.371713+00:00
Period seconds: 2007.91117
Period age: 00:33:27.911169

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(3795242938)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= 'Ci::Build' AND ("ci_builds"."status" 
IN ('running')) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= 1) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = 'Ci::Build' 
AND ("ci_builds"."status" IN ('pending')) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= TRUE AND "projects"."pending_delete" 
= FALSE AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> 0) AND (projects.visibility_level=20 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (na...
Full query 		650
0.32/sec
1.00/call
0.01%		 378,461.18 ms
188.485 ms/sec
582.248 ms/call
17.91%		 8,183
4.08/sec
12.59/call
0.07%		 383,490,102 blks
190.99K blks/sec
589.99K blks/call
31.24%		 273 blks
0.14 blks/sec
0.42 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 35.02 ms
0.017 ms/sec
0.054 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(297297570)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		597
0.30/sec
1.00/call
0.01%		 342,536.62 ms
170.594 ms/sec
573.763 ms/call
16.21%		 7,625
3.80/sec
12.77/call
0.06%		 352,512,773 blks
175.57K blks/sec
590.48K blks/call
28.71%		 314 blks
0.16 blks/sec
0.53 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 24.78 ms
0.012 ms/sec
0.042 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(3135725993)		 SELECT "projects".* FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?
Full query 		508
0.25/sec
1.00/call
0.01%		 169,533.91 ms
84.433 ms/sec
333.728 ms/call
8.02%		 50,468
25.13/sec
99.35/call
0.41%		 82,173,713 blks
40.93K blks/sec
161.76K blks/call
6.69%		 4,289 blks
2.14 blks/sec
8.44 blks/call
0.19%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 36.43 ms
0.018 ms/sec
0.072 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(874562767)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= 'Ci::Build' AND ("ci_builds"."status" 
IN ('running')) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= 1) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = 'Ci::Build' 
AND ("ci_builds"."status" IN ('pending')) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= TRUE AND "projects"."pending_delete" 
= FALSE AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> 0) AND (projects.visibility_level=20 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (na...
Full query 		264
0.13/sec
1.00/call
0.00%		 144,726.03 ms
72.078 ms/sec
548.205 ms/call
6.85%		 11,293
5.62/sec
42.78/call
0.09%		 149,444,266 blks
74.43K blks/sec
566.08K blks/call
12.17%		 19 blks
0.01 blks/sec
0.07 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 6.44 ms
0.003 ms/sec
0.024 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(727780750)		 SELECT "events"."id" AS t0_r0, "events"."project_id" 
AS t0_r1, "events"."author_id" 
AS t0_r2, "events"."target_id" 
AS t0_r3, "events"."created_at" 
AS t0_r4, "events"."updated_at" 
AS t0_r5, "events"."action" AS 
t0_r6, "events"."target_type" AS 
t0_r7, "events"."group_id" AS t0_r8,
 "users"."id" AS t1_r0, "users"."email" 
AS t1_r1, "users"."encrypted_password" 
AS t1_r2, "users"."reset_password_token" 
AS t1_r3, "users"."reset_password_sent_at" 
AS t1_r4, "users"."remember_created_at" 
AS t1_r5, "users"."sign_in_count" 
AS t1_r6, "users"."current_sign_in_at" 
AS t1_r7, "users"."last_sign_in_at" 
AS t1_r8, "users"."current_sign_in_ip" 
AS t1_r9, "users"."last_sign_in_ip" 
AS t1_r10, "users"."created_at" 
AS t1_r11, "users"."updated_at" 
AS t1_r12, "users"."name" AS t1_r13,
 "users"."admin" AS t1_r14, "users"."projects_limit" 
AS t1_r15, "users"."skype" AS t1_r16,
 "users"."linkedin" AS t1_r17, 
"users"."twitter" AS t1_r18, "users"."bio" 
AS t1_r19, "users"."failed_attempts" 
AS t1_r20, "users"."locked_at" 
A...
Full query 		442
0.22/sec
1.00/call
0.01%		 72,467.74 ms
36.091 ms/sec
163.954 ms/call
3.43%		 5,851
2.91/sec
13.24/call
0.05%		 11,105,532 blks
5.54K blks/sec
25.13K blks/call
0.90%		 232,259 blks
115.67 blks/sec
525.47 blks/call
10.39%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 59,779.93 ms
29.772 ms/sec
135.249 ms/call
14.85%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(2995677756)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."type" = 'Ci::Build' 
AND ("ci_builds"."status" IN ('pending')) 
AND "ci_builds"."runner_id" IS 
NULL AND "ci_builds"."project_id" 
IN (SELECT "projects"."id" FROM 
"projects" INNER JOIN "project_ci_cd_settings" 
ON "project_ci_cd_settings"."project_id" 
= "projects"."id" LEFT JOIN project_features 
ON projects.id = project_features.project_id 
WHERE "projects"."namespace_id" 
IN (WITH RECURSIVE "base_and_descendants" 
AS ((SELECT "namespaces".* FROM 
"namespaces" INNER JOIN "ci_runner_namespaces" 
ON "ci_runner_namespaces"."namespace_id" 
= "namespaces"."id" WHERE "namespaces"."type" 
= 'Group' AND "ci_runner_namespaces"."runner_id" 
= 1659331)
UNION
(SELECT "namespaces".* 
FROM "namespaces", "base_and_descendants" 
WHERE "namespaces"."type" = 'Group' 
AND "namespaces"."parent_id" = 
"base_and_descendants"."id")) SELECT 
"namespaces"."id" FROM "base_and_descendants" 
AS "namespaces") AND "project_ci_cd_settings"."group_runners_enabled" 
= TRUE AND ("p...
Full query 		422
0.21/sec
1.00/call
0.01%		 48,479.66 ms
24.144 ms/sec
114.881 ms/call
2.29%		 155
0.08/sec
0.37/call
0.00%		 39,669,374 blks
19.76K blks/sec
94.01K blks/call
3.23%		 8 blks
0.00 blks/sec
0.02 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.56 ms
0.001 ms/sec
0.004 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(3357552630)		 SELECT "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "p...
Full query 		509,942
253.97/sec
1.00/call
6.96%		 42,212.99 ms
21.023 ms/sec
0.083 ms/call
2.00%		 499,972
249.00/sec
0.98/call
4.04%		 4,543,849 blks
2.27K blks/sec
8.91 blks/call
0.37%		 26,965 blks
13.43 blks/sec
0.05 blks/call
1.21%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 11,719.33 ms
5.837 ms/sec
0.023 ms/call
2.91%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(180341781)		 SELECT COUNT(*) FROM (SELECT ? AS 
one FROM "ci_pipelines" WHERE "ci_pipelines"."project_id" 
= ? AND "ci_pipelines"."source" 
!= ? AND ("ci_pipelines"."status" 
IN (?,?,?)) ORDER BY "ci_pipelines"."id" 
DESC LIMIT ?) subquery_for_count
Full query 		1,036
0.52/sec
1.00/call
0.01%		 35,832.69 ms
17.846 ms/sec
34.588 ms/call
1.70%		 1,036
0.52/sec
1.00/call
0.01%		 586,842 blks
292.26 blks/sec
566.45 blks/call
0.05%		 251,030 blks
125.02 blks/sec
242.31 blks/call
11.23%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 31,968.83 ms
15.921 ms/sec
30.858 ms/call
7.94%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(770363114)		 SELECT "users".* FROM "users" WHERE 
("users"."state" IN (?)) AND (ghost 
IS NOT TRUE) AND "users"."user_type" 
IS NULL AND "users"."bot_type" 
IS NULL AND (("users"."name" ILIKE 
? OR "users"."username" ILIKE ?) 
OR "users"."email" = ?) ORDER BY 
CASE
  WHEN users.name = ? THEN 
?
  WHEN users.username = ? THEN 
?
  WHEN users.email = ? THEN ?
 ELSE ?
END
, "users"."name" ASC 
LIMIT ?
Full query 		370
0.18/sec
1.00/call
0.01%		 35,635.67 ms
17.748 ms/sec
96.313 ms/call
1.69%		 3,627
1.81/sec
9.80/call
0.03%		 3,225,624 blks
1.61K blks/sec
8.72K blks/call
0.26%		 23,564 blks
11.74 blks/sec
63.69 blks/call
1.05%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 271.04 ms
0.135 ms/sec
0.733 ms/call
0.07%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		4,477
2.23/sec
1.00/call
0.06%		 35,438.73 ms
17.650 ms/sec
7.916 ms/call
1.68%		 756
0.38/sec
0.17/call
0.01%		 16,968,239 blks
8.46K blks/sec
3.80K blks/call
1.38%		 44,398 blks
22.11 blks/sec
9.92 blks/call
1.99%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 4,110.61 ms
2.047 ms/sec
0.918 ms/call
1.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(62153368)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL)
Full query 		85,292
42.48/sec
1.00/call
1.16%		 35,179.78 ms
17.521 ms/sec
0.412 ms/call
1.67%		 1,264,145
629.58/sec
14.82/call
10.21%		 2,223,033 blks
1.11K blks/sec
26.06 blks/call
0.18%		 58,016 blks
28.89 blks/sec
0.68 blks/call
2.59%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 14,989.44 ms
7.465 ms/sec
0.176 ms/call
3.72%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		624,870
311.20/sec
1.00/call
8.53%		 34,706.64 ms
17.285 ms/sec
0.056 ms/call
1.64%		 618,459
308.01/sec
0.99/call
5.00%		 3,228,604 blks
1.61K blks/sec
5.17 blks/call
0.26%		 17,082 blks
8.51 blks/sec
0.03 blks/call
0.76%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 20,794.72 ms
10.356 ms/sec
0.033 ms/call
5.17%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(1459966649)		 SELECT "notes".* FROM "notes" WHERE 
"notes"."type" IN (?, ?) AND "notes"."noteable_type" 
IN (?, ?) AND "notes"."system" 
= ? AND "notes"."resolved_at" IS 
NULL AND "notes"."noteable_id" 
IN (?, ?, ?, ?, ?, ?, ?, ?, ?, 
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
 ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
 ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
 ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
 ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 
?, ?, ?, ?, ?, ?, ?)
Full query 		353
0.18/sec
1.00/call
0.00%		 31,586.81 ms
15.731 ms/sec
89.481 ms/call
1.50%		 8,139
4.05/sec
23.06/call
0.07%		 340,213 blks
169.44 blks/sec
963.78 blks/call
0.03%		 123,035 blks
61.28 blks/sec
348.54 blks/call
5.50%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 30,306.37 ms
15.093 ms/sec
85.854 ms/call
7.53%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		134
0.07/sec
1.00/call
0.00%		 25,494.70 ms
12.697 ms/sec
190.259 ms/call
1.21%		 134
0.07/sec
1.00/call
0.00%		 19,255,476 blks
9.59K blks/sec
143.70K blks/call
1.57%		 4 blks
0.00 blks/sec
0.03 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.07 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(2454174675)		 SELECT "ci_triggers".* FROM "ci_triggers" 
WHERE "ci_triggers"."token" = ? 
LIMIT ?
Full query 		3,354
1.67/sec
1.00/call
0.05%		 25,429.61 ms
12.665 ms/sec
7.582 ms/call
1.20%		 30
0.01/sec
0.01/call
0.00%		 3,094,608 blks
1.55K blks/sec
922.66 blks/call
0.25%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		134
0.07/sec
1.00/call
0.00%		 23,147.11 ms
11.528 ms/sec
172.740 ms/call
1.10%		 134
0.07/sec
1.00/call
0.00%		 18,577,843 blks
9.26K blks/sec
138.65K blks/call
1.51%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(433678461)		 SELECT "users".* FROM "users" WHERE 
"users"."id" = ? LIMIT ?
Full query 		504,565
251.29/sec
1.00/call
6.89%		 19,663.96 ms
9.793 ms/sec
0.039 ms/call
0.93%		 504,408
251.21/sec
1.00/call
4.07%		 2,018,171 blks
1.01K blks/sec
4.00 blks/call
0.16%		 13 blks
0.01 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.24 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(4166587697)		 SELECT COUNT(*) FROM "projects" 
WHERE "projects"."id" IN (SELECT 
DISTINCT "events"."project_id" 
FROM "events" WHERE (action = ? 
OR (target_type IN (?,?) AND action 
IN (?,?,?)) OR (target_type = ? 
AND action = ?)) AND "events"."author_id" 
= ? AND (created_at > ?)) AND "projects"."visibility_level" 
= ? AND (namespace_id != ?)
Full query 		193
0.10/sec
1.00/call
0.00%		 19,116.90 ms
9.521 ms/sec
99.051 ms/call
0.90%		 193
0.10/sec
1.00/call
0.00%		 37,190 blks
18.52 blks/sec
192.69 blks/call
0.00%		 55,449 blks
27.62 blks/sec
287.30 blks/call
2.48%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 18,598.59 ms
9.263 ms/sec
96.366 ms/call
4.62%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(3842224781)		 SELECT COUNT(*) FROM (SELECT ? AS 
one FROM "ci_pipelines" WHERE "ci_pipelines"."project_id" 
= ? AND "ci_pipelines"."source" 
!= ? AND "ci_pipelines"."ref" = 
? LIMIT ?) subquery_for_count
Full query 		22,015
10.96/sec
1.00/call
0.30%		 19,033.61 ms
9.479 ms/sec
0.865 ms/call
0.90%		 22,015
10.96/sec
1.00/call
0.18%		 3,653,237 blks
1.82K blks/sec
165.94 blks/call
0.30%		 60,089 blks
29.93 blks/sec
2.73 blks/call
2.69%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 7,476.81 ms
3.724 ms/sec
0.340 ms/call
1.86%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2443232796)		 SELECT COUNT(*) FROM (WITH RECURSIVE 
"base_and_ancestors" AS ((SELECT 
"namespaces".* FROM ((SELECT "namespaces".* 
FROM "namespaces" INNER JOIN "members" 
ON "namespaces"."id" = "members"."source_id" 
WHERE "members"."type" = ? AND 
"members"."source_type" = ? AND 
"namespaces"."type" = ? AND "members"."user_id" 
= ? AND "members"."requested_at" 
IS NULL)
UNION
(SELECT namespaces.* 
FROM "projects" INNER JOIN "project_authorizations" 
ON "projects"."id" = "project_authorizations"."project_id" 
INNER JOIN "namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" WHERE 
"project_authorizations"."user_id" 
= ?)) namespaces WHERE "namespaces"."type" 
= ?)
UNION
(SELECT "namespaces".* 
FROM "namespaces", "base_and_ancestors" 
WHERE "namespaces"."type" = ? AND 
"namespaces"."id" = "base_and_ancestors"."parent_id")),
 "base_and_descendants" AS ((SELECT 
"namespaces".* FROM "namespaces" 
INNER JOIN "members" ON "namespaces"."id" 
= "members"."source_id" WHERE "members"."type" 
= ? AND "members"."source_type" 
...
Full query 		401
0.20/sec
1.00/call
0.01%		 18,640.21 ms
9.283 ms/sec
46.484 ms/call
0.88%		 401
0.20/sec
1.00/call
0.00%		 14,816,376 blks
7.38K blks/sec
36.95K blks/call
1.21%		 113 blks
0.06 blks/sec
0.28 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 5.64 ms
0.003 ms/sec
0.014 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.108)

Start: 2020-03-09T19:33:09.531386+00:00
End: 2020-03-09T20:07:04.795741+00:00
Period seconds: 2035.26435
Period age: 00:33:55.264355

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(297297570)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		656
0.32/sec
1.00/call
0.01%		 567,018.01 ms
278.597 ms/sec
864.357 ms/call
27.88%		 7,218
3.55/sec
11.00/call
0.05%		 388,556,980 blks
190.92K blks/sec
592.32K blks/call
35.41%		 342 blks
0.17 blks/sec
0.52 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 20.87 ms
0.010 ms/sec
0.032 ms/call
0.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(3795242938)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		669
0.33/sec
1.00/call
0.01%		 557,413.70 ms
273.878 ms/sec
833.204 ms/call
27.41%		 8,429
4.14/sec
12.60/call
0.06%		 397,353,085 blks
195.24K blks/sec
593.96K blks/call
36.21%		 361 blks
0.18 blks/sec
0.54 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 15.05 ms
0.007 ms/sec
0.022 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(874562767)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		250
0.12/sec
1.00/call
0.00%		 196,729.84 ms
96.661 ms/sec
786.919 ms/call
9.67%		 9,406
4.62/sec
37.62/call
0.06%		 141,434,874 blks
69.50K blks/sec
565.74K blks/call
12.89%		 11 blks
0.01 blks/sec
0.04 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.05 ms
0.001 ms/sec
0.004 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		4,462
2.19/sec
1.00/call
0.06%		 48,591.20 ms
23.875 ms/sec
10.890 ms/call
2.39%		 791
0.39/sec
0.18/call
0.01%		 19,430,756 blks
9.55K blks/sec
4.36K blks/call
1.77%		 84,265 blks
41.40 blks/sec
18.89 blks/call
5.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2,473.15 ms
1.215 ms/sec
0.554 ms/call
2.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(3357552630)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		516,566
253.81/sec
1.00/call
6.53%		 37,626.70 ms
18.487 ms/sec
0.073 ms/call
1.85%		 506,210
248.72/sec
0.98/call
3.50%		 4,604,883 blks
2.27K blks/sec
8.91 blks/call
0.42%		 23,590 blks
11.59 blks/sec
0.05 blks/call
1.40%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3,372.01 ms
1.657 ms/sec
0.007 ms/call
2.75%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(62153368)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL)
Full query 		86,148
42.33/sec
1.00/call
1.09%		 34,958.97 ms
17.177 ms/sec
0.406 ms/call
1.72%		 1,331,367
654.15/sec
15.45/call
9.20%		 2,290,613 blks
1.13K blks/sec
26.59 blks/call
0.21%		 72,765 blks
35.75 blks/sec
0.84 blks/call
4.33%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 11,830.60 ms
5.813 ms/sec
0.137 ms/call
9.66%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		136
0.07/sec
1.00/call
0.00%		 31,927.82 ms
15.687 ms/sec
234.763 ms/call
1.57%		 136
0.07/sec
1.00/call
0.00%		 19,563,263 blks
9.62K blks/sec
143.85K blks/call
1.78%		 70 blks
0.03 blks/sec
0.51 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 4.03 ms
0.002 ms/sec
0.030 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		136
0.07/sec
1.00/call
0.00%		 29,497.27 ms
14.493 ms/sec
216.892 ms/call
1.45%		 136
0.07/sec
1.00/call
0.00%		 18,868,928 blks
9.28K blks/sec
138.75K blks/call
1.72%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(2454174675)		 SELECT  "ci_triggers".* FROM "ci_triggers" 
WHERE "ci_triggers"."token" = ? 
LIMIT ?
Full query 		3,107
1.53/sec
1.00/call
0.04%		 26,774.72 ms
13.155 ms/sec
8.618 ms/call
1.32%		 32
0.02/sec
0.01/call
0.00%		 2,865,515 blks
1.41K blks/sec
922.28 blks/call
0.26%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		1,076
0.53/sec
1.00/call
0.01%		 24,277.77 ms
11.929 ms/sec
22.563 ms/call
1.19%		 1,076
0.53/sec
1.00/call
0.01%		 5,106,356 blks
2.51K blks/sec
4.75K blks/call
0.47%		 305,098 blks
149.91 blks/sec
283.55 blks/call
18.17%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 4,396.65 ms
2.160 ms/sec
4.086 ms/call
3.59%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		626,462
307.80/sec
1.00/call
7.92%		 18,788.99 ms
9.232 ms/sec
0.030 ms/call
0.92%		 626,462
307.80/sec
1.00/call
4.33%		 3,122,315 blks
1.54K blks/sec
4.98 blks/call
0.28%		 12,409 blks
6.10 blks/sec
0.02 blks/call
0.74%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 380.53 ms
0.187 ms/sec
0.001 ms/call
0.31%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		633,707
311.36/sec
1.00/call
8.02%		 18,100.30 ms
8.893 ms/sec
0.029 ms/call
0.89%		 626,955
308.05/sec
0.99/call
4.33%		 3,270,321 blks
1.61K blks/sec
5.16 blks/call
0.30%		 20,465 blks
10.06 blks/sec
0.03 blks/call
1.22%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2,094.38 ms
1.029 ms/sec
0.003 ms/call
1.71%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(3842224781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND "ci_pipelines"."ref" = ? 
LIMIT ?) subquery_for_count
Full query 		22,603
11.11/sec
1.00/call
0.29%		 16,913.24 ms
8.310 ms/sec
0.748 ms/call
0.83%		 22,603
11.11/sec
1.00/call
0.16%		 3,667,932 blks
1.81K blks/sec
162.28 blks/call
0.33%		 67,100 blks
32.97 blks/sec
2.97 blks/call
4.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,275.01 ms
0.626 ms/sec
0.056 ms/call
1.04%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(2747454572)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND (project_mirror_data.next_execution_timestamp 
- project_mirror_data.last_update_at) 
<= ?::interval
AND project_mirror_data.last_update_at 
< NOW() - ?::interval
;
Full query 		136
0.07/sec
1.00/call
0.00%		 16,593.78 ms
8.153 ms/sec
122.013 ms/call
0.82%		 136
0.07/sec
1.00/call
0.00%		 9,232,624 blks
4.54K blks/sec
67.89K blks/call
0.84%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(2555545475)		 SELECT  "projects".* FROM "projects" 
WHERE "projects"."id" = ? LIMIT 
?
Full query 		372,267
182.91/sec
1.00/call
4.71%		 15,855.36 ms
7.790 ms/sec
0.043 ms/call
0.78%		 372,263
182.91/sec
1.00/call
2.57%		 1,577,795 blks
775.23 blks/sec
4.24 blks/call
0.14%		 11 blks
0.01 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.22 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(1252209353)		 SELECT  "deployments".* FROM "deployments" 
WHERE "deployments"."environment_id" 
= ? AND "deployments"."status" 
= ? ORDER BY deployments.id DESC 
LIMIT ?
Full query 		7,604
3.74/sec
1.00/call
0.10%		 14,536.65 ms
7.142 ms/sec
1.912 ms/call
0.71%		 6,657
3.27/sec
0.88/call
0.05%		 2,717,129 blks
1.34K blks/sec
357.33 blks/call
0.25%		 23,627 blks
11.61 blks/sec
3.11 blks/call
1.41%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 5,718.23 ms
2.810 ms/sec
0.752 ms/call
4.67%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(3912557115)		 SELECT  "project_features".* FROM 
"project_features" WHERE "project_features"."project_id" 
= ? LIMIT ?
Full query 		460,059
226.04/sec
1.00/call
5.82%		 14,271.52 ms
7.012 ms/sec
0.031 ms/call
0.70%		 460,059
226.04/sec
1.00/call
3.18%		 1,828,016 blks
898.17 blks/sec
3.97 blks/call
0.17%		 13,126 blks
6.45 blks/sec
0.03 blks/call
0.78%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2,268.25 ms
1.114 ms/sec
0.005 ms/call
1.85%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(180341781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND ("ci_pipelines"."status" 
IN (?,?,?)) ORDER BY "ci_pipelines"."id" 
DESC LIMIT ?) subquery_for_count
Full query 		1,028
0.51/sec
1.00/call
0.01%		 13,373.67 ms
6.571 ms/sec
13.009 ms/call
0.66%		 1,028
0.51/sec
1.00/call
0.01%		 622,258 blks
305.74 blks/sec
605.31 blks/call
0.06%		 266,563 blks
130.97 blks/sec
259.30 blks/call
15.87%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 8,812.15 ms
4.330 ms/sec
8.572 ms/call
7.19%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(2364908498)		 SELECT relnamespace::regnamespace 
as schemaname,
       relname as 
relname,
       pg_total_relation_size(
oid) bytes
  FROM pg_class
 WHERE 
relkind = ?;

Full query 		271
0.13/sec
1.00/call
0.00%		 12,735.95 ms
6.258 ms/sec
46.996 ms/call
0.63%		 104,064
51.13/sec
384.00/call
0.72%		 29,490 blks
14.49 blks/sec
108.82 blks/call
0.00%		 2 blks
0.00 blks/sec
0.01 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.04 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(3856180078)		           SELECT relname,seq_tup_read,
idx_tup_fetch,n_tup_ins,n_tup_upd,
n_tup_del,n_tup_hot_upd,n_dead_tup,
seq_scan
          FROM pg_stat_user_tables
         WHERE relname IN (SELECT 
tablename FROM pg_tables WHERE 
tableowner = ?)
          GROUP 
BY relname,seq_tup_read,idx_tup_fetch,
n_tup_ins,n_tup_upd,n_tup_del,n_tup_hot_upd,
n_dead_tup,seq_scan

Full query 		271
0.13/sec
1.00/call
0.00%		 12,122.61 ms
5.956 ms/sec
44.733 ms/call
0.60%		 86,720
42.61/sec
320.00/call
0.60%		 1,373,177 blks
674.69 blks/sec
5.07K blks/call
0.13%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.109)

Start: 2020-03-09T19:33:10.608089+00:00
End: 2020-03-09T20:07:35.177649+00:00
Period seconds: 2064.56956
Period age: 00:34:24.56956

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(3795242938)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		662
0.32/sec
1.00/call
0.01%		 574,564.71 ms
278.298 ms/sec
867.923 ms/call
26.88%		 7,419
3.59/sec
11.21/call
0.05%		 391,788,934 blks
189.77K blks/sec
591.83K blks/call
36.53%		 353 blks
0.17 blks/sec
0.53 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 17.21 ms
0.008 ms/sec
0.026 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(297297570)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		629
0.30/sec
1.00/call
0.01%		 563,504.38 ms
272.940 ms/sec
895.873 ms/call
26.36%		 7,511
3.64/sec
11.94/call
0.05%		 372,102,139 blks
180.24K blks/sec
591.58K blks/call
34.70%		 283 blks
0.14 blks/sec
0.45 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 49.01 ms
0.024 ms/sec
0.078 ms/call
0.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(874562767)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
= ? AND ("ci_builds"."status" IN 
(?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" = ? AND 
("ci_builds"."status" IN (?)) AND 
"ci_builds"."runner_id" IS NULL 
AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
UNION
...
Full query 		243
0.12/sec
1.00/call
0.00%		 192,365.53 ms
93.175 ms/sec
791.628 ms/call
9.00%		 9,202
4.46/sec
37.87/call
0.06%		 137,483,445 blks
66.60K blks/sec
565.78K blks/call
12.82%		 18 blks
0.01 blks/sec
0.07 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 10.63 ms
0.005 ms/sec
0.044 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		4,544
2.20/sec
1.00/call
0.06%		 66,924.61 ms
32.416 ms/sec
14.728 ms/call
3.13%		 789
0.38/sec
0.17/call
0.01%		 19,287,309 blks
9.35K blks/sec
4.25K blks/call
1.80%		 58,215 blks
28.20 blks/sec
12.81 blks/call
3.10%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 16,643.98 ms
8.062 ms/sec
3.663 ms/call
7.83%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(3357552630)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		524,802
254.19/sec
1.00/call
6.53%		 50,622.89 ms
24.520 ms/sec
0.096 ms/call
2.37%		 514,486
249.20/sec
0.98/call
3.44%		 4,675,184 blks
2.27K blks/sec
8.91 blks/call
0.44%		 28,219 blks
13.67 blks/sec
0.05 blks/call
1.50%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 14,759.00 ms
7.149 ms/sec
0.028 ms/call
6.94%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(62153368)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL)
Full query 		89,313
43.26/sec
1.00/call
1.11%		 36,261.19 ms
17.564 ms/sec
0.406 ms/call
1.70%		 1,350,795
654.27/sec
15.12/call
9.04%		 2,410,560 blks
1.17K blks/sec
26.99 blks/call
0.22%		 76,485 blks
37.05 blks/sec
0.86 blks/call
4.08%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 11,747.31 ms
5.690 ms/sec
0.132 ms/call
5.53%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		138
0.07/sec
1.00/call
0.00%		 32,636.33 ms
15.808 ms/sec
236.495 ms/call
1.53%		 138
0.07/sec
1.00/call
0.00%		 19,944,882 blks
9.67K blks/sec
144.53K blks/call
1.86%		 24 blks
0.01 blks/sec
0.17 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3.10 ms
0.002 ms/sec
0.022 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		637,398
308.73/sec
1.00/call
7.93%		 32,496.40 ms
15.740 ms/sec
0.051 ms/call
1.52%		 630,557
305.42/sec
0.99/call
4.22%		 3,292,033 blks
1.60K blks/sec
5.16 blks/call
0.31%		 18,138 blks
8.79 blks/sec
0.03 blks/call
0.97%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 15,809.29 ms
7.657 ms/sec
0.025 ms/call
7.44%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		138
0.07/sec
1.00/call
0.00%		 30,494.10 ms
14.770 ms/sec
220.972 ms/call
1.43%		 138
0.07/sec
1.00/call
0.00%		 19,210,963 blks
9.31K blks/sec
139.21K blks/call
1.79%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(2454174675)		 SELECT  "ci_triggers".* FROM "ci_triggers" 
WHERE "ci_triggers"."token" = ? 
LIMIT ?
Full query 		3,335
1.62/sec
1.00/call
0.04%		 28,714.43 ms
13.908 ms/sec
8.610 ms/call
1.34%		 28
0.01/sec
0.01/call
0.00%		 3,077,174 blks
1.50K blks/sec
922.69 blks/call
0.29%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		1,042
0.50/sec
1.00/call
0.01%		 28,678.06 ms
13.891 ms/sec
27.522 ms/call
1.34%		 1,042
0.50/sec
1.00/call
0.01%		 5,402,229 blks
2.62K blks/sec
5.19K blks/call
0.50%		 518,819 blks
251.30 blks/sec
497.91 blks/call
27.65%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 6,684.63 ms
3.238 ms/sec
6.415 ms/call
3.15%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(180341781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND ("ci_pipelines"."status" 
IN (?,?,?)) ORDER BY "ci_pipelines"."id" 
DESC LIMIT ?) subquery_for_count
Full query 		1,076
0.52/sec
1.00/call
0.01%		 26,497.28 ms
12.834 ms/sec
24.626 ms/call
1.24%		 1,076
0.52/sec
1.00/call
0.01%		 590,578 blks
286.05 blks/sec
548.86 blks/call
0.06%		 282,569 blks
136.87 blks/sec
262.61 blks/call
15.06%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 21,810.15 ms
10.564 ms/sec
20.270 ms/call
10.26%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		632,800
306.50/sec
1.00/call
7.87%		 20,005.60 ms
9.690 ms/sec
0.032 ms/call
0.94%		 632,785
306.50/sec
1.00/call
4.24%		 3,150,084 blks
1.53K blks/sec
4.98 blks/call
0.29%		 16,402 blks
7.94 blks/sec
0.03 blks/call
0.87%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 560.68 ms
0.272 ms/sec
0.001 ms/call
0.26%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(2068829014)		 SELECT  "uploads".* FROM "uploads" 
WHERE "uploads"."uploader" = ? 
AND "uploads"."path" IN (?, ?) 
ORDER BY "uploads"."id" ASC LIMIT 
?
Full query 		72,970
35.34/sec
1.00/call
0.91%		 18,168.44 ms
8.800 ms/sec
0.249 ms/call
0.85%		 72,967
35.34/sec
1.00/call
0.49%		 797,092 blks
386.08 blks/sec
10.92 blks/call
0.07%		 6,995 blks
3.39 blks/sec
0.10 blks/call
0.37%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 13,416.02 ms
6.498 ms/sec
0.184 ms/call
6.31%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(3842224781)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_pipelines" WHERE 
"ci_pipelines"."project_id" = ? 
AND "ci_pipelines"."source" != 
? AND "ci_pipelines"."ref" = ? 
LIMIT ?) subquery_for_count
Full query 		22,881
11.08/sec
1.00/call
0.28%		 17,640.57 ms
8.544 ms/sec
0.771 ms/call
0.83%		 22,881
11.08/sec
1.00/call
0.15%		 3,643,799 blks
1.77K blks/sec
159.25 blks/call
0.34%		 46,244 blks
22.40 blks/sec
2.02 blks/call
2.46%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 896.27 ms
0.434 ms/sec
0.039 ms/call
0.42%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(2747454572)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND (project_mirror_data.next_execution_timestamp 
- project_mirror_data.last_update_at) 
<= ?::interval
AND project_mirror_data.last_update_at 
< NOW() - ?::interval
;
Full query 		138
0.07/sec
1.00/call
0.00%		 16,528.54 ms
8.006 ms/sec
119.772 ms/call
0.77%		 138
0.07/sec
1.00/call
0.00%		 9,347,524 blks
4.53K blks/sec
67.74K blks/call
0.87%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(3663561168)		 SELECT COUNT(*) FROM "issues" INNER 
JOIN "projects" ON "projects"."id" 
= "issues"."project_id" LEFT JOIN 
project_features ON projects.id 
= project_features.project_id WHERE 
(
      issues.confidential IS 
NOT TRUE
      OR (issues.confidential 
= ?
        AND (issues.author_id 
= ?
          OR EXISTS (SELECT 
? FROM issue_assignees WHERE user_id 
= ? AND issue_id = issues.id)
         OR EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= issues.project_id) AND (project_authorizations.access_level 
>= ?))))) AND (EXISTS (SELECT ? 
FROM "project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id) AND (project_authorizations.access_level 
>= ?)) OR projects.visibility_level 
IN (?,?)) AND ("project_features"."issues_access_level" 
IS NULL OR "project_features"."issues_access_level" 
IN (?,?) OR ("project_features"."issues_access_level" 
= ? AND EXISTS (S...
Full query 		1,324
0.64/sec
1.00/call
0.02%		 16,395.70 ms
7.941 ms/sec
12.383 ms/call
0.77%		 1,324
0.64/sec
1.00/call
0.01%		 306,288 blks
148.35 blks/sec
231.34 blks/call
0.03%		 46,491 blks
22.52 blks/sec
35.11 blks/call
2.48%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 15,377.21 ms
7.448 ms/sec
11.614 ms/call
7.23%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(2555545475)		 SELECT  "projects".* FROM "projects" 
WHERE "projects"."id" = ? LIMIT 
?
Full query 		378,860
183.51/sec
1.00/call
4.71%		 16,380.88 ms
7.934 ms/sec
0.043 ms/call
0.77%		 378,855
183.50/sec
1.00/call
2.54%		 1,604,481 blks
777.15 blks/sec
4.24 blks/call
0.15%		 7 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.95 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(4170569384)		 SELECT  "merge_request_diffs".* 
FROM "merge_request_diffs" WHERE 
"merge_request_diffs"."id" = ? 
LIMIT ?
Full query 		59,245
28.70/sec
1.00/call
0.74%		 16,099.76 ms
7.798 ms/sec
0.272 ms/call
0.75%		 59,245
28.70/sec
1.00/call
0.40%		 291,731 blks
141.30 blks/sec
4.92 blks/call
0.03%		 4,640 blks
2.25 blks/sec
0.08 blks/call
0.25%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 13,960.73 ms
6.762 ms/sec
0.236 ms/call
6.57%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2203646989)		 SELECT "protected_branch_push_access_levels".* 
FROM "protected_branch_push_access_levels" 
WHERE "protected_branch_push_access_levels"."protected_branch_id" 
= ?
Full query 		44,747
21.67/sec
1.00/call
0.56%		 14,737.92 ms
7.138 ms/sec
0.329 ms/call
0.69%		 51,864
25.12/sec
1.16/call
0.35%		 174,394 blks
84.47 blks/sec
3.90 blks/call
0.02%		 10,151 blks
4.92 blks/sec
0.23 blks/call
0.54%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 13,320.73 ms
6.452 ms/sec
0.298 ms/call
6.27%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Conclusions

  • [P1] For some query groups, total_time > 30.00% of overall timing, observed on: node: 10.220.16.106. Such a high percentage means that those queries are "major contributors" to resource consumption on those nodes. In other words, if a query group has total_time which is 30.00% of overall timing, it means that during the observation period, 30.00% of time CPUs were working on that node processing queries from this group.

Recommendations

  • [P1] For some query groups, total_time > 30.00% of overall timing. To reduce total_time for particular query group consider the following tactics:
    • perform query micro-optimization (take particular query examples related to the group, use EXPLAIN and EXPLAIN (BUFFERS, ANALYZE) to optimize it; consider using Joe to boost the optimization process);
    • if the frequency of execution is high (check the calls / second metric), try to find a way to reduce the frequency, changing the application code and/or, if applicable, applying caching.

  Table of contents

L001 Table Sizes

Observations

Data collected: 2020-03-09 20:06:15 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

The list is limited to 20 items. Total: 377.

# Table Rows ▼ Total size Table size Index(es) Size TOAST Size
  ===== TOTAL ===== ~11B 5.46 TiB 2.81 TiB 1.65 TiB 1.01 TiB
1 merge_request_diff_files ~2B 1.70 TiB (30.98%) 885.22 GiB (30.80%) 35.69 GiB (2.12%) 810.30 GiB (78.71%)
2 ci_builds ~440M 0.96 TiB (17.44%) 446.72 GiB (15.54%) 394.35 GiB (23.42%) 133.64 GiB (12.98%)
3 merge_request_diff_commits ~3B 808.76 GiB (14.48%) 618.35 GiB (21.51%) 175.14 GiB (10.40%) 15.28 GiB (1.48%)
4 notes ~255M 342.45 GiB (6.13%) 171.27 GiB (5.96%) 160.38 GiB (9.53%) 10.81 GiB (1.05%)
5 ci_build_trace_sections ~3B 325.72 GiB (5.83%) 156.48 GiB (5.44%) 169.24 GiB (10.05%)
6 events ~581M 156.93 GiB (2.81%) 39.07 GiB (1.36%) 117.86 GiB (7.00%) 8.00 KiB (0.00%)
7 web_hook_logs ~74M 135.74 GiB (2.43%) 88.72 GiB (3.09%) 6.06 GiB (0.36%) 40.96 GiB (3.98%)
8 ci_job_artifacts ~383M 117.26 GiB (2.10%) 57.54 GiB (2.00%) 59.73 GiB (3.55%) 8.00 KiB (0.00%)
9 ci_pipelines ~116M 90.49 GiB (1.62%) 24.17 GiB (0.84%) 66.29 GiB (3.94%) 33.58 MiB (0.00%)
10 merge_requests ~42M 83.20 GiB (1.49%) 19.28 GiB (0.67%) 62.11 GiB (3.69%) 1.81 GiB (0.18%)
11 audit_events ~257M 82.35 GiB (1.47%) 61.45 GiB (2.14%) 20.90 GiB (1.24%) 6.63 MiB (0.00%)
12 sent_notifications ~315M 74.10 GiB (1.33%) 42.04 GiB (1.46%) 32.06 GiB (1.90%) 8.00 KiB (0.00%)
13 issues ~25M 70.26 GiB (1.26%) 16.21 GiB (0.56%) 47.06 GiB (2.79%) 7.01 GiB (0.68%)
14 push_event_payloads ~434M 62.23 GiB (1.11%) 52.83 GiB (1.84%) 9.40 GiB (0.56%) 8.00 KiB (0.00%)
15 projects ~13M 61.30 GiB (1.10%) 3.42 GiB (0.12%) 57.62 GiB (3.42%) 277.99 MiB (0.03%)
16 ci_builds_metadata ~368M 48.40 GiB (0.87%) 19.24 GiB (0.67%) 29.16 GiB (1.73%) 8.00 KiB (0.00%)
17 ci_stages ~215M 48.18 GiB (0.86%) 17.72 GiB (0.62%) 30.47 GiB (1.81%) 8.00 KiB (0.00%)
18 taggings ~188M 47.61 GiB (0.85%) 11.92 GiB (0.41%) 35.69 GiB (2.12%) 8.00 KiB (0.00%)
19 deployments ~53M 34.00 GiB (0.61%) 8.47 GiB (0.29%) 25.53 GiB (1.52%) 8.00 KiB (0.00%)
20 resource_label_events ~37M 23.34 GiB (0.42%) 16.60 GiB (0.58%) 6.75 GiB (0.40%) 192.00 KiB (0.00%)

Conclusions

Recommendations

  Table of contents

L003 Integer (int2, int4) Out-of-range Risks in PKs

Observations

Data collected: 2020-03-09 20:06:24 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

Table PK Type Current max value ▼ Capacity used, %
events id int4 678,056,915 31.57
ci_job_artifacts id int4 498,752,209 23.22
ci_builds id int4 464,992,198 21.65
ci_builds_metadata id int4 384,859,851 17.92
sent_notifications id int4 315,066,358 14.67
web_hook_logs id int4 302,539,832 14.09
notes id int4 302,056,775 14.07
audit_events id int4 257,781,473 12
ci_stages id int4 227,718,686 10.6

Conclusions

  • [P1] High risks of out-of-range errors for an integer column. The columns listed below, being part of a primary key, have high risks to reach 100% of the integer capacity (2^31-1, or 2147483647 for int4 columns, and 2^15-1, or 32767 for int2 columns; see the documentation. If it happens, INSERTs of new rows will not be possible (unless they use some non-incremental values, such as some negative values) and fixing it will require a long downtime. 9 such columns are found:
    • events: reached value 678056915, or 31.57% of int4 capacity
    • ci_job_artifacts: reached value 498752209, or 23.22% of int4 capacity
    • ci_builds: reached value 464992198, or 21.65% of int4 capacity
    • ci_builds_metadata: reached value 384859851, or 17.92% of int4 capacity
    • sent_notifications: reached value 315066358, or 14.67% of int4 capacity
    • etc.

Recommendations

  • [P1] High risks of out-of-range errors for an integer column. Consider using int8 in all PK columns, always. To convert existing columns to int8, consider the following approaches:
    1. Blocking ALTER TABLE .. ALTER COLUMN: a straightforward solution requiring significant downtime (a maintenance window).
    2. "New column": create a new column, update it in batches (running not longer than a few seconds, to avoid blocking issues), and then switch to using it, redefining all the constraints. Notice, that to redefine a primary key constraint, ALTER TABLE .. ALTER COLUMN .. SET NOT NULL will be needed. It is a blocking operation in all Postgres versions up to 12 (where it might be lightweight if a proper CHECK constraint is defined first; such constraint can be defined in a non-blocking way). Since Postgres 11, it is possible to use a trick: when adding a column, use DEFAULT with NOT NULL, it will be a non-blocking operation. For all Postgres versions prior to 11, a specific downtime (maintenance window) will be needed anyway.
    3. "New table": create a new table with the same schema as the existing one, capture all ongoing changes to an additional "log" table, copy existing data from the old table to the new one, and switch. This method, as the previous one, is non-trivial and requires careful development and testing under load (consider using Nancy for database experiments developing this solution). This approach is non-blocking regardless of Postgres version, but it requires significantly more efforts to implement.