postgres-checkup report for gitlab_production: 2020-01-05 (auto-generated)

PostgreSQL Checkup. Project: 'gitlab_production'. Database: 'gitlabhq_production'

Epoch number: '20200105001'

NOTICE: while most reports describe the “current database”, some of them may contain cluster-wide information describing all databases in the cluster.

Last modified at: 2020-01-05 13:45:28 +0000

 

Table of contents

A002 Version Information
A003 Postgres Settings
A004 Cluster Information
A005 Extensions
A006 Postgres Setting Deviations
A007 Altered Settings
D004 pg_stat_statements and pg_stat_kcache Settings
F001 Autovacuum: Current Settings
F002 Autovacuum: Transaction ID Wraparound Check
F003 Autovacuum: Dead Tuples
F004 Autovacuum: Heap Bloat (Estimated)
F005 Autovacuum: Btree Index Bloat (Estimated)
F008 Autovacuum: Resource Usage
G001 Memory-related Settings
G002 Connections and Current Activity
G003 Timeouts, Locks, Deadlocks
H001 Invalid Indexes
H002 Unused Indexes
H003 Non-indexed Foreign Keys
H004 Redundant Indexes
K001 Globally Aggregated Query Metrics
K002 Workload Type ("The First Word" Analysis)
K003 Top-20 Queries by total_time
L001 Table Sizes
L003 Integer (int2, int4) Out-of-range Risks in PKs

Issues found

The empty lines represent reports for which Conclusions and Recommendations are not yet implemented.

Report P1 P2 P3
A002 Version Information 0 2 1
A003 Postgres Settings
A004 Cluster Information
A005 Extensions
A006 Postgres Setting Deviations
A007 Altered Settings
D004 pg_stat_statements and pg_stat_kcache Settings
F001 Autovacuum: Current Settings 0 0 0
F002 Autovacuum: Transaction ID Wraparound Check 0 0 0
F003 Autovacuum: Dead Tuples
F004 Autovacuum: Heap Bloat (Estimated) 1 0 0
F005 Autovacuum: Btree Index Bloat (Estimated) 1 0 0
F008 Autovacuum: Resource Usage
G001 Memory-related Settings
G002 Connections and Current Activity 0 0 0
G003 Timeouts, Locks, Deadlocks
H001 Invalid Indexes
H002 Unused Indexes 0 0 1
H003 Non-indexed Foreign Keys
H004 Redundant Indexes 0 0 1
K001 Globally Aggregated Query Metrics
K002 Workload Type ("The First Word" Analysis)
K003 Top-20 Queries by total_time 0 0 0
L001 Table Sizes
L003 Integer (int2, int4) Out-of-range Risks in PKs 1 0 0

  Table of contents

A002 Version Information

Observations

Data collected: 2020-01-05 13:39:46 +0000 UTC

Master (10.220.16.106)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica servers:

Replica (10.220.16.101)

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu (Ubuntu 9.6.15-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.102)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.103)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.104)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.105)

PostgreSQL 9.6.14 on x86_64-pc-linux-gnu (Ubuntu 9.6.14-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.107)

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu (Ubuntu 9.6.15-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.108)

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu (Ubuntu 9.6.15-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Replica (10.220.16.109)

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu (Ubuntu 9.6.15-1.pgdg16.04+1), compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609, 64-bit

Conclusions

  • [P2] Postgres minor version on the master and replica(s) differ. Nodes 10.220.16.102, 10.220.16.105, 10.220.16.109, 10.220.16.101, 10.220.16.103, 10.220.16.104, 10.220.16.106, 10.220.16.107, 10.220.16.108 use Postgres 9.6.14, 9.6.15.

  • Postgres major version being used is 9.6 and it is currently supported by PostgreSQL Global Development Group. End of life is scheduled 2021-11-11. It means that in case of bugs and security issues, updates (new minor versions) with fixes will be released and available for use. Read more: Versioning Policy.

  • [P2] The minor versions being used (9.6.15, 9.6.14) are not up-to-date (the newest version: 9.6.16). See the full list of changes between 9.6.15 and 9.6.16 and between 9.6.14 and 9.6.16.

Recommendations

  Table of contents

A003 Postgres Settings

Observations

Data collected: 2020-01-05 13:39:47 +0000 UTC

Master (10.220.16.106)

▼ Category Setting Value Unit Pretty value
Autovacuum autovacuum on
Autovacuum autovacuum_analyze_scale_factor 0.005
Autovacuum autovacuum_analyze_threshold 50
Autovacuum autovacuum_freeze_max_age 200000000
Autovacuum autovacuum_max_workers 6
Autovacuum autovacuum_multixact_freeze_max_age 400000000
Autovacuum autovacuum_naptime 30 s
Autovacuum autovacuum_vacuum_cost_delay 5 ms
Autovacuum autovacuum_vacuum_cost_limit 6000
Autovacuum autovacuum_vacuum_scale_factor 0.005
Autovacuum autovacuum_vacuum_threshold 50
Client Connection Defaults / Locale and Formatting DateStyle ISO, MDY
Client Connection Defaults / Locale and Formatting IntervalStyle postgres
Client Connection Defaults / Locale and Formatting TimeZone GMT
Client Connection Defaults / Locale and Formatting client_encoding UTF8
Client Connection Defaults / Locale and Formatting default_text_search_config pg_catalog.english
Client Connection Defaults / Locale and Formatting extra_float_digits 0
Client Connection Defaults / Locale and Formatting lc_collate C.UTF-8
Client Connection Defaults / Locale and Formatting lc_ctype C.UTF-8
Client Connection Defaults / Locale and Formatting lc_messages C
Client Connection Defaults / Locale and Formatting lc_monetary C
Client Connection Defaults / Locale and Formatting lc_numeric C
Client Connection Defaults / Locale and Formatting lc_time C
Client Connection Defaults / Locale and Formatting server_encoding UTF8
Client Connection Defaults / Locale and Formatting timezone_abbreviations Default
Client Connection Defaults / Other Defaults dynamic_library_path $libdir
Client Connection Defaults / Other Defaults gin_fuzzy_search_limit 0
Client Connection Defaults / Other Defaults tcp_keepalives_count 9
Client Connection Defaults / Other Defaults tcp_keepalives_idle 300 s
Client Connection Defaults / Other Defaults tcp_keepalives_interval 75 s
Client Connection Defaults / Shared Library Preloading local_preload_libraries
Client Connection Defaults / Shared Library Preloading session_preload_libraries
Client Connection Defaults / Shared Library Preloading shared_preload_libraries pg_stat_statements
Client Connection Defaults / Statement Behavior bytea_output hex
Client Connection Defaults / Statement Behavior check_function_bodies on
Client Connection Defaults / Statement Behavior client_min_messages notice
Client Connection Defaults / Statement Behavior default_tablespace
Client Connection Defaults / Statement Behavior default_transaction_deferrable off
Client Connection Defaults / Statement Behavior default_transaction_isolation read committed
Client Connection Defaults / Statement Behavior default_transaction_read_only off
Client Connection Defaults / Statement Behavior gin_pending_list_limit 4096 kB 4.00 MiB
Client Connection Defaults / Statement Behavior idle_in_transaction_session_timeout 30000 ms
Client Connection Defaults / Statement Behavior lock_timeout 0 ms
Client Connection Defaults / Statement Behavior search_path "$user", public
Client Connection Defaults / Statement Behavior session_replication_role origin
Client Connection Defaults / Statement Behavior statement_timeout 15000 ms
Client Connection Defaults / Statement Behavior temp_tablespaces
Client Connection Defaults / Statement Behavior transaction_deferrable off
Client Connection Defaults / Statement Behavior transaction_isolation read committed
Client Connection Defaults / Statement Behavior transaction_read_only off
Client Connection Defaults / Statement Behavior vacuum_freeze_min_age 50000000
Client Connection Defaults / Statement Behavior vacuum_freeze_table_age 150000000
Client Connection Defaults / Statement Behavior vacuum_multixact_freeze_min_age 5000000
Client Connection Defaults / Statement Behavior vacuum_multixact_freeze_table_age 150000000
Client Connection Defaults / Statement Behavior xmlbinary base64
Client Connection Defaults / Statement Behavior xmloption content
Connections and Authentication / Connection Settings bonjour off
Connections and Authentication / Connection Settings bonjour_name
Connections and Authentication / Connection Settings listen_addresses 0.0.0.0
Connections and Authentication / Connection Settings max_connections 300
Connections and Authentication / Connection Settings port 5432
Connections and Authentication / Connection Settings superuser_reserved_connections 3
Connections and Authentication / Connection Settings unix_socket_directories /var/opt/gitlab/postgresql
Connections and Authentication / Connection Settings unix_socket_group
Connections and Authentication / Connection Settings unix_socket_permissions 0777
Connections and Authentication / Security and Authentication authentication_timeout 60 s
Connections and Authentication / Security and Authentication db_user_namespace off
Connections and Authentication / Security and Authentication krb_caseins_users off
Connections and Authentication / Security and Authentication krb_server_keyfile FILE:/etc/postgresql-common/krb5.keytab
Connections and Authentication / Security and Authentication password_encryption on
Connections and Authentication / Security and Authentication row_security on
Connections and Authentication / Security and Authentication ssl on
Connections and Authentication / Security and Authentication ssl_ca_file
Connections and Authentication / Security and Authentication ssl_cert_file /var/opt/gitlab/postgresql/server.crt
Connections and Authentication / Security and Authentication ssl_ciphers HIGH:MEDIUM:+3DES:!aNULL:!SSLv3:!TLSv1
Connections and Authentication / Security and Authentication ssl_crl_file
Connections and Authentication / Security and Authentication ssl_ecdh_curve prime256v1
Connections and Authentication / Security and Authentication ssl_key_file /var/opt/gitlab/postgresql/server.key
Connections and Authentication / Security and Authentication ssl_prefer_server_ciphers on
Customized Options pg_stat_statements.max 5000
Customized Options pg_stat_statements.save on
Customized Options pg_stat_statements.track top
Customized Options pg_stat_statements.track_utility on
Developer Options allow_system_table_mods off
Developer Options ignore_checksum_failure off
Developer Options ignore_system_indexes off
Developer Options post_auth_delay 0 s
Developer Options pre_auth_delay 0 s
Developer Options trace_notify off
Developer Options trace_recovery_messages log
Developer Options trace_sort off
Developer Options zero_damaged_pages off
Error Handling data_sync_retry off
Error Handling exit_on_error off
Error Handling restart_after_crash on
File Locations config_file /var/opt/gitlab/postgresql/postgresql.conf
File Locations data_directory /var/opt/gitlab/postgresql/data
File Locations external_pid_file
File Locations hba_file /var/opt/gitlab/postgresql/data/pg_hba.conf
File Locations ident_file /var/opt/gitlab/postgresql/pg_ident.conf
Lock Management deadlock_timeout 5000 ms
Lock Management max_locks_per_transaction 128
Lock Management max_pred_locks_per_transaction 64
Preset Options block_size 8192
Preset Options data_checksums off
Preset Options debug_assertions off
Preset Options integer_datetimes on
Preset Options max_function_args 100
Preset Options max_identifier_length 63
Preset Options max_index_keys 32
Preset Options segment_size 131072 8kB 1.00 GiB
Preset Options server_version 9.6.14
Preset Options server_version_num 90614
Preset Options wal_block_size 8192
Preset Options wal_segment_size 2048 8kB 16.00 MiB
Process Title cluster_name pg-ha-cluster
Process Title update_process_title on
Query Tuning / Genetic Query Optimizer geqo on
Query Tuning / Genetic Query Optimizer geqo_effort 5
Query Tuning / Genetic Query Optimizer geqo_generations 0
Query Tuning / Genetic Query Optimizer geqo_pool_size 0
Query Tuning / Genetic Query Optimizer geqo_seed 0
Query Tuning / Genetic Query Optimizer geqo_selection_bias 2
Query Tuning / Genetic Query Optimizer geqo_threshold 12
Query Tuning / Other Planner Options constraint_exclusion partition
Query Tuning / Other Planner Options cursor_tuple_fraction 0.1
Query Tuning / Other Planner Options default_statistics_target 1000
Query Tuning / Other Planner Options force_parallel_mode off
Query Tuning / Other Planner Options from_collapse_limit 8
Query Tuning / Other Planner Options join_collapse_limit 8
Query Tuning / Planner Cost Constants cpu_index_tuple_cost 0.005
Query Tuning / Planner Cost Constants cpu_operator_cost 0.0025
Query Tuning / Planner Cost Constants cpu_tuple_cost 0.01
Query Tuning / Planner Cost Constants effective_cache_size 43352064 8kB 330.75 GiB
Query Tuning / Planner Cost Constants min_parallel_relation_size 1024 8kB 8.00 MiB
Query Tuning / Planner Cost Constants parallel_setup_cost 1000
Query Tuning / Planner Cost Constants parallel_tuple_cost 0.1
Query Tuning / Planner Cost Constants random_page_cost 1.5
Query Tuning / Planner Cost Constants seq_page_cost 4
Query Tuning / Planner Method Configuration enable_bitmapscan on
Query Tuning / Planner Method Configuration enable_hashagg on
Query Tuning / Planner Method Configuration enable_hashjoin on
Query Tuning / Planner Method Configuration enable_indexonlyscan on
Query Tuning / Planner Method Configuration enable_indexscan on
Query Tuning / Planner Method Configuration enable_material on
Query Tuning / Planner Method Configuration enable_mergejoin on
Query Tuning / Planner Method Configuration enable_nestloop on
Query Tuning / Planner Method Configuration enable_seqscan on
Query Tuning / Planner Method Configuration enable_sort on
Query Tuning / Planner Method Configuration enable_tidscan on
Replication track_commit_timestamp off
Replication / Master Server synchronous_standby_names
Replication / Master Server vacuum_defer_cleanup_age 0
Replication / Sending Servers max_replication_slots 32
Replication / Sending Servers max_wal_senders 32
Replication / Sending Servers wal_keep_segments 2048
Replication / Sending Servers wal_sender_timeout 60000 ms
Replication / Standby Servers hot_standby on
Replication / Standby Servers hot_standby_feedback on
Replication / Standby Servers max_standby_archive_delay 30000 ms
Replication / Standby Servers max_standby_streaming_delay 30000 ms
Replication / Standby Servers wal_receiver_status_interval 10 s
Replication / Standby Servers wal_receiver_timeout 60000 ms
Replication / Standby Servers wal_retrieve_retry_interval 5000 ms
Reporting and Logging / What to Log application_name checkup
Reporting and Logging / What to Log debug_pretty_print on
Reporting and Logging / What to Log debug_print_parse off
Reporting and Logging / What to Log debug_print_plan off
Reporting and Logging / What to Log debug_print_rewritten off
Reporting and Logging / What to Log log_autovacuum_min_duration 0 ms
Reporting and Logging / What to Log log_checkpoints on
Reporting and Logging / What to Log log_connections on
Reporting and Logging / What to Log log_disconnections on
Reporting and Logging / What to Log log_duration off
Reporting and Logging / What to Log log_error_verbosity default
Reporting and Logging / What to Log log_hostname off
Reporting and Logging / What to Log log_line_prefix %m [%p, %x]: [%l-1] user=%u, db=%d, app=%a, client=%h
Reporting and Logging / What to Log log_lock_waits on
Reporting and Logging / What to Log log_replication_commands off
Reporting and Logging / What to Log log_statement ddl
Reporting and Logging / What to Log log_temp_files 0 kB 0.00 bytes
Reporting and Logging / What to Log log_timezone GMT
Reporting and Logging / When to Log log_min_duration_statement 1000 ms
Reporting and Logging / When to Log log_min_error_statement error
Reporting and Logging / When to Log log_min_messages warning
Reporting and Logging / Where to Log event_source PostgreSQL
Reporting and Logging / Where to Log log_destination stderr
Reporting and Logging / Where to Log log_directory /var/log/gitlab/postgresql
Reporting and Logging / Where to Log log_file_mode 0640
Reporting and Logging / Where to Log log_filename postgresql.log
Reporting and Logging / Where to Log log_rotation_age 0 min
Reporting and Logging / Where to Log log_rotation_size 0 kB 0.00 bytes
Reporting and Logging / Where to Log log_truncate_on_rotation off
Reporting and Logging / Where to Log logging_collector on
Reporting and Logging / Where to Log syslog_facility local0
Reporting and Logging / Where to Log syslog_ident postgres
Reporting and Logging / Where to Log syslog_sequence_numbers on
Reporting and Logging / Where to Log syslog_split_messages on
Resource Usage / Asynchronous Behavior backend_flush_after 0 8kB 0.00 bytes
Resource Usage / Asynchronous Behavior effective_io_concurrency 100
Resource Usage / Asynchronous Behavior max_parallel_workers_per_gather 0
Resource Usage / Asynchronous Behavior max_worker_processes 8
Resource Usage / Asynchronous Behavior old_snapshot_threshold -1 min
Resource Usage / Background Writer bgwriter_delay 200 ms
Resource Usage / Background Writer bgwriter_flush_after 64 8kB 512.00 KiB
Resource Usage / Background Writer bgwriter_lru_maxpages 100
Resource Usage / Background Writer bgwriter_lru_multiplier 2
Resource Usage / Cost-Based Vacuum Delay vacuum_cost_delay 0 ms
Resource Usage / Cost-Based Vacuum Delay vacuum_cost_limit 200
Resource Usage / Cost-Based Vacuum Delay vacuum_cost_page_dirty 20
Resource Usage / Cost-Based Vacuum Delay vacuum_cost_page_hit 1
Resource Usage / Cost-Based Vacuum Delay vacuum_cost_page_miss 10
Resource Usage / Disk temp_file_limit -1 kB
Resource Usage / Kernel Resources max_files_per_process 1000
Resource Usage / Memory autovacuum_work_mem -1 kB
Resource Usage / Memory dynamic_shared_memory_type posix
Resource Usage / Memory huge_pages try
Resource Usage / Memory maintenance_work_mem 2097152 kB 2.00 GiB
Resource Usage / Memory max_prepared_transactions 0
Resource Usage / Memory max_stack_depth 2048 kB 2.00 MiB
Resource Usage / Memory replacement_sort_tuples 150000
Resource Usage / Memory shared_buffers 14450688 8kB 110.25 GiB
Resource Usage / Memory temp_buffers 1024 8kB 8.00 MiB
Resource Usage / Memory track_activity_query_size 1024
Resource Usage / Memory work_mem 102400 kB 100.00 MiB
Statistics / Monitoring log_executor_stats off
Statistics / Monitoring log_parser_stats off
Statistics / Monitoring log_planner_stats off
Statistics / Monitoring log_statement_stats off
Statistics / Query and Index Statistics Collector stats_temp_directory pg_stat_tmp
Statistics / Query and Index Statistics Collector track_activities on
Statistics / Query and Index Statistics Collector track_counts on
Statistics / Query and Index Statistics Collector track_functions none
Statistics / Query and Index Statistics Collector track_io_timing on
Version and Platform Compatibility / Other Platforms and Clients transform_null_equals off
Version and Platform Compatibility / Previous PostgreSQL Versions array_nulls on
Version and Platform Compatibility / Previous PostgreSQL Versions backslash_quote safe_encoding
Version and Platform Compatibility / Previous PostgreSQL Versions default_with_oids off
Version and Platform Compatibility / Previous PostgreSQL Versions escape_string_warning on
Version and Platform Compatibility / Previous PostgreSQL Versions lo_compat_privileges off
Version and Platform Compatibility / Previous PostgreSQL Versions operator_precedence_warning off
Version and Platform Compatibility / Previous PostgreSQL Versions quote_all_identifiers off
Version and Platform Compatibility / Previous PostgreSQL Versions sql_inheritance on
Version and Platform Compatibility / Previous PostgreSQL Versions standard_conforming_strings on
Version and Platform Compatibility / Previous PostgreSQL Versions synchronize_seqscans on
Write-Ahead Log / Archiving archive_command /usr/bin/envdir /etc/wal-e.d/env /opt/wal-e/bin/wal-e wal-push %p
Write-Ahead Log / Archiving archive_mode on
Write-Ahead Log / Archiving archive_timeout 60 s
Write-Ahead Log / Checkpoints checkpoint_completion_target 0.7
Write-Ahead Log / Checkpoints checkpoint_flush_after 32 8kB 256.00 KiB
Write-Ahead Log / Checkpoints checkpoint_timeout 300 s
Write-Ahead Log / Checkpoints checkpoint_warning 30 s
Write-Ahead Log / Checkpoints max_wal_size 320 16MB 5.00 GiB
Write-Ahead Log / Checkpoints min_wal_size 5 16MB 80.00 MiB
Write-Ahead Log / Settings commit_delay 0
Write-Ahead Log / Settings commit_siblings 5
Write-Ahead Log / Settings fsync on
Write-Ahead Log / Settings full_page_writes on
Write-Ahead Log / Settings synchronous_commit on
Write-Ahead Log / Settings wal_buffers 2048 8kB 16.00 MiB
Write-Ahead Log / Settings wal_compression off
Write-Ahead Log / Settings wal_level replica
Write-Ahead Log / Settings wal_log_hints on
Write-Ahead Log / Settings wal_sync_method fdatasync
Write-Ahead Log / Settings wal_writer_delay 200 ms
Write-Ahead Log / Settings wal_writer_flush_after 128 8kB 1.00 MiB

  Table of contents

A004 Cluster Information

Observations

Data collected: 2020-01-05 13:39:48 +0000 UTC

▼ Indicator 10.220.16.106 10.220.16.101 10.220.16.102 10.220.16.103 10.220.16.104 10.220.16.105 10.220.16.107 10.220.16.108 10.220.16.109
Config file /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/postgresql.conf
Role Master Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false) Replica (delay: 00:00:00; paused: false)
Replicas async/streaming: 10.220.16.107, 10.220.16.102, 10.220.16.104, 10.220.16.101, 10.220.16.110, 10.220.16.103, 10.220.16.105, 10.220.16.111, 10.220.16.108, 10.220.16.109, 10.220.16.112
Started At 2019-10-28 16:07:17+00 2019-12-12 07:32:00+00 2019-12-12 07:32:03+00 2019-12-12 07:32:05+00 2019-12-12 07:32:03+00 2019-12-12 07:32:09+00 2019-12-12 07:31:59+00 2019-12-12 07:32:09+00 2019-12-12 07:32:10+00
Uptime 68 days 21:30:48 24 days 06:03:44 24 days 06:04:09 24 days 06:04:35 24 days 06:05:05 24 days 06:05:27 24 days 06:06:50 24 days 06:07:11 24 days 06:07:39
Checkpoints 43362 261256 264575 263808 263644 263097 261109 261117 261508
Forced Checkpoints 77.6% 99.2% 99.2% 99.2% 99.2% 99.2% 99.2% 99.2% 99.2%
Checkpoint MB/sec 11.549858 10.436410 10.437682 10.437651 10.437877 10.437909 10.437067 10.436560 10.436469
Database Name gitlabhq_production gitlabhq_production gitlabhq_production gitlabhq_production gitlabhq_production gitlabhq_production gitlabhq_production gitlabhq_production gitlabhq_production
Database Size 4699 GB 4699 GB 4699 GB 4699 GB 4699 GB 4699 GB 4699 GB 4699 GB 4699 GB
Stats Since 2019-10-28 16:07:35+00 2019-12-12 07:32:12+00 2019-12-12 07:32:11+00 2019-12-12 07:32:13+00 2019-12-12 07:32:12+00 2019-12-12 07:32:20+00 2019-12-12 07:32:11+00 2019-12-12 07:32:21+00 2019-12-12 07:32:25+00
Stats Age 68 days 21:30:30 24 days 06:03:32 24 days 06:04:01 24 days 06:04:26 24 days 06:04:56 24 days 06:05:15 24 days 06:06:38 24 days 06:06:58 24 days 06:07:24
Cache Effectiveness 99.25% 99.73% 99.74% 99.73% 99.74% 99.74% 99.73% 99.73% 99.73%
Successful Commits 99.67% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00%
Conflicts 4 2 0 0 0 0 0 0 0
Temp Files: total size 500 GB 1175 GB 1214 GB 1164 GB 1192 GB 1212 GB 1154 GB 1204 GB 1207 GB
Temp Files: total number of files 4061 15425 15986 14550 15103 16138 14297 15944 16095
Temp Files: total number of files per day 58 642 666 606 629 672 595 664 670
Temp Files: avg file size 126 MB 78 MB 78 MB 82 MB 81 MB 77 MB 83 MB 77 MB 77 MB
Deadlocks 871 0 0 0 0 0 0 0 0
Deadlocks per day 12 0 0 0 0 0 0 0 0

Databases sizes

Database ▼ Size
gitlabhq_production 4.59 TiB
postgres 6.99 MiB
template0 6.91 MiB
template1 6.91 MiB

Conclusions

Recommendations

  Table of contents

A005 Extensions

Observations

Data collected: 2020-01-05 13:39:48 +0000 UTC

Master (10.220.16.106)

▼ Database Extension name Installed version Default version Is old
gitlabhq_production btree_gin 1.0 1.0
gitlabhq_production dblink 1.2 1.2
gitlabhq_production pageinspect 1.5 1.5
gitlabhq_production pg_buffercache 1.2 1.2
gitlabhq_production pg_repack 1.4.4 1.4.4
gitlabhq_production pg_stat_statements 1.4 1.4
gitlabhq_production pg_trgm 1.3 1.3
gitlabhq_production pgstattuple 1.4 1.4
gitlabhq_production plpgsql 1.0 1.0

Conclusions

Recommendations

  Table of contents

A006 Postgres Setting Deviations

Observations

Data collected: 2020-01-05 13:39:49 +0000 UTC

Settings (pg_settings) that Differ

▼ Setting 10.220.16.106 10.220.16.101 10.220.16.102 10.220.16.103 10.220.16.104 10.220.16.105 10.220.16.107 10.220.16.108 10.220.16.109
server_version 9.6.14 9.6.15 9.6.14 9.6.14 9.6.14 9.6.14 9.6.15 9.6.15 9.6.15
server_version_num 90614 90615 90614 90614 90614 90614 90615 90615 90615

Configs(pg_config) that differ

▼ Config 10.220.16.106 10.220.16.101 10.220.16.102 10.220.16.103 10.220.16.104 10.220.16.105 10.220.16.107 10.220.16.108 10.220.16.109
CONFIGURE '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.15-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.14-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.15-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.15-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=/usr/include' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-silent-rules' '--libdir=/usr/lib/x86_64-linux-gnu' '--libexecdir=/usr/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--disable-dependency-tracking' '--with-tcl' '--with-perl' '--with-python' '--with-pam' '--with-openssl' '--with-libxml' '--with-libxslt' 'PYTHON=/usr/bin/python' '--mandir=/usr/share/postgresql/9.6/man' '--docdir=/usr/share/doc/postgresql-doc-9.6' '--sysconfdir=/etc/postgresql-common' '--datarootdir=/usr/share/' '--datadir=/usr/share/postgresql/9.6' '--bindir=/usr/lib/postgresql/9.6/bin' '--libdir=/usr/lib/x86_64-linux-gnu/' '--libexecdir=/usr/lib/postgresql/' '--includedir=/usr/include/postgresql/' '--enable-nls' '--enable-integer-datetimes' '--enable-thread-safety' '--enable-tap-tests' '--enable-debug' '--enable-dtrace' '--disable-rpath' '--with-uuid=e2fs' '--with-gnu-ld' '--with-pgport=5432' '--with-system-tzdata=/usr/share/zoneinfo' '--with-systemd' '--with-selinux' 'MKDIR_P=/bin/mkdir -p' 'TAR=/bin/tar' 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -pie -fno-omit-frame-pointer' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' 'DPKG_VERSION=9.6.15-1.pgdg16.04+1' 'DPKG_VENDOR=Ubuntu' '--with-gssapi' '--with-ldap' '--with-includes=/usr/include/mit-krb5' '--with-libs=/usr/lib/mit-krb5' '--with-libs=/usr/lib/x86_64-linux-gnu/mit-krb5' 'build_alias=x86_64-linux-gnu' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
VERSION PostgreSQL 9.6.14 PostgreSQL 9.6.15 PostgreSQL 9.6.14 PostgreSQL 9.6.14 PostgreSQL 9.6.14 PostgreSQL 9.6.14 PostgreSQL 9.6.15 PostgreSQL 9.6.15 PostgreSQL 9.6.15

Conclusions

Recommendations

  Table of contents

A007 Altered Settings

Observations

Data collected: 2020-01-05 13:39:50 +0000 UTC

Master (10.220.16.106)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 9 autovacuum_naptime idle_in_transaction_session_timeout log_connections log_disconnections log_min_duration_statement maintenance_work_mem max_wal_size track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 44 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_rotation_age log_rotation_size log_statement log_temp_files logging_collector random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica servers:

Replica (10.220.16.101)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 7 autovacuum_naptime idle_in_transaction_session_timeout log_connections log_disconnections log_min_duration_statement track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 46 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_destination log_directory log_file_mode log_filename log_line_prefix log_lock_waits log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.102)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.103)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.104)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.105)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.107)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.108)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Replica (10.220.16.109)

Source Settings count Changed settings
/var/opt/gitlab/postgresql/data/postgresql.auto.conf 4 autovacuum_naptime idle_in_transaction_session_timeout track_io_timing wal_keep_segments
/var/opt/gitlab/postgresql/postgresql.base.conf 2 autovacuum_freeze_max_age min_wal_size
/var/opt/gitlab/postgresql/postgresql.conf 49 archive_command archive_mode archive_timeout autovacuum_analyze_scale_factor autovacuum_max_workers autovacuum_vacuum_cost_delay autovacuum_vacuum_cost_limit autovacuum_vacuum_scale_factor checkpoint_completion_target checkpoint_timeout deadlock_timeout default_statistics_target default_text_search_config effective_cache_size effective_io_concurrency hot_standby_feedback lc_messages log_autovacuum_min_duration log_checkpoints log_connections log_destination log_directory log_disconnections log_file_mode log_filename log_line_prefix log_lock_waits log_min_duration_statement log_rotation_age log_rotation_size log_statement log_temp_files logging_collector maintenance_work_mem max_wal_size random_page_cost shared_buffers shared_preload_libraries ssl ssl_ca_file ssl_cert_file ssl_ciphers ssl_key_file tcp_keepalives_count tcp_keepalives_idle tcp_keepalives_interval track_activity_query_size unix_socket_directories work_mem
default 211

Conclusions

Recommendations

  Table of contents

D004 pg_stat_statements and pg_stat_kcache Settings

Observations

Data collected: 2020-01-05 13:39:51 +0000 UTC

Master (10.220.16.106)

pg_stat_statements extension settings

Setting Value Unit Type Min value Max value
pg_stat_statements.max 5000 integer 100 2147483647
pg_stat_statements.save on bool
pg_stat_statements.track top enum
pg_stat_statements.track_utility on bool

Conclusions

Recommendations

  Table of contents

F001 Autovacuum: Current Settings

Observations

Data collected: 2020-01-05 13:39:52 +0000 UTC

Master (10.220.16.106)

▼ Setting name Value Unit Pretty value
autovacuum on
autovacuum_analyze_scale_factor 0.005
autovacuum_analyze_threshold 50
autovacuum_freeze_max_age 200000000
autovacuum_max_workers 6
autovacuum_multixact_freeze_max_age 400000000
autovacuum_naptime 30 s
autovacuum_vacuum_cost_delay 5 ms
autovacuum_vacuum_cost_limit 6000
autovacuum_vacuum_scale_factor 0.005
autovacuum_vacuum_threshold 50
autovacuum_work_mem -1 kB
maintenance_work_mem 2097152 kB 2.00 GiB
vacuum_cost_delay 0 ms
vacuum_cost_limit 200
vacuum_cost_page_dirty 20
vacuum_cost_page_hit 1
vacuum_cost_page_miss 10
vacuum_defer_cleanup_age 0
vacuum_freeze_min_age 50000000
vacuum_freeze_table_age 150000000
vacuum_multixact_freeze_min_age 5000000
vacuum_multixact_freeze_table_age 150000000

Tuned tables

▼ Namespace Relation Options
pg_toast pg_toast_33145 autovacuum_enabled=true
pg_toast pg_toast_33153 autovacuum_enabled=true
pg_toast pg_toast_33161 autovacuum_enabled=true
pg_toast pg_toast_33208 autovacuum_enabled=true
pg_toast pg_toast_33221 autovacuum_enabled=true
pg_toast pg_toast_33229 autovacuum_enabled=true
pg_toast pg_toast_33237 autovacuum_enabled=true
pg_toast pg_toast_33248 autovacuum_enabled=true
pg_toast pg_toast_33262 autovacuum_enabled=true
pg_toast pg_toast_33272 autovacuum_enabled=true
pg_toast pg_toast_33322 autovacuum_enabled=true
pg_toast pg_toast_33356 autovacuum_enabled=true
pg_toast pg_toast_33369 autovacuum_enabled=true
pg_toast pg_toast_33383 autovacuum_enabled=true
pg_toast pg_toast_33393 autovacuum_enabled=true
pg_toast pg_toast_33400 autovacuum_enabled=true
pg_toast pg_toast_33426 autovacuum_enabled=true
pg_toast pg_toast_33439 autovacuum_enabled=true
pg_toast pg_toast_33447 autovacuum_enabled=true
pg_toast pg_toast_33462 autovacuum_enabled=true
pg_toast pg_toast_33476 autovacuum_enabled=true
pg_toast pg_toast_33497 autovacuum_enabled=true
pg_toast pg_toast_33506 autovacuum_enabled=true
pg_toast pg_toast_33514 autovacuum_enabled=true
pg_toast pg_toast_33527 autovacuum_enabled=true
pg_toast pg_toast_33541 autovacuum_enabled=true
pg_toast pg_toast_33553 autovacuum_enabled=true
pg_toast pg_toast_33568 autovacuum_enabled=true
pg_toast pg_toast_33587 autovacuum_enabled=true
pg_toast pg_toast_33597 autovacuum_enabled=true
pg_toast pg_toast_33614 autovacuum_enabled=true
pg_toast pg_toast_33625 autovacuum_enabled=true
pg_toast pg_toast_33634 autovacuum_enabled=true
pg_toast pg_toast_33642 autovacuum_enabled=true
pg_toast pg_toast_33650 autovacuum_enabled=true
pg_toast pg_toast_33659 autovacuum_enabled=true
pg_toast pg_toast_33667 autovacuum_enabled=true
pg_toast pg_toast_33675 autovacuum_enabled=true
pg_toast pg_toast_33696 autovacuum_enabled=true
pg_toast pg_toast_33706 autovacuum_enabled=true
pg_toast pg_toast_33749 autovacuum_enabled=true
pg_toast pg_toast_33756 autovacuum_enabled=true
pg_toast pg_toast_33768 autovacuum_enabled=true
pg_toast pg_toast_33776 autovacuum_enabled=true
pg_toast pg_toast_33791 autovacuum_enabled=true
pg_toast pg_toast_33801 autovacuum_enabled=true
pg_toast pg_toast_33824 autovacuum_enabled=true
pg_toast pg_toast_33835 autovacuum_enabled=true
pg_toast pg_toast_33851 autovacuum_enabled=true
pg_toast pg_toast_33863 autovacuum_enabled=true
pg_toast pg_toast_33872 autovacuum_enabled=true
pg_toast pg_toast_33885 autovacuum_enabled=true
pg_toast pg_toast_33898 autovacuum_enabled=true
pg_toast pg_toast_33909 autovacuum_enabled=true
pg_toast pg_toast_33959 autovacuum_enabled=true
public abuse_reports autovacuum_enabled=true
public appearances autovacuum_enabled=true
public application_settings autovacuum_enabled=true
public approval_merge_request_rules_users autovacuum_enabled=on
public approvals autovacuum_enabled=true
public approver_groups autovacuum_enabled=true
public approvers autovacuum_enabled=on
public audit_events autovacuum_enabled=true
public award_emoji autovacuum_enabled=on
public boards autovacuum_enabled=true
public broadcast_messages autovacuum_enabled=true
public ci_builds autovacuum_enabled=true
public ci_pipelines autovacuum_enabled=true
public ci_runner_projects autovacuum_enabled=true
public ci_runners autovacuum_enabled=on
public ci_trigger_requests autovacuum_enabled=true
public ci_triggers autovacuum_enabled=true
public ci_variables autovacuum_enabled=true
public deploy_keys_projects autovacuum_enabled=true
public deployments autovacuum_enabled=on
public emails autovacuum_enabled=true
public environments autovacuum_enabled=true
public forked_project_links autovacuum_enabled=true
public geo_nodes autovacuum_enabled=true
public historical_data autovacuum_enabled=true
public identities autovacuum_enabled=true
public index_statuses autovacuum_enabled=true
public issue_metrics autovacuum_enabled=true
public issues autovacuum_enabled=true
public keys autovacuum_enabled=true
public label_links autovacuum_enabled=true
public label_priorities autovacuum_enabled=true
public labels autovacuum_enabled=on
public ldap_group_links autovacuum_enabled=true
public lfs_objects autovacuum_enabled=true
public lfs_objects_projects autovacuum_enabled=true
public licenses autovacuum_enabled=true
public lists autovacuum_enabled=true
public members autovacuum_enabled=true
public merge_request_diffs autovacuum_enabled=true
public merge_request_metrics autovacuum_enabled=true
public merge_requests autovacuum_enabled=true
public merge_requests_closing_issues autovacuum_enabled=true
public milestones autovacuum_enabled=on
public namespaces autovacuum_enabled=on
public notes autovacuum_enabled=true
public notification_settings autovacuum_enabled=on
public oauth_access_grants autovacuum_enabled=true
public oauth_access_tokens autovacuum_enabled=true
public oauth_applications autovacuum_enabled=true
public pages_domains autovacuum_enabled=true
public path_locks autovacuum_enabled=true
public personal_access_tokens autovacuum_enabled=true
public project_authorizations autovacuum_enabled=true
public project_features autovacuum_enabled=true
public project_group_links autovacuum_enabled=true
public project_import_data autovacuum_enabled=on
public project_mirror_data autovacuum_vacuum_cost_limit=10000
autovacuum_enabled=on
public project_repository_states autovacuum_enabled=on
public projects autovacuum_vacuum_cost_limit=7000
autovacuum_enabled=on
public protected_branch_merge_access_levels autovacuum_enabled=true
public protected_branch_push_access_levels autovacuum_enabled=true
public protected_branches autovacuum_enabled=true
public push_rules autovacuum_enabled=true
public releases autovacuum_enabled=on
public remote_mirrors autovacuum_enabled=true
public sent_notifications autovacuum_enabled=true
public services autovacuum_enabled=on
public snippets autovacuum_enabled=on
public spam_logs autovacuum_enabled=true
public subscriptions autovacuum_enabled=true
public taggings autovacuum_enabled=true
public tags autovacuum_enabled=true
public todos autovacuum_enabled=true
public trending_projects autovacuum_enabled=true
public u2f_registrations autovacuum_enabled=true
public untracked_files_for_uploads autovacuum_enabled=on
public user_agent_details autovacuum_enabled=true
public users autovacuum_enabled=on
public users_star_projects autovacuum_enabled=true
public web_hooks autovacuum_enabled=true

Replicas settings

Setting 10.220.16.101 10.220.16.102 10.220.16.103 10.220.16.104 10.220.16.105 10.220.16.107 10.220.16.108 10.220.16.109
hot_standby_feedback on on on on on on on on

Conclusions

Recommendations

All good, no recommendations here.

  Table of contents

F002 Autovacuum: Transaction ID Wraparound Check

Observations

  • Data collected: 2020-01-05 13:38:08 +0000 UTC
  • Current database: gitlabhq_production

Master (10.220.16.106)

Databases

# Database ▼ Age Capacity used, % Warning datfrozenxid
1 gitlabhq_production 199865870 10.25 3782895890
2 postgres 175167286 8.98 3807594474
3 template0 69729065 3.58 3913032695
4 template1 69722977 3.58 3913038783

Tables in the observed database

The list is limited to 10 items. Total: 309.

# Relation Age ▼ Capacity used, % Warning rel_relfrozenxid toast_relfrozenxid
1 group_deletion_schedules 199865870 10.25 3782895890 0
2 import_failures 199789420 10.25 3782972340 3782998211
3 board_assignees 199690643 10.24 3783071117 0
4 packages_packages 197184794 10.11 3785576966 3913270350
5 packages_dependencies 196989494 10.1 3785772266 3821700977
6 notes* 195906642 10.05 3918744956 3786855118
7 ci_subscriptions_projects 193324368 9.91 3789437392 0
8 issue_metrics* 192563827 9.88 3790197933 0
9 application_settings* 189460560 9.72 3793301200 3793310973
10 packages_build_infos 189460560 9.72 3793301200 0

* This table has specific autovacuum settings. See 'F001 Autovacuum: Current settings'

Conclusions

Recommendations

All good, no recommendations here.

  Table of contents

F003 Autovacuum: Dead Tuples

Observations

Data collected: 2020-01-05 13:38:09 +0000 UTC
Current database: gitlabhq_production
Stats reset: 2 mons 7 days 21:31:00 ago (2019-10-28 16:07:35 +0000 UTC)

Master (10.220.16.106)

The list is limited to 20 items. Total: 199.

# Relation reltype Since last autovacuum Since last vacuum Autovacuum Count Vacuum Count n_tup_ins n_tup_upd n_tup_del pg_class.reltuples n_live_tup n_dead_tup ▼Dead Tuples Ratio, %
1 pg_toast.pg_toast_2619 t 00:00:04.981429 97668 0 15178421 0 15177711 18249 18074 258 1.41
2 project_feature_usages r 00:01:21.731546 152131 0 5597 24446519 427 16070 16070 141 0.87
3 pg_catalog.pg_statistic r 00:05:39.659642 21304 0 137 2648410 24 18750 18739 143 0.76
4 clusters_applications_helm r 19:48:56.564365 486 0 4950 18444 3174 17754 17757 131 0.73
5 project_mirror_data* r 00:00:21.69651 47 days 06:18:56.94176 59137 1 171013 1379004350 44483 2342410 2342410 16768 0.71
6 pg_toast.pg_toast_33776* t 01:27:31.550685 8344 0 631043 0 630117 19031 19031 132 0.69
7 packages_dependency_links r 2 days 13:20:17.413395 55 0 42776 0 9184 33420 33478 211 0.63
8 clusters_kubernetes_namespaces r 3 days 12:38:19.896923 34 0 10777 1609 5409 38742 38824 237 0.61
9 board_group_recent_visits r 04:22:47.318382 3114 0 5716 269978 1198 28235 28236 152 0.54
10 pg_toast.pg_toast_21968240 t 10:43:45.969181 1403 0 51566 0 48336 25166 25161 136 0.54
11 clusters r 1 day 17:24:25.803232 64 0 8239 35457 5335 29849 29855 160 0.53
12 project_authorizations* r 2 days 03:52:46.141014 34 0 22145365 0 7666606 57318944 57322629 281348 0.49
13 protected_branch_unprotect_access_levels r 00:31:09.822589 4798 0 417629 0 413466 27376 27377 129 0.47
14 merge_requests* r 23:05:20.051308 519 0 5441046 71813265 1394803 37099024 37110042 175919 0.47
15 list_user_preferences r 19:11:12.443639 2184 0 96498 316979 15898 151883 151955 714 0.47
16 namespace_root_storage_statistics r 00:33:09.268867 7814 0 197755 20132446 4178 817343 817390 3775 0.46
17 ci_runner_projects* r 3 days 23:28:44.741675 26 0 69422 0 53375 405980 406328 1842 0.45
18 ci_variables* r 1 day 05:24:14.251417 1412 0 201489 1476190 56157 1042645 1043033 4596 0.44
19 ci_builds* r 1 day 16:44:55.341743 102 0 58284812 163534603 5172121 371888800 371989112 1636202 0.44
20 packages_packages r 4 days 06:39:16.7864 23 0 27028 0 4862 56823 57060 251 0.44

* This table has specific autovacuum settings. See 'F001 Autovacuum: Current settings'

Conclusions

Recommendations

  Table of contents

F004 Autovacuum: Heap Bloat (Estimated)

⚠ This report is based on estimations. The errors in bloat estimates may be significant (in some cases, up to 15% and even more). Use it only as an indicator of potential issues.

Observations

Data collected: 2020-01-05 13:38:09 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

The list is limited to 20 items. Total: 301.

# Table Real Size ▼ Estimated bloat Est. bloat, bytes Est. bloat factor Est. bloat level, % Live Data Size Last vacuum Fillfactor
  ===== TOTAL ===== 3.33 TiB 74.04 GiB 79,491,923,968 1.02 2.17 ~3.26 TiB
1 merge_request_diff_files 1.47 TiB 34.66 GiB 37,213,806,592 1.02 2.30 ~1.44 TiB 2020-01-02 03:04:19 (auto) 100
2 ci_builds* 495.10 GiB 11.24 GiB 12,068,069,376 1.02 2.27 ~483.86 GiB 2020-01-03 20:53:14 (auto) 100
3 merge_request_diff_commits 577.89 GiB 6.70 GiB 7,189,929,984 1.01 1.16 ~571.19 GiB 2020-01-04 21:59:08 (auto) 100
4 notes* 157.74 GiB 2.22 GiB 2,376,155,136 1.01 1.40 ~155.53 GiB 2020-01-05 03:22:21 (auto) 100
5 project_mirror_data* 2.14 GiB 1.92 GiB 2,059,657,216 9.73 89.72 ~224.95 MiB 2020-01-05 13:37:48 (auto) 100
6 sent_notifications* 36.92 GiB 1.92 GiB 2,055,192,576 1.05 5.19 ~35.00 GiB 2020-01-04 23:35:42 (auto) 100
7 ci_job_artifacts 47.50 GiB 1.74 GiB 1,867,833,344 1.04 3.66 ~45.76 GiB 2020-01-05 11:28:41 (auto) 100
8 ci_pipelines* 20.79 GiB 1.72 GiB 1,844,232,192 1.09 8.26 ~19.07 GiB 2020-01-05 05:45:28 (auto) 100
9 audit_events* 51.95 GiB 1.46 GiB 1,565,663,232 1.03 2.81 ~50.50 GiB 2020-01-04 21:38:51 (auto) 100
10 merge_request_diffs* 12.89 GiB 922.01 MiB 966,795,264 1.08 6.99 ~11.99 GiB 2020-01-04 06:13:06 (auto) 100
11 merge_requests* 18.01 GiB 894.64 MiB 938,090,496 1.05 4.85 ~17.13 GiB 2020-01-04 14:32:50 (auto) 100
12 ci_stages 15.00 GiB 856.79 MiB 898,400,256 1.06 5.58 ~14.17 GiB 2020-01-05 04:10:47 (auto) 100
13 web_hook_logs 99.80 GiB 730.39 MiB 765,861,888 1.01 0.71 ~99.08 GiB 2020-01-05 02:02:50 (auto) 100
14 deployments* 6.94 GiB 651.95 MiB 683,614,208 1.10 9.18 ~6.30 GiB 2020-01-04 00:51:23 (auto) 100
15 ci_build_trace_sections 127.60 GiB 577.86 MiB 605,921,280 1.00 0.44 ~127.04 GiB 2020-01-04 21:55:42 (auto) 100
16 projects* 3.57 GiB 539.62 MiB 565,829,632 1.17 14.77 ~3.05 GiB 2020-01-05 13:31:20 (auto) 100
17 resource_label_events 13.46 GiB 524.76 MiB 550,248,448 1.04 3.81 ~12.95 GiB 2020-01-04 11:49:55 (auto) 100
18 users* 2.41 GiB 460.34 MiB 482,697,216 1.23 18.71 ~1.96 GiB 2020-01-05 12:09:39 (auto) 100
19 services* 857.46 MiB 416.83 MiB 437,075,968 1.95 48.61 ~440.63 MiB 2020-01-02 01:25:52 (auto) 100
20 todos* 3.56 GiB 312.93 MiB 328,122,368 1.09 8.60 ~3.25 GiB 2020-01-03 22:10:35 (auto) 100

* This table has specific autovacuum settings. See 'F001 Autovacuum: Current settings'

Conclusions

  • The estimated table (heap) bloat in this DB is low, just ~2.17% (~74.04 GiB). No action is needed now. Keep watching it though.

  • [P1] The following 2 tables have significant size (>1 MiB) and bloat estimate > 90.00%:

    • project_feature_usages: size 17.25 MiB, can be reduced 25.66 times, by ~16.58 MiB (~96.10%)
    • ci_build_trace_chunks: size 4.34 MiB, can be reduced 69.38 times, by ~4.28 MiB (~98.56%)

  • [P2] There are 10 tables with size > 1 MiB and table bloat estimate >= 40.00% and < 90.00%:

    • project_mirror_data: size 2.14 GiB, can be reduced 9.73 times, by ~1.92 GiB (~89.72%)
    • services: size 857.46 MiB, can be reduced 1.95 times, by ~416.83 MiB (~48.61%)
    • ci_runners: size 371.92 MiB, can be reduced 3.43 times, by ~263.57 MiB (~70.87%)
    • lists: size 465.29 MiB, can be reduced 1.75 times, by ~199.33 MiB (~42.84%)
    • issue_links: size 44.62 MiB, can be reduced 2.10 times, by ~23.40 MiB (~52.44%)
    • etc.

Recommendations

  • [P1] Reduce and prevent the high level of table bloat:

    • to prevent a high level of bloat in the future, tune autovacuum: consider more aggressive autovacuum settings (see F001);
    • eliminate or reduce the current table bloat using one of the approaches listed below.

  • The following tables have size > 1 MiB and table bloat estimate > 40.00%. Use this list to reduce the bloat applying one of the approaches described below. Here are these tables: project_mirror_data, services, ci_runners, lists, issue_links, project_feature_usages, ci_pipeline_schedules, import_export_uploads, ci_build_trace_chunks, clusters, lfs_file_locks, cluster_providers_gcp.

  • If you want to get exact bloat numbers, clone the database, get table sizes, then apply database-wide VACUUM FULL (it eliminate all the bloat), and get new table sizes. Then compare old and new numbers.

  • To reduce the table bloat, consider one of the following approaches:

  • Read more on this topic:

  Table of contents

F005 Autovacuum: Btree Index Bloat (Estimated)

⚠ This report is based on estimations. The errors in bloat estimates may be significant (in some cases, up to 15% and even more). Use it only as an indicator of potential issues.

Observations

Data collected: 2020-01-05 13:38:10 +0000 UTC
Current database: gitlabhq_production
ℹ This report considers only btree indexes.

Master (10.220.16.106)

The list is limited to 20 items. Total: 1043.

# Index (Table) Index Size Table Size ▼ Estimated bloat Est. bloat, bytes Est. bloat factor Est. bloat level, % Live Data Size Fillfactor
  ===== TOTAL ===== 1.15 TiB 2.47 TiB 162.98 GiB 174,995,193,856 1.16 13.92 0.99 TiB
1 index_merge_request_diff_commits_on_sha (merge_request_diff_commits) 102.67 GiB 563.73 GiB 16.49 GiB 17,703,165,952 1.19 16.06 ~86.18 GiB 90
2 index_ci_builds_on_token_encrypted (ci_builds*) 21.68 GiB 374.42 GiB 10.66 GiB 11,441,233,920 1.97 49.17 ~11.02 GiB 90
3 index_ci_build_trace_sections_on_section_name_id (ci_build_trace_sections) 45.24 GiB 127.65 GiB 7.94 GiB 8,514,854,912 1.21 17.53 ~37.31 GiB 90
4 index_ci_build_trace_sections_on_project_id (ci_build_trace_sections) 44.37 GiB 127.65 GiB 7.06 GiB 7,579,672,576 1.19 15.91 ~37.31 GiB 90
5 index_projects_on_repository_storage (projects*) 7.07 GiB 3.30 GiB 6.73 GiB 7,218,192,384 20.66 95.16 ~350.18 MiB 90
6 index_projects_on_last_activity_at (projects*) 6.14 GiB 3.30 GiB 5.90 GiB 6,325,813,248 25.07 96.01 ~250.67 MiB 90
7 index_projects_on_pending_delete (projects*) 5.91 GiB 3.30 GiB 5.66 GiB 6,072,238,080 24.10 95.85 ~250.68 MiB 90
8 index_projects_on_star_count (projects*) 5.66 GiB 3.30 GiB 5.41 GiB 5,807,046,656 23.09 95.67 ~250.68 MiB 90
9 index_events_on_target_type_and_target_id (events) 15.60 GiB 34.42 GiB 5.01 GiB 5,369,864,192 1.47 32.06 ~10.60 GiB 90
10 index_ci_builds_on_auto_canceled_by_id (ci_builds*) 9.55 GiB 374.42 GiB 4.88 GiB 5,233,254,400 2.04 51.09 ~4.67 GiB 90
11 index_projects_on_mirror_user_id (projects*) 5.09 GiB 3.30 GiB 4.85 GiB 5,204,418,560 21.18 95.28 ~245.98 MiB 90
12 index_ci_builds_on_token (ci_builds*) 11.52 GiB 374.42 GiB 3.91 GiB 4,192,124,928 1.51 33.92 ~7.61 GiB 90
13 index_ci_job_artifacts_on_expire_at_and_job_id (ci_job_artifacts) 10.42 GiB 47.51 GiB 3.72 GiB 3,992,363,008 1.56 35.69 ~6.70 GiB 90
14 index_notes_on_discussion_id (notes*) 15.95 GiB 148.53 GiB 3.54 GiB 3,798,048,768 1.29 22.19 ~12.41 GiB 90
15 index_notes_on_commit_id (notes*) 7.99 GiB 148.53 GiB 3.24 GiB 3,469,082,624 1.68 40.47 ~4.76 GiB 90
16 index_ci_job_artifacts_on_file_store (ci_job_artifacts) 9.18 GiB 47.51 GiB 2.57 GiB 2,753,257,472 1.39 27.96 ~6.61 GiB 90
17 index_sent_notifications_on_reply_key (sent_notifications*) 16.96 GiB 36.93 GiB 2.51 GiB 2,685,542,400 1.17 14.75 ~14.46 GiB 90
18 index_notes_on_review_id (notes*) 4.89 GiB 148.53 GiB 2.02 GiB 2,163,687,424 1.70 41.27 ~2.87 GiB 90
19 index_ci_builds_on_protected (ci_builds*) 9.53 GiB 374.42 GiB 1.82 GiB 1,944,068,096 1.23 19.02 ~7.72 GiB 90
20 index_ci_builds_on_runner_id (ci_builds*) 9.42 GiB 374.42 GiB 1.79 GiB 1,915,584,512 1.23 18.96 ~7.63 GiB 90

* This table has specific autovacuum settings. See 'F001 Autovacuum: Current settings'

Conclusions

  • The estimated index (btree only) bloat in this DB is low, just ~13.92% (~162.98 GiB). No action is needed now. Keep watching it though.

  • [P1] The following 28 indexes have significant size (>1 MiB) and bloat estimate > 90.00%:

    • index_projects_on_repository_storage: size 7.07 GiB, can be reduced 20.66 times, by ~6.76 GiB (~95.16%)
    • index_projects_on_last_activity_at: size 6.14 GiB, can be reduced 25.07 times, by ~5.92 GiB (~96.01%)
    • index_projects_on_pending_delete: size 5.91 GiB, can be reduced 24.10 times, by ~5.68 GiB (~95.85%)
    • index_projects_on_star_count: size 5.66 GiB, can be reduced 23.09 times, by ~5.44 GiB (~95.67%)
    • index_projects_on_mirror_user_id: size 5.09 GiB, can be reduced 21.18 times, by ~4.88 GiB (~95.28%)
    • etc.

  • [P2] There are 141 indexes with size > 1 MiB and index bloat estimate >= 40.00% and < 90.00%:

    • index_ci_builds_on_token_encrypted: size 21.68 GiB, can be reduced 1.97 times, by ~11.74 GiB (~49.17%)
    • index_ci_builds_on_auto_canceled_by_id: size 9.55 GiB, can be reduced 2.04 times, by ~5.34 GiB (~51.09%)
    • index_notes_on_commit_id: size 7.99 GiB, can be reduced 1.68 times, by ~3.71 GiB (~40.47%)
    • index_notes_on_review_id: size 4.89 GiB, can be reduced 1.70 times, by ~2.31 GiB (~41.27%)
    • index_merge_requests_on_title: size 3.40 GiB, can be reduced 1.69 times, by ~1.59 GiB (~40.93%)
    • etc.

Recommendations

  • [P1] Reduce and prevent a high level of index bloat:

    • to prevent a high level of bloat in the future, tune autovacuum: consider more aggressive autovacuum settings (see F001);
    • eliminate or reduce the current index bloat using one of the approaches listed below.

  • The following indexes have size > 1 MiB and index bloat estimate > 40.00%. Use this list to reduce the bloat applying one of the approaches described below. Here are these indexes: index_ci_builds_on_token_encrypted, index_projects_on_repository_storage, index_projects_on_last_activity_at, index_projects_on_pending_delete, index_projects_on_star_count, index_ci_builds_on_auto_canceled_by_id, index_projects_on_mirror_user_id, index_notes_on_commit_id, index_notes_on_review_id, index_merge_requests_on_title, index_ci_pipelines_on_auto_canceled_by_id, index_ci_pipelines_on_pipeline_schedule_id, index_merge_requests_on_milestone_id, index_projects_on_last_repository_check_failed, index_projects_on_mirror_last_successful_update_at, index_projects_on_runners_token, index_merge_request_metrics, index_projects_on_last_repository_updated_at, index_todos_on_commit_id, index_todos_on_group_id, index_merge_request_metrics_on_first_deployed_to_production_at, index_merge_request_metrics_on_latest_closed_by_id, index_resource_label_events_on_epic_id, index_resource_label_events_on_merge_request_id, index_merge_requests_on_state_and_merge_status, projects_pkey, index_personal_access_tokens_on_token_digest, index_users_on_managing_group_id, index_users_on_group_view, index_members_on_expires_at, index_members_on_requested_at, users_reset_password_token_key, index_gpg_signatures_on_gpg_key_subkey_id, personal_access_tokens_pkey, index_users_on_bot_type, index_users_on_static_object_token, index_users_on_state_and_internal, index_users_on_ghost, index_users_on_admin, index_users_on_state_and_internal_ee, index_users_on_state, index_protected_branch_push_access_levels_on_group_id, index_protected_branch_push_access_levels_on_user_id, index_protected_branch_merge_access_levels_on_group_id, index_protected_branch_merge_access_levels_on_user_id, index_ci_builds_project_id_and_status_for_live_jobs_partial2, idx_merge_requests_on_state_id_and_merge_status, index_uploads_on_checksum, index_users_on_accepted_term_id, index_ci_builds_on_artifacts_expire_at, index_ci_pipelines_on_merge_request_id, index_personal_access_tokens_on_user_id, index_merge_requests_on_updated_by_id, index_internal_ids_on_namespace_id, index_namespaces_on_file_template_project_id, index_projects_on_id_partial_for_visibility, index_keys_on_last_used_at, index_remote_mirrors_on_last_successful_update_at, index_keys_on_fingerprint_sha256, index_web_hooks_on_type, index_merge_requests_on_merge_user_id, ci_variables_pkey, index_index_statuses_on_project_id, index_timelogs_on_merge_request_id, index_boards_on_milestone_id, index_boards_on_group_id, releases_pkey, index_statuses_pkey, ci_runner_projects_pkey, web_hooks_pkey, index_releases_on_author_id, index_merge_request_metrics_on_latest_closed_at, index_milestones_on_group_id, index_milestones_on_title, index_ci_runner_projects_on_runner_id, namespace_aggregation_schedules_pkey, index_namespace_aggregation_schedules_on_namespace_id, namespace_root_storage_statistics_pkey, index_namespace_root_storage_statistics_on_namespace_id, namespace_statistics_pkey, project_group_links_pkey, deploy_keys_projects_pkey, index_merge_requests_on_id_and_merge_jid, project_import_data_pkey, index_deploy_keys_projects_on_project_id, idx_merge_requests_on_id_and_merge_jid, index_deploy_tokens_on_token, index_snippets_on_updated_at, index_web_hooks_on_project_id, index_snippets_on_project_id_and_visibility_level, ci_runner_namespaces_pkey, snippets_pkey, pages_domains_pkey, index_pages_domains_on_verified_at_and_enabled_until, index_container_expiration_policies_on_next_run_at_and_enabled, deploy_tokens_pkey, index_pages_domains_on_project_id_and_enabled_until, index_pages_domains_on_verified_at, index_pages_domains_on_domain, index_projects_on_mirror_and_mirror_trigger_builds_both_true, protected_branch_unprotect_access_levels_pkey, index_protected_branch_unprotect_access, index_issues_on_moved_to_id, index_ci_runner_namespaces_on_runner_id_and_namespace_id, index_protected_branch_unprotect_access_levels_on_group_id, index_users_on_public_email, index_snippets_on_author_id, index_pages_domains_on_remove_at, index_pages_domains_on_project_id, index_emails_on_confirmation_token, approval_project_rules_pkey, ci_group_variables_pkey, remote_mirrors_pkey, index_ci_pipelines_on_external_pull_request_id, index_milestone, index_remote_mirrors_on_project_id, index_clusters_kubernetes_namespaces_on_cluster_project_id, index_epics_on_closed_by_id, index_epics_on_assignee_id, index_label_priorities_on_priority, label_priorities_pkey, index_approval_project_rules_on_project_id, index_projects_on_pool_repository_id, index_epics_on_parent_id, index_badges_on_group_id, index_users_on_unconfirmed_email, index_pages_domains_need_auto_ssl_renewal, index_epics_on_end_date, index_epics_on_start_date, index_namespaces_on_trial_ends_on, clusters_kubernetes_namespaces_pkey, index_epics_on_group_id, index_draft_notes_on_discussion_id, index_clusters_kubernetes_namespaces_on_cluster_id, index_pages_domains_on_wildcard, index_pages_domains_on_domain_type, index_board_group_recent_visits_on_group_id.

  • And the following indexes also have size > 1 MiB and index bloat estimate > 40.00%. However, they belong to the highly bloated tables (see F004), so if you plan to process those tables you may not need to use this additional list. Here are these indexes: index_project_mirror_data_on_status, index_project_mirror_data_on_jid, index_ci_runners_on_contacted_at, index_ci_runners_on_is_shared, index_ci_runners_on_locked, index_ci_runners_on_runner_type, index_ci_runners_on_token, index_project_mirror_data_on_last_update_at_and_retry_count, index_project_mirror_data_on_last_successful_update_at, ci_runners_pkey, index_lists_on_user_id, index_lists_on_milestone_id, index_ci_pipeline_schedules_on_next_run_at_and_active, ci_build_trace_chunks_pkey, issue_links_pkey, index_issue_links_on_target_id, index_issue_links_on_source_id, idx_proj_feat_usg_on_jira_dvcs_server_last_sync_at_and_proj_id, index_ci_build_trace_chunks_on_build_id_and_chunk_index, index_project_feature_usages_on_project_id, project_feature_usages_pkey, index_issue_links_on_source_id_and_target_id, index_import_export_uploads_on_project_id, import_export_uploads_pkey, index_import_export_uploads_on_updated_at, idx_proj_feat_usg_on_jira_dvcs_cloud_last_sync_at_and_proj_id, ci_pipeline_schedules_pkey, index_ci_pipeline_schedules_on_project_id, index_ci_pipeline_schedules_on_owner_id, index_clusters_on_enabled, clusters_pkey, index_clusters_on_user_id.

  • If you want to get exact bloat numbers, clone the database, get index sizes, then apply database-wide VACUUM FULL (it eliminates all the bloat), and get new table sizes. Then compare old and new numbers.

  • To reduce the index bloat, consider one of the following approaches:

    • VACUUM FULL (⚠ requires downtime / maintenance window),
    • REINDEX (REINDEX INDEX, REINDEX TABLE; ⚠ requires downtime / maintenance window),
    • recreating indexes online using CREATE INDEX CONCURRENTLY, DROP INDEX CONCURRENTLY and renaming (not trivial for indexes supporting PK, FK) // REINDEX CONCURRENTLY is available in Postgres 12+,
    • one of the tools reducing the bloat online, without interrupting the operations:

  • Read more on this topic:

  Table of contents

F008 Autovacuum: Resource Usage

Observations

Data collected: 2020-01-05 13:38:21 +0000 UTC

Settings

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_vacuum_cost_delay 5 ms
autovacuum_vacuum_cost_limit 6000
autovacuum_work_mem -1 kB
log_autovacuum_min_duration 0 ms
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
vacuum_cost_limit 200
work_mem 102400 kB 100.00 MiB

CPU

Cpu count you can see in report A001

RAM

Ram amount you can see in report A001

Max workers memory: 12 GiB

DISK

⚠ Warning: collection of current impact on disks is not yet implemented. Please refer to Postgres logs and see current read and write IO bandwidth caused by autovacuum.

Conclusions

Recommendations

  Table of contents

G001 Memory-related Settings

Observations

Data collected: 2020-01-05 13:39:52 +0000 UTC

Master (10.220.16.106)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica servers:

Replica (10.220.16.101)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.102)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.103)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.104)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.105)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.107)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.108)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Replica (10.220.16.109)

Setting name Value Unit Pretty value
autovacuum_max_workers 6
autovacuum_work_mem -1 kB
effective_cache_size 43352064 8kB 330.75 GiB
maintenance_work_mem 2097152 kB 2.00 GiB
max_connections 300
shared_buffers 14450688 8kB 110.25 GiB
temp_buffers 1024 8kB 8.00 MiB
work_mem 102400 kB 100.00 MiB

Conclusions

Recommendations

  Table of contents

G002 Connections and Current Activity

Observations

Data collected: 2020-01-05 13:39:53 +0000 UTC

Master (10.220.16.106)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 153 2 0 0 0
2 gitlab ALL databases active 2 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 138 2 0 0 0
5 pgbouncer ALL databases idle 4 0 0 0 0
6 gitlab-superuser ALL databases idle 1 0 0 0 0
7 postgres_exporter ALL databases idle 1 0 0 0 0
8 gitlab-monitor ALL databases idle 1 0 0 0 0
9 gitlab ALL databases idle in transaction 5 0 0 0 0
10 gitlab gitlabhq_production active 2 0 0 0 0
11 gitlab-checkup gitlabhq_production active 1 0 0 0 0
12 gitlab gitlabhq_production idle 138 2 0 0 0
13 pgbouncer gitlabhq_production idle 4 0 0 0 0
14 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
15 postgres_exporter gitlabhq_production idle 1 0 0 0 0
16 gitlab gitlabhq_production idle in transaction 5 0 0 0 0
17 gitlab-superuser postgres idle 1 0 0 0 0

Replica servers:

Replica (10.220.16.101)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 32 0 0 0 0
2 gitlab ALL databases active 2 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 23 0 0 0 0
5 pgbouncer ALL databases idle 3 0 0 0 0
6 gitlab-monitor ALL databases idle 1 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab gitlabhq_production active 2 0 0 0 0
10 gitlab-checkup gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 23 0 0 0 0
12 pgbouncer gitlabhq_production idle 3 0 0 0 0
13 postgres_exporter gitlabhq_production idle 1 0 0 0 0
14 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.102)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 31 0 0 0 0
2 gitlab-checkup ALL databases active 1 0 0 0 0
3 gitlab ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 23 0 0 0 0
5 pgbouncer ALL databases idle 3 0 0 0 0
6 gitlab-monitor ALL databases idle 1 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab-checkup gitlabhq_production active 1 0 0 0 0
10 gitlab gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 23 0 0 0 0
12 pgbouncer gitlabhq_production idle 3 0 0 0 0
13 postgres_exporter gitlabhq_production idle 1 0 0 0 0
14 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.103)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 33 0 0 0 0
2 gitlab-checkup ALL databases active 1 0 0 0 0
3 gitlab ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 25 0 0 0 0
5 pgbouncer ALL databases idle 3 0 0 0 0
6 gitlab-monitor ALL databases idle 1 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab-checkup gitlabhq_production active 1 0 0 0 0
10 gitlab gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 25 0 0 0 0
12 pgbouncer gitlabhq_production idle 3 0 0 0 0
13 postgres_exporter gitlabhq_production idle 1 0 0 0 0
14 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.104)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 31 0 0 0 0
2 gitlab-checkup ALL databases active 1 0 0 0 0
3 gitlab ALL databases idle 24 0 0 0 0
4 pgbouncer ALL databases idle 3 0 0 0 0
5 gitlab-monitor ALL databases idle 1 0 0 0 0
6 postgres_exporter ALL databases idle 1 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 gitlab-checkup gitlabhq_production active 1 0 0 0 0
9 gitlab gitlabhq_production idle 24 0 0 0 0
10 pgbouncer gitlabhq_production idle 3 0 0 0 0
11 postgres_exporter gitlabhq_production idle 1 0 0 0 0
12 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
13 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.105)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 32 0 0 0 0
2 gitlab-checkup ALL databases active 1 0 0 0 0
3 gitlab ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 24 0 0 0 0
5 pgbouncer ALL databases idle 3 0 0 0 0
6 gitlab-monitor ALL databases idle 1 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab-checkup gitlabhq_production active 1 0 0 0 0
10 gitlab gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 24 0 0 0 0
12 pgbouncer gitlabhq_production idle 3 0 0 0 0
13 postgres_exporter gitlabhq_production idle 1 0 0 0 0
14 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.107)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 33 0 0 0 0
2 gitlab ALL databases active 2 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 23 0 0 0 0
5 pgbouncer ALL databases idle 3 0 0 0 0
6 gitlab-monitor ALL databases idle 2 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab gitlabhq_production active 2 0 0 0 0
10 gitlab-checkup gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 23 0 0 0 0
12 pgbouncer gitlabhq_production idle 3 0 0 0 0
13 gitlab-monitor gitlabhq_production idle 2 0 0 0 0
14 postgres_exporter gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.108)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 31 0 0 0 0
2 gitlab ALL databases active 2 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 22 0 0 0 0
5 pgbouncer ALL databases idle 3 0 0 0 0
6 gitlab-monitor ALL databases idle 1 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab gitlabhq_production active 2 0 0 0 0
10 gitlab-checkup gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 22 0 0 0 0
12 pgbouncer gitlabhq_production idle 3 0 0 0 0
13 postgres_exporter gitlabhq_production idle 1 0 0 0 0
14 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Replica (10.220.16.109)

# User DB Current state Count State changed >1m ago State changed >1h ago Tx age >1m Tx age >1h
1 ALL users ALL databases ALL states 32 0 0 0 0
2 gitlab ALL databases active 3 0 0 0 0
3 gitlab-checkup ALL databases active 1 0 0 0 0
4 gitlab ALL databases idle 22 0 0 0 0
5 pgbouncer ALL databases idle 3 0 0 0 0
6 gitlab-monitor ALL databases idle 1 0 0 0 0
7 gitlab-superuser ALL databases idle 1 0 0 0 0
8 postgres_exporter ALL databases idle 1 0 0 0 0
9 gitlab gitlabhq_production active 3 0 0 0 0
10 gitlab-checkup gitlabhq_production active 1 0 0 0 0
11 gitlab gitlabhq_production idle 22 0 0 0 0
12 pgbouncer gitlabhq_production idle 3 0 0 0 0
13 postgres_exporter gitlabhq_production idle 1 0 0 0 0
14 gitlab-monitor gitlabhq_production idle 1 0 0 0 0
15 gitlab-superuser postgres idle 1 0 0 0 0

Conclusions

Recommendations

All good, no recommendations here.

  Table of contents

G003 Timeouts, Locks, Deadlocks

Observations

Data collected: 2020-01-05 13:39:53 +0000 UTC

Master (10.220.16.106)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

# Database Conflicts ▼ Deadlocks Stats reset at Stat reset
1 gitlabhq_production 4 871 2019-10-28T16:07:34.673503+00:00 68 days 21:30:49
2 postgres 0 0 2019-10-28T16:07:29.861973+00:00 68 days 21:30:53

Replica servers:

Replica (10.220.16.101)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:12.080501+00:00 24 days 06:03:36
gitlabhq_production 2 0 2019-12-12T07:32:12.114073+00:00 24 days 06:03:36

Replica (10.220.16.102)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:11.427924+00:00 24 days 06:04:05
gitlabhq_production 0 0 2019-12-12T07:32:11.45222+00:00 24 days 06:04:05

Replica (10.220.16.103)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:12.581377+00:00 24 days 06:04:32
gitlabhq_production 0 0 2019-12-12T07:32:12.982689+00:00 24 days 06:04:31

Replica (10.220.16.104)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:11.156846+00:00 24 days 06:05:02
gitlabhq_production 0 0 2019-12-12T07:32:12.131981+00:00 24 days 06:05:01

Replica (10.220.16.105)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:17.615779+00:00 24 days 06:05:23
gitlabhq_production 0 0 2019-12-12T07:32:20.485667+00:00 24 days 06:05:20

Replica (10.220.16.107)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:10.361007+00:00 24 days 06:06:44
gitlabhq_production 0 0 2019-12-12T07:32:11.376209+00:00 24 days 06:06:43

Replica (10.220.16.108)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:20.760583+00:00 24 days 06:07:04
gitlabhq_production 0 0 2019-12-12T07:32:20.698715+00:00 24 days 06:07:04

Replica (10.220.16.109)

Timeouts

Setting name Value Unit Pretty value
authentication_timeout 60 s
idle_in_transaction_session_timeout 30000 ms
statement_timeout 15000 ms

Locks

Setting name Value Unit Pretty value
deadlock_timeout 5000 ms
lock_timeout 0 ms
max_locks_per_transaction 128
max_pred_locks_per_transaction 64

Databases data

Database Conflicts ▼ Deadlocks Stats reset at Stat reset
postgres 0 0 2019-12-12T07:32:21.689386+00:00 24 days 06:07:32
gitlabhq_production 0 0 2019-12-12T07:32:24.58281+00:00 24 days 06:07:29

Conclusions

Recommendations

  Table of contents

H001 Invalid Indexes

Observations

Data collected: 2020-01-05 13:38:23 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

Invalid indexes not found

Conclusions

Recommendations

  Table of contents

H002 Unused Indexes

Observations

Data collected: 2020-01-05 13:39:54 +0000 UTC
Current database: gitlabhq_production

Stats reset: 2 mons 7 days 21:31:00 ago (2019-10-28 16:07:35 +0000 UTC)

Never Used Indexes

The list is limited to 20 items. Total: 68.

# Table Index 10.220.16.106 usage 10.220.16.101 usage 10.220.16.102 usage 10.220.16.103 usage 10.220.16.104 usage 10.220.16.105 usage 10.220.16.107 usage 10.220.16.108 usage 10.220.16.109 usage ▼ Index size Table size Supports FK
  =====TOTAL===== 43.51 GiB 1.74 TiB
1 ci_builds index_ci_builds_on_protected 0 0 0 0 0 0 0 0 0 9.54 GiB 374.32 GiB
2 ci_builds index_ci_builds_on_queued_at 0 0 0 0 0 0 0 9.38 GiB 374.32 GiB
3 ci_job_artifacts index_ci_job_artifacts_on_file_store 0 0 0 0 0 0 0 0 0 9.18 GiB 47.50 GiB
4 audit_events analytics_index_audit_events_on_created_at_and_author_id 0 0 6.54 GiB 51.95 GiB
5 merge_request_diffs index_merge_request_diffs_on_merge_request_id_and_id_partial 0 0 0 0 0 0 0 0 0 1.69 GiB 12.89 GiB Yes
6 projects index_projects_on_runners_token 0 0 0 0 0 0 0 0 0 655.43 MiB 3.30 GiB Yes
7 projects index_projects_on_last_repository_check_failed 0 0 0 0 0 0 0 0 0 652.18 MiB 3.30 GiB Yes
8 issues index_issues_on_state 0 0 0 0 0 0 0 0 0 562.47 MiB 14.53 GiB Yes
9 projects idx_project_repository_check_partial 0 0 0 0 0 0 0 0 0 530.77 MiB 3.30 GiB
10 members index_members_on_access_level 0 416.06 MiB 1.58 GiB
11 projects index_on_projects_lower_path 0 0 0 0 0 0 0 0 0 406.08 MiB 3.30 GiB Yes
12 merge_requests index_merge_requests_on_state_and_merge_status 0 0 0 0 0 0 0 0 0 350.52 MiB 16.62 GiB
13 merge_requests index_merge_requests_on_source_project_and_branch_state_opened 0 0 0 0 0 0 0 0 0 328.05 MiB 16.62 GiB
14 project_repositories index_project_repositories_on_shard_id 0 0 0 0 0 0 0 0 0 301.92 MiB 1.39 GiB
15 project_repository_states idx_repository_states_on_last_wiki_verification_ran_at 0 0 0 0 0 0 0 0 0 243.43 MiB 836.51 MiB
16 project_repository_states idx_repository_states_on_last_repository_verification_ran_at 0 0 0 0 0 0 0 0 0 227.13 MiB 836.51 MiB
17 users index_users_on_state_and_internal 0 0 0 0 0 0 0 0 0 208.16 MiB 2.41 GiB Yes
18 users index_users_on_accepted_term_id 0 0 0 0 0 0 0 0 0 201.68 MiB 2.41 GiB Yes
19 merge_requests index_merge_requests_on_target_project_id_and_iid_opened 0 0 0 0 0 0 0 0 165.99 MiB 16.62 GiB Yes
20 ci_runners index_ci_runners_on_contacted_at 0 0 0 0 0 0 0 141.97 MiB 371.92 MiB

Rarely Used Indexes

The list is limited to 20 items. Total: 256.

# Table Index 10.220.16.106 usage 10.220.16.101 usage 10.220.16.102 usage 10.220.16.103 usage 10.220.16.104 usage 10.220.16.105 usage 10.220.16.107 usage 10.220.16.108 usage 10.220.16.109 usage ▼ Index size Table size Comment Supports FK
  =====TOTAL===== 653.28 GiB 9.43 TiB
1 merge_request_diff_commits index_merge_request_diff_commits_on_sha scans: 52,763,595/hour, writes: 282,457,584/hour scans: 91,371/hour, writes: 0/hour scans: 91,691/hour, writes: 0/hour scans: 91,419/hour, writes: 0/hour scans: 91,772/hour, writes: 0/hour scans: 91,699/hour, writes: 0/hour scans: 91,460/hour, writes: 0/hour scans: 91,285/hour, writes: 0/hour scans: 91,605/hour, writes: 0/hour 102.69 GiB 563.57 GiB Low Scans, High Writes
2 ci_build_trace_sections index_ci_build_trace_sections_on_section_name_id scans: 225,078/hour, writes: 366,714,552/hour 45.24 GiB 127.62 GiB Low Scans, High Writes
3 ci_build_trace_sections index_ci_build_trace_sections_on_project_id scans: 282,947/hour, writes: 366,714,552/hour 44.37 GiB 127.62 GiB Low Scans, High Writes
4 events index_events_on_project_id_and_created_at scans: 39,813/hour, writes: 69,115,883/hour scans: 198,060/hour, writes: 0/hour scans: 215,488/hour, writes: 0/hour scans: 191,811/hour, writes: 0/hour scans: 195,187/hour, writes: 0/hour scans: 206,530/hour, writes: 0/hour scans: 219,632/hour, writes: 0/hour scans: 177,550/hour, writes: 0/hour scans: 205,867/hour, writes: 0/hour 20.40 GiB 34.41 GiB Low Scans, High Writes
5 ci_builds index_ci_builds_on_status_and_type_and_runner_id scans: 151,301,441/hour, writes: 226,991,807/hour scans: 6,393,049/hour, writes: 0/hour scans: 6,381,828/hour, writes: 0/hour scans: 6,393,132/hour, writes: 0/hour scans: 6,388,075/hour, writes: 0/hour scans: 6,404,459/hour, writes: 0/hour scans: 6,391,826/hour, writes: 0/hour scans: 6,396,206/hour, writes: 0/hour scans: 6,384,010/hour, writes: 0/hour 19.89 GiB 374.32 GiB Low Scans, High Writes
6 ci_builds ci_builds_gitlab_monitor_metrics scans: 13,804/hour, writes: 226,991,807/hour scans: 3,028/hour, writes: 0/hour scans: 3,075/hour, writes: 0/hour scans: 3,019/hour, writes: 0/hour scans: 3,048/hour, writes: 0/hour scans: 3,018/hour, writes: 0/hour scans: 3,101/hour, writes: 0/hour scans: 3,097/hour, writes: 0/hour scans: 3,045/hour, writes: 0/hour 16.53 GiB 374.32 GiB Low Scans, High Writes
7 notes index_notes_on_discussion_id scans: 23,074,070/hour, writes: 47,303,792/hour scans: 74,737/hour, writes: 0/hour scans: 75,447/hour, writes: 0/hour scans: 75,023/hour, writes: 0/hour scans: 75,930/hour, writes: 0/hour scans: 75,189/hour, writes: 0/hour scans: 75,015/hour, writes: 0/hour scans: 75,989/hour, writes: 0/hour scans: 75,662/hour, writes: 0/hour 15.95 GiB 148.49 GiB Low Scans, High Writes Yes
8 events analytics_index_events_on_created_at_and_author_id scans: 254,110/hour, writes: 69,115,883/hour scans: 362,430/hour, writes: 0/hour scans: 362,633/hour, writes: 0/hour scans: 363,298/hour, writes: 0/hour scans: 363,113/hour, writes: 0/hour scans: 364,151/hour, writes: 0/hour scans: 362,680/hour, writes: 0/hour scans: 363,104/hour, writes: 0/hour scans: 361,737/hour, writes: 0/hour 15.52 GiB 34.41 GiB Low Scans, High Writes
9 events index_events_on_author_id_and_project_id scans: 12,676,412/hour, writes: 69,115,883/hour 13.76 GiB 34.41 GiB Low Scans, High Writes
10 events index_events_on_project_id_and_id scans: 13,293,982/hour, writes: 69,115,883/hour 13.21 GiB 34.41 GiB Low Scans, High Writes
11 events index_events_on_action scans: 22/hour, writes: 69,115,883/hour scans: 750/hour, writes: 0/hour scans: 676/hour, writes: 0/hour scans: 778/hour, writes: 0/hour scans: 798/hour, writes: 0/hour scans: 744/hour, writes: 0/hour scans: 696/hour, writes: 0/hour scans: 722/hour, writes: 0/hour scans: 702/hour, writes: 0/hour 12.57 GiB 34.41 GiB Low Scans, High Writes
12 ci_builds index_ci_builds_on_project_id_and_name_and_ref scans: 98/hour, writes: 226,991,807/hour 11.47 GiB 374.32 GiB Low Scans, High Writes
13 ci_job_artifacts index_ci_job_artifacts_on_expire_at_and_job_id scans: 168,716/hour, writes: 155,429,517/hour 10.42 GiB 47.50 GiB Low Scans, High Writes
14 ci_builds index_ci_builds_on_auto_canceled_by_id scans: 885,230/hour, writes: 226,991,807/hour 9.56 GiB 374.32 GiB Low Scans, High Writes Yes
15 ci_builds tmp_build_stage_position_index scans: 1,023/hour, writes: 226,991,807/hour 9.50 GiB 374.32 GiB Low Scans, High Writes
16 ci_builds index_ci_builds_on_runner_id scans: 36,643,824/hour, writes: 226,991,807/hour scans: 1,977,412/hour, writes: 0/hour scans: 1,974,627/hour, writes: 0/hour scans: 1,977,836/hour, writes: 0/hour scans: 1,976,160/hour, writes: 0/hour scans: 1,979,810/hour, writes: 0/hour scans: 1,976,491/hour, writes: 0/hour scans: 1,975,568/hour, writes: 0/hour scans: 1,975,889/hour, writes: 0/hour 9.43 GiB 374.32 GiB Low Scans, High Writes Yes
17 ci_builds index_ci_builds_on_updated_at scans: 10,654/hour, writes: 226,991,807/hour 8.85 GiB 374.32 GiB Low Scans, High Writes
18 ci_builds index_ci_builds_on_user_id scans: 13,718/hour, writes: 226,991,807/hour 8.68 GiB 374.32 GiB Low Scans, High Writes Yes
19 ci_builds_metadata index_ci_builds_metadata_on_project_id scans: 282,945/hour, writes: 100,998,819/hour 8.09 GiB 15.42 GiB Low Scans, High Writes
20 ci_job_artifacts index_ci_job_artifacts_on_project_id scans: 286,454/hour, writes: 155,429,517/hour scans: 332/hour, writes: 0/hour scans: 290/hour, writes: 0/hour scans: 310/hour, writes: 0/hour scans: 381/hour, writes: 0/hour scans: 375/hour, writes: 0/hour scans: 313/hour, writes: 0/hour scans: 294/hour, writes: 0/hour scans: 374/hour, writes: 0/hour 7.15 GiB 47.50 GiB Low Scans, High Writes

Conclusions

  • [P3] 68 unused index(es) have been found.

Recommendations

  • [P3] Use the database migration provided below to drop the unused indexes. Keep in mind, that under load, it is recommended to use DROP INDEX CONCURRENTLY (and CREATE INDEX CONCURRENTLY if reverting is needed) to avoid blocking issues.
  • Be careful dropping the indexes. If you have multiple setups of your software, the analysis of just a single setup might be not enough. Some indexes might be used (and therefore, needed) only on a limited number of setups. Also, in some cases, developers prepare indexes for new features in advance – in such cases, dropping those indexes is not a good idea.
  • If there are some doubts, consider a more careful approach. Before actualdropping, indexes disable listed in this report. For this, use queries like `UPDATE pg_index SET indisvalid = false WHERE indexrelid::regclass = (select oid from pg_class where relname = 'u_users_email');. Indexes will continue to get updates. In case of some performance degradations, re-enable the corresponding indexes, setting indisvalid to true. If everything looks fine, after a significant period of observations, proceed with DROP INDEX CONCURRENTLY.
  • "DO" database migrations
DROP INDEX CONCURRENTLY analytics_index_audit_events_on_created_at_and_author_id; -- 6687 MB, Never Used Indexes, table audit_events  
DROP INDEX CONCURRENTLY index_ci_builds_on_artifacts_expire_at; -- 82 MB, Never Used Indexes, table ci_builds  
DROP INDEX CONCURRENTLY index_ci_builds_on_project_id_for_successfull_pages_deploy; -- 87 MB, Never Used Indexes, table ci_builds  
DROP INDEX CONCURRENTLY index_ci_builds_on_protected; -- 9761 MB, Never Used Indexes, table ci_builds  
DROP INDEX CONCURRENTLY index_ci_builds_on_queued_at; -- 9597 MB, Never Used Indexes, table ci_builds  
DROP INDEX CONCURRENTLY index_ci_builds_metadata_on_build_id_and_has_exposed_artifacts; -- 480 kB, Never Used Indexes, table ci_builds_metadata  
DROP INDEX CONCURRENTLY index_ci_job_artifacts_on_file_store; -- 9392 MB, Never Used Indexes, table ci_job_artifacts  
DROP INDEX CONCURRENTLY index_ci_pipeline_schedules_on_next_run_at_and_active; -- 28 MB, Never Used Indexes, table ci_pipeline_schedules  
DROP INDEX CONCURRENTLY index_ci_runners_on_contacted_at; -- 142 MB, Never Used Indexes, table ci_runners  
DROP INDEX CONCURRENTLY index_ci_runners_on_is_shared; -- 136 MB, Never Used Indexes, table ci_runners  
DROP INDEX CONCURRENTLY index_ci_runners_on_locked; -- 132 MB, Never Used Indexes, table ci_runners  
DROP INDEX CONCURRENTLY index_ci_runners_on_token; -- 121 MB, Never Used Indexes, table ci_runners  
DROP INDEX CONCURRENTLY index_cluster_providers_gcp_on_cloud_run; -- 296 kB, Never Used Indexes, table cluster_providers_gcp  
DROP INDEX CONCURRENTLY index_container_expiration_policies_on_next_run_at_and_enabled; -- 9424 kB, Never Used Indexes, table container_expiration_policies  
DROP INDEX CONCURRENTLY index_deploy_tokens_on_token_and_expires_at_and_id; -- 12 MB, Never Used Indexes, table deploy_tokens  
DROP INDEX CONCURRENTLY partial_index_deployments_for_legacy_successful_deployments; -- 16 kB, Never Used Indexes, table deployments  
DROP INDEX CONCURRENTLY index_design_management_designs_versions_on_event; -- 128 kB, Never Used Indexes, table design_management_designs_versions  
DROP INDEX CONCURRENTLY index_gitlab_subscription_histories_on_gitlab_subscription_id; -- 400 kB, Never Used Indexes, table gitlab_subscription_histories  
DROP INDEX CONCURRENTLY index_grafana_integrations_on_enabled; -- 16 kB, Never Used Indexes, table grafana_integrations  
DROP INDEX CONCURRENTLY index_import_failures_on_correlation_id_value; -- 16 kB, Never Used Indexes, table import_failures  
DROP INDEX CONCURRENTLY index_import_failures_on_project_id; -- 16 kB, Never Used Indexes, table import_failures  
DROP INDEX CONCURRENTLY index_issues_on_state; -- 562 MB, Never Used Indexes, table issues  
DROP INDEX CONCURRENTLY idx_jira_connect_subscriptions_on_installation_id; -- 16 kB, Never Used Indexes, table jira_connect_subscriptions  
DROP INDEX CONCURRENTLY index_keys_on_last_used_at; -- 112 MB, Never Used Indexes, table keys  
DROP INDEX CONCURRENTLY index_label_priorities_on_priority; -- 3624 kB, Never Used Indexes, table label_priorities  
DROP INDEX CONCURRENTLY index_labels_on_group_id_and_title; -- 8192 bytes, Never Used Indexes, table labels  
DROP INDEX CONCURRENTLY index_labels_on_project_id_and_title; -- 8192 bytes, Never Used Indexes, table labels  
DROP INDEX CONCURRENTLY index_members_on_access_level; -- 416 MB, Never Used Indexes, table members  
DROP INDEX CONCURRENTLY index_merge_request_diffs_on_merge_request_id_and_id_partial; -- 1730 MB, Never Used Indexes, table merge_request_diffs  
DROP INDEX CONCURRENTLY index_merge_request_metrics_on_latest_closed_at; -- 44 MB, Never Used Indexes, table merge_request_metrics  
DROP INDEX CONCURRENTLY idx_merge_requests_on_state_id_and_merge_status; -- 105 MB, Never Used Indexes, table merge_requests  
DROP INDEX CONCURRENTLY index_merge_requests_on_id_and_merge_jid; -- 11 MB, Never Used Indexes, table merge_requests  
DROP INDEX CONCURRENTLY index_merge_requests_on_source_project_and_branch_state_opened; -- 328 MB, Never Used Indexes, table merge_requests  
DROP INDEX CONCURRENTLY index_merge_requests_on_state_and_merge_status; -- 351 MB, Never Used Indexes, table merge_requests  
DROP INDEX CONCURRENTLY index_merge_requests_on_target_project_id_and_iid_opened; -- 166 MB, Never Used Indexes, table merge_requests  
DROP INDEX CONCURRENTLY index_namespaces_on_created_at; -- 134 MB, Never Used Indexes, table namespaces  
DROP INDEX CONCURRENTLY index_namespaces_on_ldap_sync_last_successful_update_at; -- 135 MB, Never Used Indexes, table namespaces  
DROP INDEX CONCURRENTLY index_namespaces_on_ldap_sync_last_update_at; -- 135 MB, Never Used Indexes, table namespaces  
DROP INDEX CONCURRENTLY index_namespaces_on_plan_id; -- 135 MB, Never Used Indexes, table namespaces  
DROP INDEX CONCURRENTLY index_namespaces_on_require_two_factor_authentication; -- 135 MB, Never Used Indexes, table namespaces  
DROP INDEX CONCURRENTLY index_packages_dependency_links_on_dependency_id; -- 968 kB, Never Used Indexes, table packages_dependency_links  
DROP INDEX CONCURRENTLY index_pages_domains_need_auto_ssl_renewal; -- 1160 kB, Never Used Indexes, table pages_domains  
DROP INDEX CONCURRENTLY index_pages_domains_on_verified_at_and_enabled_until; -- 5952 kB, Never Used Indexes, table pages_domains  
DROP INDEX CONCURRENTLY index_pages_domains_on_wildcard; -- 1072 kB, Never Used Indexes, table pages_domains  
DROP INDEX CONCURRENTLY index_project_custom_attributes_on_key_and_value; -- 16 kB, Never Used Indexes, table project_custom_attributes  
DROP INDEX CONCURRENTLY index_project_mirror_data_on_last_successful_update_at; -- 90 MB, Never Used Indexes, table project_mirror_data  
DROP INDEX CONCURRENTLY index_project_repositories_on_shard_id; -- 302 MB, Never Used Indexes, table project_repositories  
DROP INDEX CONCURRENTLY idx_repository_states_on_last_repository_verification_ran_at; -- 227 MB, Never Used Indexes, table project_repository_states  
DROP INDEX CONCURRENTLY idx_repository_states_on_last_wiki_verification_ran_at; -- 243 MB, Never Used Indexes, table project_repository_states  
DROP INDEX CONCURRENTLY idx_repository_states_on_repository_failure_partial; -- 16 kB, Never Used Indexes, table project_repository_states  
DROP INDEX CONCURRENTLY idx_repository_states_on_wiki_failure_partial; -- 8192 bytes, Never Used Indexes, table project_repository_states  
DROP INDEX CONCURRENTLY idx_repository_states_outdated_checksums; -- 23 MB, Never Used Indexes, table project_repository_states  
DROP INDEX CONCURRENTLY idx_project_repository_check_partial; -- 531 MB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_on_projects_lower_path; -- 406 MB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_projects_on_last_repository_check_at; -- 16 kB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_projects_on_last_repository_check_failed; -- 652 MB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_projects_on_mirror_and_mirror_trigger_builds_both_true; -- 5304 kB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_projects_on_runners_token; -- 655 MB, Never Used Indexes, table projects  
DROP INDEX CONCURRENTLY index_prometheus_metrics_on_group; -- 40 kB, Never Used Indexes, table prometheus_metrics  
DROP INDEX CONCURRENTLY index_remote_mirrors_on_last_successful_update_at; -- 50 MB, Never Used Indexes, table remote_mirrors  
DROP INDEX CONCURRENTLY index_snippets_on_updated_at; -- 11 MB, Never Used Indexes, table snippets  
DROP INDEX CONCURRENTLY index_term_agreements_on_term_id; -- 103 MB, Never Used Indexes, table term_agreements  
DROP INDEX CONCURRENTLY index_term_agreements_on_user_id; -- 87 MB, Never Used Indexes, table term_agreements  
DROP INDEX CONCURRENTLY index_timelogs_on_spent_at; -- 19 MB, Never Used Indexes, table timelogs  
DROP INDEX CONCURRENTLY index_timelogs_on_user_id; -- 38 MB, Never Used Indexes, table timelogs  
DROP INDEX CONCURRENTLY index_users_on_accepted_term_id; -- 202 MB, Never Used Indexes, table users  
DROP INDEX CONCURRENTLY index_users_on_state_and_internal; -- 208 MB, Never Used Indexes, table users  
DROP INDEX CONCURRENTLY index_zoom_meetings_on_issue_status; -- 16 kB, Never Used Indexes, table zoom_meetings
  • "UNDO" database migrations
CREATE INDEX CONCURRENTLY analytics_index_audit_events_on_created_at_and_author_id ON public.audit_events USING btree (created_at, author_id); -- table audit_events  
CREATE INDEX CONCURRENTLY index_ci_builds_on_artifacts_expire_at ON public.ci_builds USING btree (artifacts_expire_at) WHERE (artifacts_file <> ''::text); -- table ci_builds  
CREATE INDEX CONCURRENTLY index_ci_builds_on_project_id_for_successfull_pages_deploy ON public.ci_builds USING btree (project_id) WHERE (((type)::text = 'GenericCommitStatus'::text) AND ((stage)::text = 'deploy'::text) AND ((name)::text = 'pages:deploy'::text) AND ((status)::text = 'success'::text)); -- table ci_builds  
CREATE INDEX CONCURRENTLY index_ci_builds_on_protected ON public.ci_builds USING btree (protected); -- table ci_builds  
CREATE INDEX CONCURRENTLY index_ci_builds_on_queued_at ON public.ci_builds USING btree (queued_at); -- table ci_builds  
CREATE INDEX CONCURRENTLY index_ci_builds_metadata_on_build_id_and_has_exposed_artifacts ON public.ci_builds_metadata USING btree (build_id) WHERE (has_exposed_artifacts IS TRUE); -- table ci_builds_metadata  
CREATE INDEX CONCURRENTLY index_ci_job_artifacts_on_file_store ON public.ci_job_artifacts USING btree (file_store); -- table ci_job_artifacts  
CREATE INDEX CONCURRENTLY index_ci_pipeline_schedules_on_next_run_at_and_active ON public.ci_pipeline_schedules USING btree (next_run_at, active); -- table ci_pipeline_schedules  
CREATE INDEX CONCURRENTLY index_ci_runners_on_contacted_at ON public.ci_runners USING btree (contacted_at); -- table ci_runners  
CREATE INDEX CONCURRENTLY index_ci_runners_on_is_shared ON public.ci_runners USING btree (is_shared); -- table ci_runners  
CREATE INDEX CONCURRENTLY index_ci_runners_on_locked ON public.ci_runners USING btree (locked); -- table ci_runners  
CREATE INDEX CONCURRENTLY index_ci_runners_on_token ON public.ci_runners USING btree (token); -- table ci_runners  
CREATE INDEX CONCURRENTLY index_cluster_providers_gcp_on_cloud_run ON public.cluster_providers_gcp USING btree (cloud_run); -- table cluster_providers_gcp  
CREATE INDEX CONCURRENTLY index_container_expiration_policies_on_next_run_at_and_enabled ON public.container_expiration_policies USING btree (next_run_at, enabled); -- table container_expiration_policies  
CREATE INDEX CONCURRENTLY index_deploy_tokens_on_token_and_expires_at_and_id ON public.deploy_tokens USING btree (token, expires_at, id) WHERE (revoked IS FALSE); -- table deploy_tokens  
CREATE INDEX CONCURRENTLY partial_index_deployments_for_legacy_successful_deployments ON public.deployments USING btree (id) WHERE ((finished_at IS NULL) AND (status = 2)); -- table deployments  
CREATE INDEX CONCURRENTLY index_design_management_designs_versions_on_event ON public.design_management_designs_versions USING btree (event); -- table design_management_designs_versions  
CREATE INDEX CONCURRENTLY index_gitlab_subscription_histories_on_gitlab_subscription_id ON public.gitlab_subscription_histories USING btree (gitlab_subscription_id); -- table gitlab_subscription_histories  
CREATE INDEX CONCURRENTLY index_grafana_integrations_on_enabled ON public.grafana_integrations USING btree (enabled) WHERE (enabled IS TRUE); -- table grafana_integrations  
CREATE INDEX CONCURRENTLY index_import_failures_on_correlation_id_value ON public.import_failures USING btree (correlation_id_value); -- table import_failures  
CREATE INDEX CONCURRENTLY index_import_failures_on_project_id ON public.import_failures USING btree (project_id); -- table import_failures  
CREATE INDEX CONCURRENTLY index_issues_on_state ON public.issues USING btree (state); -- table issues  
CREATE INDEX CONCURRENTLY idx_jira_connect_subscriptions_on_installation_id ON public.jira_connect_subscriptions USING btree (jira_connect_installation_id); -- table jira_connect_subscriptions  
CREATE INDEX CONCURRENTLY index_keys_on_last_used_at ON public.keys USING btree (last_used_at DESC NULLS LAST); -- table keys  
CREATE INDEX CONCURRENTLY index_label_priorities_on_priority ON public.label_priorities USING btree (priority); -- table label_priorities  
CREATE INDEX CONCURRENTLY index_labels_on_group_id_and_title ON public.labels USING btree (group_id, title) WHERE (project_id = NULL::integer); -- table labels  
CREATE INDEX CONCURRENTLY index_labels_on_project_id_and_title ON public.labels USING btree (project_id, title) WHERE (group_id = NULL::integer); -- table labels  
CREATE INDEX CONCURRENTLY index_members_on_access_level ON public.members USING btree (access_level); -- table members  
CREATE INDEX CONCURRENTLY index_merge_request_diffs_on_merge_request_id_and_id_partial ON public.merge_request_diffs USING btree (merge_request_id, id) WHERE ((NOT stored_externally) OR (stored_externally IS NULL)); -- table merge_request_diffs  
CREATE INDEX CONCURRENTLY index_merge_request_metrics_on_latest_closed_at ON public.merge_request_metrics USING btree (latest_closed_at) WHERE (latest_closed_at IS NOT NULL); -- table merge_request_metrics  
CREATE INDEX CONCURRENTLY idx_merge_requests_on_state_id_and_merge_status ON public.merge_requests USING btree (state_id, merge_status) WHERE ((state_id = 1) AND ((merge_status)::text = 'can_be_merged'::text)); -- table merge_requests  
CREATE INDEX CONCURRENTLY index_merge_requests_on_id_and_merge_jid ON public.merge_requests USING btree (id, merge_jid) WHERE ((merge_jid IS NOT NULL) AND ((state)::text = 'locked'::text)); -- table merge_requests  
CREATE INDEX CONCURRENTLY index_merge_requests_on_source_project_and_branch_state_opened ON public.merge_requests USING btree (source_project_id, source_branch) WHERE ((state)::text = 'opened'::text); -- table merge_requests  
CREATE INDEX CONCURRENTLY index_merge_requests_on_state_and_merge_status ON public.merge_requests USING btree (state, merge_status) WHERE (((state)::text = 'opened'::text) AND ((merge_status)::text = 'can_be_merged'::text)); -- table merge_requests  
CREATE INDEX CONCURRENTLY index_merge_requests_on_target_project_id_and_iid_opened ON public.merge_requests USING btree (target_project_id, iid) WHERE ((state)::text = 'opened'::text); -- table merge_requests  
CREATE INDEX CONCURRENTLY index_namespaces_on_created_at ON public.namespaces USING btree (created_at); -- table namespaces  
CREATE INDEX CONCURRENTLY index_namespaces_on_ldap_sync_last_successful_update_at ON public.namespaces USING btree (ldap_sync_last_successful_update_at); -- table namespaces  
CREATE INDEX CONCURRENTLY index_namespaces_on_ldap_sync_last_update_at ON public.namespaces USING btree (ldap_sync_last_update_at); -- table namespaces  
CREATE INDEX CONCURRENTLY index_namespaces_on_plan_id ON public.namespaces USING btree (plan_id); -- table namespaces  
CREATE INDEX CONCURRENTLY index_namespaces_on_require_two_factor_authentication ON public.namespaces USING btree (require_two_factor_authentication); -- table namespaces  
CREATE INDEX CONCURRENTLY index_packages_dependency_links_on_dependency_id ON public.packages_dependency_links USING btree (dependency_id); -- table packages_dependency_links  
CREATE INDEX CONCURRENTLY index_pages_domains_need_auto_ssl_renewal ON public.pages_domains USING btree (certificate_source, certificate_valid_not_after) WHERE (auto_ssl_enabled = true); -- table pages_domains  
CREATE INDEX CONCURRENTLY index_pages_domains_on_verified_at_and_enabled_until ON public.pages_domains USING btree (verified_at, enabled_until); -- table pages_domains  
CREATE INDEX CONCURRENTLY index_pages_domains_on_wildcard ON public.pages_domains USING btree (wildcard); -- table pages_domains  
CREATE INDEX CONCURRENTLY index_project_custom_attributes_on_key_and_value ON public.project_custom_attributes USING btree (key, value); -- table project_custom_attributes  
CREATE INDEX CONCURRENTLY index_project_mirror_data_on_last_successful_update_at ON public.project_mirror_data USING btree (last_successful_update_at); -- table project_mirror_data  
CREATE INDEX CONCURRENTLY index_project_repositories_on_shard_id ON public.project_repositories USING btree (shard_id); -- table project_repositories  
CREATE INDEX CONCURRENTLY idx_repository_states_on_last_repository_verification_ran_at ON public.project_repository_states USING btree (project_id, last_repository_verification_ran_at) WHERE ((repository_verification_checksum IS NOT NULL) AND (last_repository_verification_failure IS NULL)); -- table project_repository_states  
CREATE INDEX CONCURRENTLY idx_repository_states_on_last_wiki_verification_ran_at ON public.project_repository_states USING btree (project_id, last_wiki_verification_ran_at) WHERE ((wiki_verification_checksum IS NOT NULL) AND (last_wiki_verification_failure IS NULL)); -- table project_repository_states  
CREATE INDEX CONCURRENTLY idx_repository_states_on_repository_failure_partial ON public.project_repository_states USING btree (last_repository_verification_failure) WHERE (last_repository_verification_failure IS NOT NULL); -- table project_repository_states  
CREATE INDEX CONCURRENTLY idx_repository_states_on_wiki_failure_partial ON public.project_repository_states USING btree (last_wiki_verification_failure) WHERE (last_wiki_verification_failure IS NOT NULL); -- table project_repository_states  
CREATE INDEX CONCURRENTLY idx_repository_states_outdated_checksums ON public.project_repository_states USING btree (project_id) WHERE (((repository_verification_checksum IS NULL) AND (last_repository_verification_failure IS NULL)) OR ((wiki_verification_checksum IS NULL) AND (last_wiki_verification_failure IS NULL))); -- table project_repository_states  
CREATE INDEX CONCURRENTLY idx_project_repository_check_partial ON public.projects USING btree (repository_storage, created_at) WHERE (last_repository_check_at IS NULL); -- table projects  
CREATE INDEX CONCURRENTLY index_on_projects_lower_path ON public.projects USING btree (lower((path)::text)); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_on_last_repository_check_at ON public.projects USING btree (last_repository_check_at) WHERE (last_repository_check_at IS NOT NULL); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_on_last_repository_check_failed ON public.projects USING btree (last_repository_check_failed); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_on_mirror_and_mirror_trigger_builds_both_true ON public.projects USING btree (id) WHERE ((mirror IS TRUE) AND (mirror_trigger_builds IS TRUE)); -- table projects  
CREATE INDEX CONCURRENTLY index_projects_on_runners_token ON public.projects USING btree (runners_token); -- table projects  
CREATE INDEX CONCURRENTLY index_prometheus_metrics_on_group ON public.prometheus_metrics USING btree ("group"); -- table prometheus_metrics  
CREATE INDEX CONCURRENTLY index_remote_mirrors_on_last_successful_update_at ON public.remote_mirrors USING btree (last_successful_update_at); -- table remote_mirrors  
CREATE INDEX CONCURRENTLY index_snippets_on_updated_at ON public.snippets USING btree (updated_at); -- table snippets  
CREATE INDEX CONCURRENTLY index_term_agreements_on_term_id ON public.term_agreements USING btree (term_id); -- table term_agreements  
CREATE INDEX CONCURRENTLY index_term_agreements_on_user_id ON public.term_agreements USING btree (user_id); -- table term_agreements  
CREATE INDEX CONCURRENTLY index_timelogs_on_spent_at ON public.timelogs USING btree (spent_at) WHERE (spent_at IS NOT NULL); -- table timelogs  
CREATE INDEX CONCURRENTLY index_timelogs_on_user_id ON public.timelogs USING btree (user_id); -- table timelogs  
CREATE INDEX CONCURRENTLY index_users_on_accepted_term_id ON public.users USING btree (accepted_term_id); -- table users  
CREATE INDEX CONCURRENTLY index_users_on_state_and_internal ON public.users USING btree (state) WHERE (ghost IS NOT TRUE); -- table users  
CREATE INDEX CONCURRENTLY index_zoom_meetings_on_issue_status ON public.zoom_meetings USING btree (issue_status); -- table zoom_meetings

  Table of contents

H003 Non-indexed Foreign Keys

Observations

Data collected: 2020-01-05 13:38:25 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

Num Schema name Table name FK name Issue Table mb writes Table scans Parent name Parent mb Parent writes Cols list Indexdef
1 public design_management_versions fk_c1440b4896 no index 0 5102 13647 users 2460 59117455 [author_id]
2 public ci_builds fk_6661f4f0e8 questionable index 383297 226996995 46 ci_resource_groups 0 6 [resource_group_id] CREATE INDEX index_for_resource_group ON public.ci_builds USING btree (resource_group_id, id) WHERE (resource_group_id IS NOT NULL)
3 public ci_builds fk_87f4cefcda questionable index 383297 226996995 46 ci_pipelines 21249 60852719 [upstream_pipeline_id] CREATE INDEX index_ci_builds_on_upstream_pipeline_id ON public.ci_builds USING btree (upstream_pipeline_id) WHERE (upstream_pipeline_id IS NOT NULL)
4 public ci_builds fk_d3130c9a7f questionable index 383297 226996995 46 ci_pipelines 21249 60852719 [commit_id] CREATE INDEX index_ci_builds_on_commit_id_and_type_and_ref ON public.ci_builds USING btree (commit_id, type, ref)
5 public ci_builds fk_d3130c9a7f questionable index 383297 226996995 46 ci_pipelines 21249 60852719 [commit_id] CREATE INDEX index_ci_builds_on_commit_id_and_artifacts_expireatandidpartial ON public.ci_builds USING btree (commit_id, artifacts_expire_at, id) WHERE (((type)::text = 'Ci::Build'::text) AND ((retried = false) OR (retried IS NULL)) AND ((name)::text = ANY ((ARRAY['sast'::character varying, 'dependency_scanning'::character varying, 'sast:container'::character varying, 'container_scanning'::character varying, 'dast'::character varying])::text[])))
6 public ci_builds fk_d3130c9a7f questionable index 383297 226996995 46 ci_pipelines 21249 60852719 [commit_id] CREATE INDEX index_ci_builds_on_commit_id_and_status_and_type ON public.ci_builds USING btree (commit_id, status, type)
7 public ci_builds fk_d3130c9a7f questionable index 383297 226996995 46 ci_pipelines 21249 60852719 [commit_id] CREATE INDEX index_ci_builds_on_commit_id_and_stage_idx_and_created_at ON public.ci_builds USING btree (commit_id, stage_idx, created_at)
8 public ci_builds fk_d3130c9a7f questionable index 383297 226996995 46 ci_pipelines 21249 60852719 [commit_id] CREATE INDEX index_ci_builds_on_commit_id_and_type_and_name_and_ref ON public.ci_builds USING btree (commit_id, type, name, ref)
9 public events fk_61fbf6ca48 questionable index 35229 69115887 0 namespaces 796 928871 [group_id] CREATE INDEX index_events_on_group_id_partial ON public.events USING btree (group_id) WHERE (group_id IS NOT NULL)
10 public ci_pipelines fk_190998ef09 questionable index 21249 60852719 14 external_pull_requests 5 143231 [external_pull_request_id] CREATE INDEX index_ci_pipelines_on_external_pull_request_id ON public.ci_pipelines USING btree (external_pull_request_id) WHERE (external_pull_request_id IS NOT NULL)
11 public ci_pipelines fk_a23be95014 questionable index 21249 60852719 14 merge_requests 17017 93155519 [merge_request_id] CREATE INDEX index_ci_pipelines_on_merge_request_id ON public.ci_pipelines USING btree (merge_request_id) WHERE (merge_request_id IS NOT NULL)
12 public merge_requests fk_641731faff questionable index 17017 93155519 62 users 2460 59117455 [updated_by_id] CREATE INDEX index_merge_requests_on_updated_by_id ON public.merge_requests USING btree (updated_by_id) WHERE (updated_by_id IS NOT NULL)
13 public merge_requests fk_ad525e1f87 questionable index 17017 93155519 62 users 2460 59117455 [merge_user_id] CREATE INDEX index_merge_requests_on_merge_user_id ON public.merge_requests USING btree (merge_user_id) WHERE (merge_user_id IS NOT NULL)
14 public issues fk_9c4516d665 questionable index 14869 19117437 8462 issues 14869 19117437 [duplicated_to_id] CREATE INDEX index_issues_on_duplicated_to_id ON public.issues USING btree (duplicated_to_id) WHERE (duplicated_to_id IS NOT NULL)
15 public issues fk_a194299be1 questionable index 14869 19117437 8462 issues 14869 19117437 [moved_to_id] CREATE INDEX index_issues_on_moved_to_id ON public.issues USING btree (moved_to_id) WHERE (moved_to_id IS NOT NULL)
16 public issues fk_df75a7c8b8 questionable index 14869 19117437 8462 epics 11 162730 [promoted_to_epic_id] CREATE INDEX index_issues_on_promoted_to_epic_id ON public.issues USING btree (promoted_to_epic_id) WHERE (promoted_to_epic_id IS NOT NULL)
17 public issues fk_ffed080f01 questionable index 14869 19117437 8462 users 2460 59117455 [updated_by_id] CREATE INDEX index_issues_on_updated_by_id ON public.issues USING btree (updated_by_id) WHERE (updated_by_id IS NOT NULL)
18 public system_note_metadata fk_fbd87415c9 questionable index 9363 20750898 0 description_versions 15 21394 [description_version_id] CREATE UNIQUE INDEX index_system_note_metadata_on_description_version_id ON public.system_note_metadata USING btree (description_version_id) WHERE (description_version_id IS NOT NULL)
19 public projects fk_0a31cca0b8 questionable index 3378 70357756 89599 users 2460 59117455 [marked_for_deletion_by_user_id] CREATE INDEX index_projects_on_marked_for_deletion_by_user_id ON public.projects USING btree (marked_for_deletion_by_user_id) WHERE (marked_for_deletion_by_user_id IS NOT NULL)
20 public projects fk_6e5c14658a questionable index 3378 70357756 89599 pool_repositories 2 29275 [pool_repository_id] CREATE INDEX index_projects_on_pool_repository_id ON public.projects USING btree (pool_repository_id) WHERE (pool_repository_id IS NOT NULL)

Conclusions

Recommendations

  Table of contents

H004 Redundant Indexes

Observations

Data collected: 2020-01-05 13:39:55 +0000 UTC
Current database: gitlabhq_production

Stats reset: 2 mons 7 days 21:31:00 ago (2019-10-28 16:07:35 +0000 UTC)

The list is limited to 20 items. Total: 39.

# Table Index Redundant to 10.220.16.106 usage 10.220.16.101 usage 10.220.16.102 usage 10.220.16.103 usage 10.220.16.104 usage 10.220.16.105 usage 10.220.16.107 usage 10.220.16.108 usage 10.220.16.109 usage ▼ Index size Table size Supports FK
  =====TOTAL===== 18.38 GiB 52.70 GiB
1 taggings index_taggings_on_taggable_id_and_taggable_type index_taggings_on_taggable_id_and_taggable_type_and_context
 0 5.90 GiB 9.76 GiB
2 ci_stages index_ci_stages_on_pipeline_id index_ci_stages_on_pipeline_id_and_name
index_ci_stages_on_pipeline_id_and_position
 0 4.64 GiB 15.00 GiB Yes
3 taggings index_taggings_on_tag_id taggings_idx
 0 3.76 GiB 9.76 GiB
4 deployments index_deployments_on_project_id_and_status index_deployments_on_project_id_and_status_and_created_at
 0 1.24 GiB 6.94 GiB
5 notification_settings index_notification_settings_on_user_id index_notifications_on_user_id_and_source_id_and_source_type
 0 493.92 MiB 1.20 GiB
6 issue_metrics index_issue_metrics index_issue_metrics_on_issue_id_and_timestamps
 0 471.75 MiB 1.09 GiB
7 users index_users_on_email_trigram index_users_on_username_trigram
 0 438.91 MiB 2.41 GiB
8 projects index_on_projects_path index_projects_on_runners_token
 0 407.36 MiB 3.30 GiB Yes
9 merge_request_assignees index_merge_request_assignees_on_merge_request_id index_merge_request_assignees_on_merge_request_id_and_user_id
 0 362.43 MiB 478.75 MiB
10 namespaces index_namespaces_on_path index_namespaces_on_runners_token
index_namespaces_on_runners_token_encrypted
 0 259.12 MiB 795.50 MiB Yes
11 personal_access_tokens index_personal_access_tokens_on_user_id index_pat_on_user_id_and_expires_at
 0 124.65 MiB 623.96 MiB
12 ci_runners index_ci_runners_on_token index_ci_runners_on_token_encrypted
 0 120.86 MiB 371.92 MiB
13 term_agreements index_term_agreements_on_user_id term_agreements_unique_index
 0 86.77 MiB 216.86 MiB
14 namespace_root_storage_statistics index_namespace_root_storage_statistics_on_namespace_id namespace_root_storage_statistics_pkey
 0 30.00 MiB 99.11 MiB
15 issue_links index_issue_links_on_source_id index_issue_links_on_source_id_and_target_id
 0 21.68 MiB 44.62 MiB
16 user_callouts index_user_callouts_on_user_id index_user_callouts_on_user_id_and_feature_name
 0 19.48 MiB 31.23 MiB
17 container_repositories index_container_repositories_on_project_id index_container_repositories_on_project_id_and_name
 0 18.75 MiB 42.00 MiB
18 ci_pipelines_config index_ci_pipelines_config_on_pipeline_id ci_pipelines_config_pkey
 0 18.24 MiB 73.68 MiB
19 namespace_aggregation_schedules index_namespace_aggregation_schedules_on_namespace_id namespace_aggregation_schedules_pkey
 0 14.83 MiB 544.00 KiB
20 project_feature_usages index_project_feature_usages_on_project_id project_feature_usages_pkey
 0 11.16 MiB 17.25 MiB

Conclusions

  • [P3] 39 redundant index(es) have been found.

Recommendations

  • [P3] Use the database migration provided below to drop the redundant indexes. Keep in mind, that under load, it is recommended to use DROP INDEX CONCURRENTLY (and CREATE INDEX CONCURRENTLY if reverting is needed) to avoid blocking issues.
  • Be careful dropping the indexes. If you have multiple setups of your software, the analysis of just a single setup might be not enough. Some indexes might be used (and therefore, needed) only on a limited number of setups. Also, in some cases, developers prepare indexes for new features in advance – in such cases, dropping those indexes is not a good idea.
  • If there are some doubts, consider a more careful approach. Before actualdropping, indexes disable listed in this report. For this, use queries like `UPDATE pg_index SET indisvalid = false WHERE indexrelid::regclass = (select oid from pg_class where relname = 'u_users_email');. Indexes will continue to get updates. In case of some performance degradations, re-enable the corresponding indexes, setting indisvalid to true. If everything looks fine, after a significant period of observations, proceed with DROP INDEX CONCURRENTLY.
  • "DO" database migrations
DROP INDEX CONCURRENTLY index_analytics_ca_group_stages_on_group_id; -- 8192 bytes, index_analytics_ca_group_stages_on_group_id_and_name, table analytics_cycle_analytics_group_stages  
DROP INDEX CONCURRENTLY index_analytics_ca_project_stages_on_project_id; -- 8192 bytes, index_analytics_ca_project_stages_on_project_id_and_name, table analytics_cycle_analytics_project_stages  
DROP INDEX CONCURRENTLY index_approval_rules_code_owners_rule_type; -- 4688 kB, index_approval_rule_name_for_code_owners_rule_type, table approval_merge_request_rules  
DROP INDEX CONCURRENTLY index_aws_roles_on_user_id; -- 16 kB, aws_roles_pkey, table aws_roles  
DROP INDEX CONCURRENTLY index_board_group_recent_visits_on_user_id; -- 896 kB, index_board_group_recent_visits_on_user_group_and_board, table board_group_recent_visits  
DROP INDEX CONCURRENTLY index_board_project_recent_visits_on_user_id; -- 3208 kB, index_board_project_recent_visits_on_user_project_and_board, table board_project_recent_visits  
DROP INDEX CONCURRENTLY index_ci_pipelines_config_on_pipeline_id; -- 18 MB, ci_pipelines_config_pkey, table ci_pipelines_config  
DROP INDEX CONCURRENTLY index_ci_runners_on_token; -- 121 MB, index_ci_runners_on_token_encrypted, table ci_runners  
DROP INDEX CONCURRENTLY index_ci_stages_on_pipeline_id; -- 4743 MB, index_ci_stages_on_pipeline_id_and_name, index_ci_stages_on_pipeline_id_and_position, table ci_stages  
DROP INDEX CONCURRENTLY index_clusters_kubernetes_namespaces_on_cluster_id; -- 1520 kB, index_kubernetes_namespaces_on_cluster_project_environment_id, kubernetes_namespaces_cluster_and_namespace, table clusters_kubernetes_namespaces  
DROP INDEX CONCURRENTLY index_container_repositories_on_project_id; -- 19 MB, index_container_repositories_on_project_id_and_name, table container_repositories  
DROP INDEX CONCURRENTLY index_deployments_on_project_id_and_status; -- 1267 MB, index_deployments_on_project_id_and_status_and_created_at, table deployments  
DROP INDEX CONCURRENTLY index_design_management_designs_versions_on_design_id; -- 104 kB, design_management_designs_versions_uniqueness, table design_management_designs_versions  
DROP INDEX CONCURRENTLY index_milestone; -- 2128 kB, index_epics_on_closed_by_id, index_epics_on_parent_id, table epics  
DROP INDEX CONCURRENTLY index_geo_node_namespace_links_on_geo_node_id; -- 8192 bytes, index_geo_node_namespace_links_on_geo_node_id_and_namespace_id, table geo_node_namespace_links  
DROP INDEX CONCURRENTLY index_issue_links_on_source_id; -- 22 MB, index_issue_links_on_source_id_and_target_id, table issue_links  
DROP INDEX CONCURRENTLY index_issue_metrics; -- 472 MB, index_issue_metrics_on_issue_id_and_timestamps, table issue_metrics  
DROP INDEX CONCURRENTLY idx_jira_connect_subscriptions_on_installation_id; -- 16 kB, idx_jira_connect_subscriptions_on_installation_id_namespace_id, table jira_connect_subscriptions  
DROP INDEX CONCURRENTLY index_list_user_preferences_on_user_id; -- 4464 kB, index_list_user_preferences_on_user_id_and_list_id, table list_user_preferences  
DROP INDEX CONCURRENTLY index_merge_request_assignees_on_merge_request_id; -- 362 MB, index_merge_request_assignees_on_merge_request_id_and_user_id, table merge_request_assignees  
DROP INDEX CONCURRENTLY index_namespace_aggregation_schedules_on_namespace_id; -- 15 MB, namespace_aggregation_schedules_pkey, table namespace_aggregation_schedules  
DROP INDEX CONCURRENTLY index_namespace_root_storage_statistics_on_namespace_id; -- 30 MB, namespace_root_storage_statistics_pkey, table namespace_root_storage_statistics  
DROP INDEX CONCURRENTLY index_namespaces_on_path; -- 259 MB, index_namespaces_on_runners_token, index_namespaces_on_runners_token_encrypted, table namespaces  
DROP INDEX CONCURRENTLY index_notification_settings_on_user_id; -- 494 MB, index_notifications_on_user_id_and_source_id_and_source_type, table notification_settings  
DROP INDEX CONCURRENTLY index_packages_packages_on_project_id; -- 1744 kB, idx_packages_packages_on_project_id_name_version_package_type, table packages_packages  
DROP INDEX CONCURRENTLY index_pages_domains_on_project_id; -- 3576 kB, index_pages_domains_on_project_id_and_enabled_until, table pages_domains  
DROP INDEX CONCURRENTLY index_pages_domains_on_verified_at; -- 4584 kB, index_pages_domains_on_verified_at_and_enabled_until, table pages_domains  
DROP INDEX CONCURRENTLY index_personal_access_tokens_on_user_id; -- 125 MB, index_pat_on_user_id_and_expires_at, table personal_access_tokens  
DROP INDEX CONCURRENTLY index_project_feature_usages_on_project_id; -- 11 MB, project_feature_usages_pkey, table project_feature_usages  
DROP INDEX CONCURRENTLY index_on_projects_path; -- 407 MB, index_projects_on_runners_token, table projects  
DROP INDEX CONCURRENTLY index_protected_environments_on_project_id; -- 96 kB, index_protected_environments_on_project_id_and_name, table protected_environments  
DROP INDEX CONCURRENTLY index_protected_tags_on_project_id; -- 632 kB, index_protected_tags_on_project_id_and_name, table protected_tags  
DROP INDEX CONCURRENTLY index_taggings_on_tag_id; -- 3842 MB, taggings_idx, table taggings  
DROP INDEX CONCURRENTLY index_taggings_on_taggable_id_and_taggable_type; -- 6033 MB, index_taggings_on_taggable_id_and_taggable_type_and_context, table taggings  
DROP INDEX CONCURRENTLY index_term_agreements_on_user_id; -- 87 MB, term_agreements_unique_index, table term_agreements  
DROP INDEX CONCURRENTLY index_user_callouts_on_user_id; -- 19 MB, index_user_callouts_on_user_id_and_feature_name, table user_callouts  
DROP INDEX CONCURRENTLY index_user_statuses_on_user_id; -- 2640 kB, user_statuses_pkey, table user_statuses  
DROP INDEX CONCURRENTLY index_users_on_email_trigram; -- 439 MB, index_users_on_username_trigram, table users  
DROP INDEX CONCURRENTLY index_vulnerabilities_on_milestone_id; -- 8192 bytes, index_vulnerabilities_on_resolved_by_id, table vulnerabilities
  • "UNDO" database migrations
CREATE INDEX CONCURRENTLY index_analytics_ca_group_stages_on_group_id ON public.analytics_cycle_analytics_group_stages USING btree (group_id); -- table analytics_cycle_analytics_group_stages  
CREATE INDEX CONCURRENTLY index_analytics_ca_project_stages_on_project_id ON public.analytics_cycle_analytics_project_stages USING btree (project_id); -- table analytics_cycle_analytics_project_stages  
CREATE INDEX CONCURRENTLY index_approval_rules_code_owners_rule_type ON public.approval_merge_request_rules USING btree (merge_request_id, rule_type) WHERE (rule_type = 2); -- table approval_merge_request_rules  
CREATE UNIQUE INDEX index_aws_roles_on_user_id ON public.aws_roles USING btree (user_id); -- table aws_roles  
CREATE INDEX CONCURRENTLY index_board_group_recent_visits_on_user_id ON public.board_group_recent_visits USING btree (user_id); -- table board_group_recent_visits  
CREATE INDEX CONCURRENTLY index_board_project_recent_visits_on_user_id ON public.board_project_recent_visits USING btree (user_id); -- table board_project_recent_visits  
CREATE INDEX CONCURRENTLY index_ci_pipelines_config_on_pipeline_id ON public.ci_pipelines_config USING btree (pipeline_id); -- table ci_pipelines_config  
CREATE INDEX CONCURRENTLY index_ci_runners_on_token ON public.ci_runners USING btree (token); -- table ci_runners  
CREATE INDEX CONCURRENTLY index_ci_stages_on_pipeline_id ON public.ci_stages USING btree (pipeline_id); -- table ci_stages  
CREATE INDEX CONCURRENTLY index_clusters_kubernetes_namespaces_on_cluster_id ON public.clusters_kubernetes_namespaces USING btree (cluster_id); -- table clusters_kubernetes_namespaces  
CREATE INDEX CONCURRENTLY index_container_repositories_on_project_id ON public.container_repositories USING btree (project_id); -- table container_repositories  
CREATE INDEX CONCURRENTLY index_deployments_on_project_id_and_status ON public.deployments USING btree (project_id, status); -- table deployments  
CREATE INDEX CONCURRENTLY index_design_management_designs_versions_on_design_id ON public.design_management_designs_versions USING btree (design_id); -- table design_management_designs_versions  
CREATE INDEX CONCURRENTLY index_milestone ON public.epics USING btree (milestone_id); -- table epics  
CREATE INDEX CONCURRENTLY index_geo_node_namespace_links_on_geo_node_id ON public.geo_node_namespace_links USING btree (geo_node_id); -- table geo_node_namespace_links  
CREATE INDEX CONCURRENTLY index_issue_links_on_source_id ON public.issue_links USING btree (source_id); -- table issue_links  
CREATE INDEX CONCURRENTLY index_issue_metrics ON public.issue_metrics USING btree (issue_id); -- table issue_metrics  
CREATE INDEX CONCURRENTLY idx_jira_connect_subscriptions_on_installation_id ON public.jira_connect_subscriptions USING btree (jira_connect_installation_id); -- table jira_connect_subscriptions  
CREATE INDEX CONCURRENTLY index_list_user_preferences_on_user_id ON public.list_user_preferences USING btree (user_id); -- table list_user_preferences  
CREATE INDEX CONCURRENTLY index_merge_request_assignees_on_merge_request_id ON public.merge_request_assignees USING btree (merge_request_id); -- table merge_request_assignees  
CREATE UNIQUE INDEX index_namespace_aggregation_schedules_on_namespace_id ON public.namespace_aggregation_schedules USING btree (namespace_id); -- table namespace_aggregation_schedules  
CREATE UNIQUE INDEX index_namespace_root_storage_statistics_on_namespace_id ON public.namespace_root_storage_statistics USING btree (namespace_id); -- table namespace_root_storage_statistics  
CREATE INDEX CONCURRENTLY index_namespaces_on_path ON public.namespaces USING btree (path); -- table namespaces  
CREATE INDEX CONCURRENTLY index_notification_settings_on_user_id ON public.notification_settings USING btree (user_id); -- table notification_settings  
CREATE INDEX CONCURRENTLY index_packages_packages_on_project_id ON public.packages_packages USING btree (project_id); -- table packages_packages  
CREATE INDEX CONCURRENTLY index_pages_domains_on_project_id ON public.pages_domains USING btree (project_id); -- table pages_domains  
CREATE INDEX CONCURRENTLY index_pages_domains_on_verified_at ON public.pages_domains USING btree (verified_at); -- table pages_domains  
CREATE INDEX CONCURRENTLY index_personal_access_tokens_on_user_id ON public.personal_access_tokens USING btree (user_id); -- table personal_access_tokens  
CREATE INDEX CONCURRENTLY index_project_feature_usages_on_project_id ON public.project_feature_usages USING btree (project_id); -- table project_feature_usages  
CREATE INDEX CONCURRENTLY index_on_projects_path ON public.projects USING btree (path); -- table projects  
CREATE INDEX CONCURRENTLY index_protected_environments_on_project_id ON public.protected_environments USING btree (project_id); -- table protected_environments  
CREATE INDEX CONCURRENTLY index_protected_tags_on_project_id ON public.protected_tags USING btree (project_id); -- table protected_tags  
CREATE INDEX CONCURRENTLY index_taggings_on_tag_id ON public.taggings USING btree (tag_id); -- table taggings  
CREATE INDEX CONCURRENTLY index_taggings_on_taggable_id_and_taggable_type ON public.taggings USING btree (taggable_id, taggable_type); -- table taggings  
CREATE INDEX CONCURRENTLY index_term_agreements_on_user_id ON public.term_agreements USING btree (user_id); -- table term_agreements  
CREATE INDEX CONCURRENTLY index_user_callouts_on_user_id ON public.user_callouts USING btree (user_id); -- table user_callouts  
CREATE INDEX CONCURRENTLY index_user_statuses_on_user_id ON public.user_statuses USING btree (user_id); -- table user_statuses  
CREATE INDEX CONCURRENTLY index_users_on_email_trigram ON public.users USING gin (email gin_trgm_ops); -- table users  
CREATE INDEX CONCURRENTLY index_vulnerabilities_on_milestone_id ON public.vulnerabilities USING btree (milestone_id); -- table vulnerabilities

  Table of contents

K001 Globally Aggregated Query Metrics

Observations

Data collected: 2020-01-05 13:39:56 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

Start: 2020-01-05T13:05:37.439573+00:00
End: 2020-01-05T13:38:26.865325+00:00
Period seconds: 1969.42575
Period age: 00:32:49.425752

Error (calls): 0.00 (0.00%)
Error (total time): 0.00 (0.00%)

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
14,411,768
7.32K/sec
1.00/call
100.00%		 2,018,713.78 ms
1025.027 ms/sec
0.140 ms/call
100.00%		 96,772,303
49.14K/sec
6.71/call
100.00%		 442,227,011 blks
224.55K blks/sec
30.69 blks/call
100.00%		 170,389 blks
86.52 blks/sec
0.01 blks/call
100.00%		 458,026 blks
232.57 blks/sec
0.03 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 96,446.77 ms
48.972 ms/sec
0.007 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica servers:

Replica (10.220.16.101)

Start: 2020-01-05T13:05:31.853449+00:00
End: 2020-01-05T13:35:50.660149+00:00
Period seconds: 1818.8067
Period age: 00:30:18.8067

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
4,970,473
2.74K/sec
1.00/call
100.00%		 877,126.34 ms
482.254 ms/sec
0.176 ms/call
100.00%		 27,693,257
15.23K/sec
5.57/call
100.00%		 456,107,253 blks
250.78K blks/sec
91.76 blks/call
100.00%		 261,690 blks
143.88 blks/sec
0.05 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 17,246.19 ms
9.482 ms/sec
0.003 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.102)

Start: 2020-01-05T13:05:32.987076+00:00
End: 2020-01-05T13:36:18.559256+00:00
Period seconds: 1845.57218
Period age: 00:30:45.57218

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
6,174,741
3.35K/sec
1.00/call
100.00%		 720,455.62 ms
390.370 ms/sec
0.117 ms/call
100.00%		 27,820,151
15.08K/sec
4.51/call
100.00%		 325,402,583 blks
176.32K blks/sec
52.70 blks/call
100.00%		 270,888 blks
146.78 blks/sec
0.04 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 20,053.70 ms
10.866 ms/sec
0.003 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.103)

Start: 2020-01-05T13:05:34.092382+00:00
End: 2020-01-05T13:36:46.554508+00:00
Period seconds: 1872.46213
Period age: 00:31:12.462126

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
8,035,279
4.30K/sec
1.00/call
100.00%		 763,391.00 ms
407.694 ms/sec
0.095 ms/call
100.00%		 27,170,288
14.52K/sec
3.38/call
100.00%		 342,435,436 blks
182.88K blks/sec
42.62 blks/call
100.00%		 213,969 blks
114.27 blks/sec
0.03 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 16,591.38 ms
8.861 ms/sec
0.002 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.104)

Start: 2020-01-05T13:05:35.201287+00:00
End: 2020-01-05T13:37:15.175732+00:00
Period seconds: 1899.97444
Period age: 00:31:39.974445

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
7,891,588
4.16K/sec
1.00/call
100.00%		 756,459.10 ms
398.142 ms/sec
0.096 ms/call
100.00%		 27,330,644
14.39K/sec
3.46/call
100.00%		 333,379,509 blks
175.47K blks/sec
42.24 blks/call
100.00%		 176,457 blks
92.87 blks/sec
0.02 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 19,203.71 ms
10.107 ms/sec
0.002 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.105)

Start: 2020-01-05T13:05:36.33304+00:00
End: 2020-01-05T13:37:43.269132+00:00
Period seconds: 1926.93609
Period age: 00:32:06.936092

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
8,120,559
4.22K/sec
1.00/call
100.00%		 824,263.71 ms
427.759 ms/sec
0.102 ms/call
100.00%		 27,994,500
14.53K/sec
3.45/call
100.00%		 340,878,337 blks
176.91K blks/sec
41.98 blks/call
100.00%		 204,747 blks
106.26 blks/sec
0.03 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 22,365.25 ms
11.607 ms/sec
0.003 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.107)

Start: 2020-01-05T13:05:38.544364+00:00
End: 2020-01-05T13:38:57.033548+00:00
Period seconds: 1998.48919
Period age: 00:33:18.489184

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
6,161,234
3.09K/sec
1.00/call
100.00%		 803,318.92 ms
401.963 ms/sec
0.130 ms/call
100.00%		 30,082,928
15.06K/sec
4.88/call
100.00%		 360,200,794 blks
180.24K blks/sec
58.46 blks/call
100.00%		 207,817 blks
103.99 blks/sec
0.03 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 15,352.99 ms
7.682 ms/sec
0.002 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.108)

Start: 2020-01-05T13:05:39.654466+00:00
End: 2020-01-05T13:39:27.213604+00:00
Period seconds: 2027.55913
Period age: 00:33:47.559138

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
8,475,020
4.18K/sec
1.00/call
100.00%		 805,428.99 ms
397.241 ms/sec
0.095 ms/call
100.00%		 30,209,044
14.90K/sec
3.56/call
100.00%		 357,846,117 blks
176.50K blks/sec
42.22 blks/call
100.00%		 278,726 blks
137.47 blks/sec
0.03 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 16,549.08 ms
8.162 ms/sec
0.002 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.109)

Start: 2020-01-05T13:05:40.779676+00:00
End: 2020-01-05T13:39:56.951447+00:00
Period seconds: 2056.17177
Period age: 00:34:16.171771

Calls Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
8,551,882
4.16K/sec
1.00/call
100.00%		 895,519.98 ms
435.528 ms/sec
0.105 ms/call
100.00%		 29,413,995
14.31K/sec
3.44/call
100.00%		 386,320,910 blks
187.89K blks/sec
45.17 blks/call
100.00%		 312,067 blks
151.77 blks/sec
0.04 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 18,350.44 ms
8.925 ms/sec
0.002 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Conclusions

Recommendations

  Table of contents

K002 Workload Type ("The First Word" Analysis)

Observations

Data collected: 2020-01-05 13:39:56 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

Start: 2020-01-05T13:05:37.439573+00:00
End: 2020-01-05T13:38:26.865325+00:00
Period seconds: 1969.42575
Period age: 00:32:49.425752

Error (calls): 0.00 (0.00%)
Error (total time): 0.00 (0.00%)

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 12,239,879
6.22K/sec
1.00/call
84.93%		 1,035,568.73 ms
525.823 ms/sec
0.085 ms/call
51.30%		 92,699,713
47.07K/sec
7.57/call
95.79%		 375,668,846 blks
190.76K blks/sec
30.69 blks/call
84.95%		 100,530 blks
51.05 blks/sec
0.01 blks/call
59.00%		 12,764 blks
6.48 blks/sec
0.00 blks/call
2.79%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 66,725.19 ms
33.881 ms/sec
0.005 ms/call
69.18%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 update 629,276
319.52/sec
1.00/call
4.37%		 867,671.11 ms
440.571 ms/sec
1.379 ms/call
42.98%		 616,833
313.20/sec
0.98/call
0.64%		 25,880,473 blks
13.15K blks/sec
41.13 blks/call
5.85%		 18,754 blks
9.52 blks/sec
0.03 blks/call
11.01%		 328,017 blks
166.55 blks/sec
0.52 blks/call
71.62%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 7,201.23 ms
3.657 ms/sec
0.011 ms/call
7.47%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3 with 89,016
45.20/sec
1.00/call
0.62%		 66,964.96 ms
34.002 ms/sec
0.752 ms/call
3.32%		 3,356,929
1.71K/sec
37.71/call
3.47%		 37,423,415 blks
19.01K blks/sec
420.41 blks/call
8.46%		 5,281 blks
2.68 blks/sec
0.06 blks/call
3.10%		 476 blks
0.24 blks/sec
0.01 blks/call
0.10%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2,576.59 ms
1.308 ms/sec
0.029 ms/call
2.67%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4 insert 56,382
28.63/sec
1.00/call
0.39%		 39,311.07 ms
19.961 ms/sec
0.697 ms/call
1.95%		 94,238
47.85/sec
1.67/call
0.10%		 3,230,660 blks
1.65K blks/sec
57.30 blks/call
0.73%		 45,732 blks
23.22 blks/sec
0.81 blks/call
26.84%		 115,939 blks
58.87 blks/sec
2.06 blks/call
25.31%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 19,942.26 ms
10.126 ms/sec
0.354 ms/call
20.68%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5 set 1,392,625
707.12/sec
1.00/call
9.66%		 8,829.58 ms
4.483 ms/sec
0.006 ms/call
0.44%		 0
0.00/sec
0.00/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6 select ... for [no key] update 4,590
2.33/sec
1.00/call
0.03%		 368.32 ms
0.187 ms/sec
0.080 ms/call
0.02%		 4,590
2.33/sec
1.00/call
0.00%		 23,617 blks
11.99 blks/sec
5.15 blks/call
0.01%		 92 blks
0.05 blks/sec
0.02 blks/call
0.05%		 830 blks
0.42 blks/sec
0.18 blks/call
0.18%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.50 ms
0.001 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica servers:

Replica (10.220.16.101)

Start: 2020-01-05T13:05:31.853449+00:00
End: 2020-01-05T13:35:50.660149+00:00
Period seconds: 1818.8067
Period age: 00:30:18.8067

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 4,747,784
2.62K/sec
1.00/call
95.52%		 860,866.09 ms
473.314 ms/sec
0.181 ms/call
98.15%		 27,308,046
15.02K/sec
5.75/call
98.61%		 451,074,666 blks
248.01K blks/sec
95.01 blks/call
98.90%		 261,690 blks
143.88 blks/sec
0.06 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 17,246.19 ms
9.482 ms/sec
0.004 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 with 222,689
122.44/sec
1.00/call
4.48%		 16,260.25 ms
8.940 ms/sec
0.073 ms/call
1.85%		 385,211
211.79/sec
1.73/call
1.39%		 5,032,587 blks
2.77K blks/sec
22.60 blks/call
1.10%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.102)

Start: 2020-01-05T13:05:32.987076+00:00
End: 2020-01-05T13:36:18.559256+00:00
Period seconds: 1845.57218
Period age: 00:30:45.57218

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 4,795,265
2.60K/sec
1.00/call
77.66%		 698,223.19 ms
378.323 ms/sec
0.146 ms/call
96.91%		 27,426,113
14.87K/sec
5.72/call
98.58%		 320,040,673 blks
173.42K blks/sec
66.74 blks/call
98.35%		 270,888 blks
146.78 blks/sec
0.06 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 20,053.70 ms
10.866 ms/sec
0.004 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 with 227,287
123.15/sec
1.00/call
3.68%		 16,440.04 ms
8.908 ms/sec
0.072 ms/call
2.28%		 394,038
213.50/sec
1.73/call
1.42%		 5,361,910 blks
2.91K blks/sec
23.59 blks/call
1.65%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3 set 1,152,189
624.30/sec
1.00/call
18.66%		 5,792.38 ms
3.139 ms/sec
0.005 ms/call
0.80%		 0
0.00/sec
0.00/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.103)

Start: 2020-01-05T13:05:34.092382+00:00
End: 2020-01-05T13:36:46.554508+00:00
Period seconds: 1872.46213
Period age: 00:31:12.462126

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 4,597,508
2.46K/sec
1.00/call
57.22%		 740,336.05 ms
395.381 ms/sec
0.161 ms/call
96.98%		 26,879,696
14.36K/sec
5.85/call
98.93%		 341,269,268 blks
182.26K blks/sec
74.23 blks/call
99.66%		 213,969 blks
114.27 blks/sec
0.05 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 16,591.38 ms
8.861 ms/sec
0.004 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 set 3,317,312
1.78K/sec
1.00/call
41.28%		 16,662.22 ms
8.899 ms/sec
0.005 ms/call
2.18%		 0
0.00/sec
0.00/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3 with 120,459
64.33/sec
1.00/call
1.50%		 6,392.72 ms
3.414 ms/sec
0.053 ms/call
0.84%		 290,592
155.19/sec
2.41/call
1.07%		 1,166,168 blks
622.80 blks/sec
9.68 blks/call
0.34%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.104)

Start: 2020-01-05T13:05:35.201287+00:00
End: 2020-01-05T13:37:15.175732+00:00
Period seconds: 1899.97444
Period age: 00:31:39.974445

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 4,306,031
2.27K/sec
1.00/call
54.56%		 726,257.40 ms
382.246 ms/sec
0.169 ms/call
96.01%		 26,927,562
14.18K/sec
6.25/call
98.53%		 331,121,765 blks
174.28K blks/sec
76.90 blks/call
99.32%		 176,457 blks
92.87 blks/sec
0.04 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 19,203.71 ms
10.107 ms/sec
0.004 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 set 3,353,432
1.77K/sec
1.00/call
42.49%		 17,015.19 ms
8.955 ms/sec
0.005 ms/call
2.25%		 0
0.00/sec
0.00/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3 with 232,125
122.17/sec
1.00/call
2.94%		 13,186.52 ms
6.940 ms/sec
0.057 ms/call
1.74%		 403,082
212.15/sec
1.74/call
1.47%		 2,257,744 blks
1.19K blks/sec
9.73 blks/call
0.68%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.105)

Start: 2020-01-05T13:05:36.33304+00:00
End: 2020-01-05T13:37:43.269132+00:00
Period seconds: 1926.93609
Period age: 00:32:06.936092

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 4,492,776
2.34K/sec
1.00/call
55.33%		 792,929.35 ms
411.497 ms/sec
0.176 ms/call
96.20%		 27,587,979
14.32K/sec
6.14/call
98.55%		 338,601,037 blks
175.72K blks/sec
75.37 blks/call
99.33%		 204,747 blks
106.26 blks/sec
0.05 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 22,365.25 ms
11.607 ms/sec
0.005 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 set 3,394,563
1.77K/sec
1.00/call
41.80%		 17,816.96 ms
9.246 ms/sec
0.005 ms/call
2.16%		 0
0.00/sec
0.00/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3 with 233,220
121.03/sec
1.00/call
2.87%		 13,517.40 ms
7.015 ms/sec
0.058 ms/call
1.64%		 406,521
210.97/sec
1.74/call
1.45%		 2,277,300 blks
1.19K blks/sec
9.76 blks/call
0.67%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.107)

Start: 2020-01-05T13:05:38.544364+00:00
End: 2020-01-05T13:38:57.033548+00:00
Period seconds: 1998.48919
Period age: 00:33:18.489184

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 5,021,906
2.52K/sec
1.00/call
81.51%		 780,262.88 ms
390.426 ms/sec
0.155 ms/call
97.13%		 29,657,137
14.84K/sec
5.91/call
98.58%		 353,590,747 blks
176.93K blks/sec
70.41 blks/call
98.16%		 207,817 blks
103.99 blks/sec
0.04 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 15,352.99 ms
7.682 ms/sec
0.003 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 with 245,072
122.63/sec
1.00/call
3.98%		 18,623.34 ms
9.319 ms/sec
0.076 ms/call
2.32%		 425,791
213.06/sec
1.74/call
1.42%		 6,610,047 blks
3.31K blks/sec
26.97 blks/call
1.84%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3 set 894,256
447.47/sec
1.00/call
14.51%		 4,432.69 ms
2.218 ms/sec
0.005 ms/call
0.55%		 0
0.00/sec
0.00/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.108)

Start: 2020-01-05T13:05:39.654466+00:00
End: 2020-01-05T13:39:27.213604+00:00
Period seconds: 2027.55913
Period age: 00:33:47.559138

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 4,800,197
2.37K/sec
1.00/call
56.64%		 774,500.79 ms
381.987 ms/sec
0.161 ms/call
96.16%		 29,776,700
14.69K/sec
6.20/call
98.57%		 355,426,662 blks
175.30K blks/sec
74.04 blks/call
99.32%		 278,726 blks
137.47 blks/sec
0.06 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 16,549.08 ms
8.162 ms/sec
0.003 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 set 3,425,613
1.69K/sec
1.00/call
40.42%		 17,407.92 ms
8.586 ms/sec
0.005 ms/call
2.16%		 0
0.00/sec
0.00/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3 with 249,210
122.91/sec
1.00/call
2.94%		 13,520.28 ms
6.668 ms/sec
0.054 ms/call
1.68%		 432,344
213.23/sec
1.73/call
1.43%		 2,419,455 blks
1.20K blks/sec
9.71 blks/call
0.68%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.109)

Start: 2020-01-05T13:05:40.779676+00:00
End: 2020-01-05T13:39:56.951447+00:00
Period seconds: 2056.17177
Period age: 00:34:16.171771

# Workload type Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1 select 4,659,466
2.27K/sec
1.00/call
54.48%		 863,490.10 ms
419.950 ms/sec
0.185 ms/call
96.42%		 28,980,631
14.10K/sec
6.22/call
98.53%		 383,900,581 blks
186.71K blks/sec
82.39 blks/call
99.37%		 312,067 blks
151.77 blks/sec
0.07 blks/call
100.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 18,350.44 ms
8.925 ms/sec
0.004 ms/call
100.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2 set 3,642,675
1.78K/sec
1.00/call
42.60%		 18,339.11 ms
8.919 ms/sec
0.005 ms/call
2.05%		 0
0.00/sec
0.00/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3 with 249,741
121.46/sec
1.00/call
2.92%		 13,690.78 ms
6.658 ms/sec
0.055 ms/call
1.53%		 433,364
210.76/sec
1.74/call
1.47%		 2,420,329 blks
1.18K blks/sec
9.69 blks/call
0.63%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Conclusions

Recommendations

  Table of contents

K003 Top-20 Queries by total_time

Observations

Data collected: 2020-01-05 13:39:56 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

Start: 2020-01-05T13:05:37.439573+00:00
End: 2020-01-05T13:38:26.865325+00:00
Period seconds: 1969.42575
Period age: 00:32:49.425752

Error (calls): 0.00 (0.00%)
Error (total time): 0.00 (0.00%)

The list is limited to 20 items.

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(1094314963)		 UPDATE "project_mirror_data" SET 
"status" = ?, "last_update_scheduled_at" 
= ? WHERE "project_mirror_data"."id" 
= ?
Full query 		114,317
58.05/sec
1.00/call
0.79%		 326,904.59 ms
165.990 ms/sec
2.860 ms/call
16.19%		 114,317
58.05/sec
1.00/call
0.12%		 4,033,362 blks
2.05K blks/sec
35.28 blks/call
0.91%		 12 blks
0.01 blks/sec
0.00 blks/call
0.01%		 102,583 blks
52.09 blks/sec
0.90 blks/call
22.40%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(1028788241)		 SELECT  "projects".* FROM "projects" 
INNER JOIN project_mirror_data 
import_state ON import_state.project_id 
= projects.id WHERE "projects"."mirror" 
= ? AND "import_state"."status" 
NOT IN (?, ?) AND (import_state.next_execution_timestamp 
<= ?) AND (import_state.retry_count 
<= ?) ORDER BY import_state.next_execution_timestamp 
LIMIT ?
Full query 		1,721
0.87/sec
1.00/call
0.01%		 243,468.27 ms
123.624 ms/sec
141.469 ms/call
12.06%		 116,341
59.07/sec
67.60/call
0.12%		 153,508,989 blks
77.95K blks/sec
89.20K blks/call
34.71%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 428 blks
0.22 blks/sec
0.25 blks/call
0.09%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(3220074313)		 UPDATE "project_mirror_data" SET 
"status" = ?, "last_update_started_at" 
= ? WHERE "project_mirror_data"."id" 
= ?
Full query 		112,982
57.37/sec
1.00/call
0.78%		 236,991.51 ms
120.335 ms/sec
2.098 ms/call
11.74%		 112,982
57.37/sec
1.00/call
0.12%		 4,345,083 blks
2.21K blks/sec
38.46 blks/call
0.98%		 2 blks
0.00 blks/sec
0.00 blks/call
0.00%		 836 blks
0.42 blks/sec
0.01 blks/call
0.18%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1425633162)		 UPDATE "project_mirror_data" SET 
"jid" = ? WHERE "project_mirror_data"."id" 
= ?
Full query 		195,198
99.11/sec
1.00/call
1.35%		 137,454.26 ms
69.794 ms/sec
0.704 ms/call
6.81%		 195,198
99.11/sec
1.00/call
0.20%		 8,369,850 blks
4.25K blks/sec
42.88 blks/call
1.89%		 12 blks
0.01 blks/sec
0.00 blks/call
0.01%		 89,080 blks
45.23 blks/sec
0.46 blks/call
19.45%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(985994114)		 UPDATE "project_mirror_data" SET 
"status" = ?, "next_execution_timestamp" 
= ?, "last_update_at" = ?, "last_successful_update_at" 
= ? WHERE "project_mirror_data"."id" 
= ?
Full query 		112,785
57.27/sec
1.00/call
0.78%		 110,693.48 ms
56.206 ms/sec
0.981 ms/call
5.48%		 112,785
57.27/sec
1.00/call
0.12%		 4,597,888 blks
2.34K blks/sec
40.77 blks/call
1.04%		 1 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,757 blks
0.89 blks/sec
0.02 blks/call
0.38%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(1781119678)		 SELECT  ? AS one FROM ((SELECT "projects".* 
FROM "projects" WHERE "projects"."namespace_id" 
IN (WITH RECURSIVE "base_and_descendants" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE "namespaces"."id" 
IN (SELECT "elasticsearch_indexed_namespaces"."namespace_id" 
FROM "elasticsearch_indexed_namespaces"))
UNION
(
SELECT "namespaces".* FROM "namespaces",
 "base_and_descendants" WHERE "namespaces"."parent_id" 
= "base_and_descendants"."id")) 
SELECT "id" FROM "base_and_descendants" 
AS "namespaces"))
UNION
(SELECT 
"projects".* FROM "projects" WHERE 
"projects"."id" IN (SELECT "elasticsearch_indexed_projects"."project_id" 
FROM "elasticsearch_indexed_projects"))) 
projects WHERE "projects"."id" 
= ? LIMIT ?
Full query 		127,496
64.74/sec
1.00/call
0.88%		 90,928.26 ms
46.170 ms/sec
0.713 ms/call
4.50%		 163
0.08/sec
0.00/call
0.00%		 39,084,166 blks
19.85K blks/sec
306.55 blks/call
8.84%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 609 blks
0.31 blks/sec
0.00 blks/call
0.13%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(1608875156)		 SELECT (CASE
          WHEN (SELECT 
count(*) FROM "ci_builds" WHERE 
"ci_builds"."stage_id" = ? AND 
("ci_builds"."retried" = ? OR "ci_builds"."retried" 
IS NULL) AND (allow_failure = ? 
OR status IN (?,?,?,?,?,?,?,?)))=(
SELECT count(*) FROM "ci_builds" 
WHERE "ci_builds"."stage_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL) 
AND (allow_failure = ? OR status 
IN (?,?,?,?,?,?,?,?)) AND ("ci_builds"."status" 
IN (?))) AND (SELECT count(*) > 
? FROM "ci_builds" WHERE "ci_builds"."stage_id" 
= ? AND ("ci_builds"."retried" 
= ? OR "ci_builds"."retried" IS 
NULL) AND "ci_builds"."allow_failure" 
= ? AND "ci_builds"."status" IN 
(?, ?)) THEN ?
          WHEN (
SELECT count(*) FROM "ci_builds" 
WHERE "ci_builds"."stage_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL) 
AND (allow_failure = ? OR status 
IN (?,?,?,?,?,?,?,?)))=(SELECT 
count(*) FROM "ci_builds" WHERE 
"ci_builds"."stage_id" = ? AND 
("ci_builds"."retried" = ? OR "ci_builds"."retried" 
IS NULL) AND (al...
Full query 		19,833
10.07/sec
1.00/call
0.14%		 76,544.76 ms
38.867 ms/sec
3.859 ms/call
3.79%		 155,136
78.77/sec
7.82/call
0.16%		 27,103,211 blks
13.77K blks/sec
1.37K blks/call
6.13%		 5 blks
0.00 blks/sec
0.00 blks/call
0.00%		 29 blks
0.01 blks/sec
0.00 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2.88 ms
0.001 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(722013345)		 SELECT  "users".* FROM "users" WHERE 
"users"."unlock_token" = ? ORDER 
BY "users"."id" ASC LIMIT ?
Full query 		8
0.00/sec
1.00/call
0.00%		 73,730.33 ms
37.437 ms/sec
9216.292 ms/call
3.65%		 0
0.00/sec
0.00/call
0.00%		 37,979,661 blks
19.29K blks/sec
4.75M blks/call
8.59%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,422 blks
0.72 blks/sec
177.75 blks/call
0.31%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(2832139909)		 WITH RECURSIVE "namespaces_cte" 
AS ((SELECT "namespaces"."id", 
"members"."access_level" FROM "namespaces" 
INNER JOIN "members" ON "namespaces"."id" 
= "members"."source_id" WHERE "members"."type" 
IN (?) AND "members"."source_type" 
= ? AND "namespaces"."type" IN 
(?) AND "members"."user_id" = ? 
AND "members"."requested_at" IS 
NULL)
UNION
(SELECT "namespaces"."id",
 GREATEST("members"."access_level",
 "namespaces_cte"."access_level") 
AS access_level FROM "namespaces" 
INNER JOIN "namespaces_cte" ON 
"namespaces_cte"."id" = "namespaces"."parent_id" 
LEFT OUTER JOIN "members" ON "members"."source_id" 
= "namespaces"."id" AND "members"."source_type" 
= ? AND "members"."requested_at" 
IS NULL AND "members"."user_id" 
= ? WHERE "namespaces"."type" IN 
(?))) SELECT "project_authorizations"."project_id",
 MAX(access_level) AS access_level 
FROM ((SELECT projects.id AS project_id,
 members.access_level FROM "projects" 
INNER JOIN "members" ON "projects"."id" 
= "members"."source_id" WHERE "members"."type" 
IN (?...
Full query 		5,318
2.70/sec
1.00/call
0.04%		 45,615.97 ms
23.162 ms/sec
8.578 ms/call
2.26%		 3,349,560
1.71K/sec
629.85/call
3.46%		 35,384,026 blks
17.97K blks/sec
6.66K blks/call
8.00%		 954 blks
0.48 blks/sec
0.18 blks/call
0.56%		 77 blks
0.04 blks/sec
0.01 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 460.39 ms
0.234 ms/sec
0.087 ms/call
0.48%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(2466197879)		               SELECT a.attname, 
format_type(a.atttypid, a.atttypmod),
                     pg_get_expr(
d.adbin, d.adrelid), a.attnotnull,
 a.atttypid, a.atttypmod,
     
                c.collname, col_description(
a.attrelid, a.attnum) AS comment
               FROM pg_attribute 
a
                LEFT JOIN pg_attrdef 
d ON a.attrelid = d.adrelid AND 
a.attnum = d.adnum
            
    LEFT JOIN pg_type t ON a.atttypid 
= t.oid
                LEFT JOIN 
pg_collation c ON a.attcollation 
= c.oid AND a.attcollation <> t.typcollation
              WHERE a.attrelid 
= ?::regclass
                 
AND a.attnum > ? AND NOT a.attisdropped
              ORDER BY a.attnum

Full query 		15,858
8.05/sec
1.00/call
0.11%		 30,762.78 ms
15.620 ms/sec
1.940 ms/call
1.52%		 355,519
180.52/sec
22.42/call
0.37%		 1,658,120 blks
841.93 blks/sec
104.56 blks/call
0.37%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(3586507030)		 SELECT  ? AS one FROM "web_hooks" 
WHERE "web_hooks"."type" IN (?) 
AND "web_hooks"."group_id" = ? 
AND "web_hooks"."job_events" = 
? LIMIT ?
Full query 		5,652
2.87/sec
1.00/call
0.04%		 29,886.28 ms
15.175 ms/sec
5.288 ms/call
1.48%		 22
0.01/sec
0.00/call
0.00%		 15,805,504 blks
8.03K blks/sec
2.80K blks/call
3.57%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(791208512)		 SELECT  ? AS one FROM ((WITH RECURSIVE 
"base_and_ancestors" AS ((SELECT 
"namespaces".* FROM ((SELECT "namespaces".* 
FROM "namespaces" INNER JOIN "members" 
ON "namespaces"."id" = "members"."source_id" 
WHERE "members"."type" IN (?) AND 
"members"."source_type" = ? AND 
"namespaces"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL)
UNION
(SELECT namespaces.* 
FROM "projects" INNER JOIN "namespaces" 
ON "namespaces"."id" = "projects"."namespace_id" 
INNER JOIN "project_authorizations" 
ON "projects"."id" = "project_authorizations"."project_id" 
WHERE "project_authorizations"."user_id" 
= ?)) namespaces WHERE "namespaces"."type" 
IN (?))
UNION
(SELECT "namespaces".* 
FROM "namespaces", "base_and_ancestors" 
WHERE "namespaces"."type" IN (?) 
AND "namespaces"."id" = "base_and_ancestors"."parent_id")),
 "base_and_descendants" AS ((SELECT 
"namespaces".* FROM "namespaces" 
INNER JOIN "members" ON "namespaces"."id" 
= "members"."source_id" WHERE "members"."type" 
IN (?) AND "member...
Full query 		17
0.01/sec
1.00/call
0.00%		 27,333.19 ms
13.879 ms/sec
1607.835 ms/call
1.35%		 17
0.01/sec
1.00/call
0.00%		 10,190,700 blks
5.18K blks/sec
599.46K blks/call
2.30%		 9 blks
0.00 blks/sec
0.53 blks/call
0.01%		 29 blks
0.01 blks/sec
1.71 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2.35 ms
0.001 ms/sec
0.138 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(2388959352)		 SELECT  ? AS one FROM "ci_builds" 
WHERE "ci_builds"."type" IN (?) 
AND "ci_builds"."commit_id" = ? 
AND ("ci_builds"."retried" = ? 
OR "ci_builds"."retried" IS NULL) 
AND (EXISTS (SELECT ? FROM "ci_job_artifacts" 
WHERE (ci_builds.id = ci_job_artifacts.job_id) 
AND "ci_job_artifacts"."file_type" 
= ?)) LIMIT ?
Full query 		92,114
46.77/sec
1.00/call
0.64%		 27,332.94 ms
13.879 ms/sec
0.297 ms/call
1.35%		 7,926
4.02/sec
0.09/call
0.01%		 4,505,860 blks
2.29K blks/sec
48.92 blks/call
1.02%		 25,958 blks
13.18 blks/sec
0.28 blks/call
15.23%		 40 blks
0.02 blks/sec
0.00 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 16,131.29 ms
8.191 ms/sec
0.175 ms/call
16.73%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(2869434099)		 SELECT  ? AS one FROM "keys" WHERE 
"keys"."fingerprint" = ? AND "keys"."id" 
!= ? LIMIT ?
Full query 		729,013
370.17/sec
1.00/call
5.06%		 25,800.34 ms
13.100 ms/sec
0.035 ms/call
1.28%		 0
0.00/sec
0.00/call
0.00%		 3,666,318 blks
1.87K blks/sec
5.03 blks/call
0.83%		 550 blks
0.28 blks/sec
0.00 blks/call
0.32%		 1,252 blks
0.64 blks/sec
0.00 blks/call
0.27%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 39.32 ms
0.020 ms/sec
0.000 ms/call
0.04%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(2546813154)		 SELECT  "projects".* FROM "projects" 
WHERE "projects"."id" = ? LIMIT 
?
Full query 		509,405
258.66/sec
1.00/call
3.53%		 25,150.31 ms
12.770 ms/sec
0.049 ms/call
1.25%		 509,388
258.65/sec
1.00/call
0.53%		 2,069,242 blks
1.06K blks/sec
4.06 blks/call
0.47%		 39 blks
0.02 blks/sec
0.00 blks/call
0.02%		 191 blks
0.10 blks/sec
0.00 blks/call
0.04%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 18.76 ms
0.010 ms/sec
0.000 ms/call
0.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		507,600
257.74/sec
1.00/call
3.52%		 21,856.04 ms
11.098 ms/sec
0.043 ms/call
1.08%		 507,600
257.74/sec
1.00/call
0.52%		 2,539,146 blks
1.29K blks/sec
5.00 blks/call
0.57%		 502 blks
0.25 blks/sec
0.00 blks/call
0.29%		 58 blks
0.03 blks/sec
0.00 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 405.82 ms
0.206 ms/sec
0.001 ms/call
0.42%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(3589441560)		 SELECT ?
Full query 		6,752,091
3.43K/sec
1.00/call
46.85%		 21,221.91 ms
10.776 ms/sec
0.003 ms/call
1.05%		 6,752,091
3.43K/sec
1.00/call
6.98%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(1783654900)		 SELECT "schema_migrations"."version" 
FROM "schema_migrations" ORDER 
BY "schema_migrations"."version" 
ASC
Full query 		34,673
17.61/sec
1.00/call
0.24%		 20,403.40 ms
10.360 ms/sec
0.588 ms/call
1.01%		 74,061,528
37.61K/sec
2.14K/call
76.53%		 346,730 blks
176.06 blks/sec
10.00 blks/call
0.08%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(3258414343)		 UPDATE "ci_builds" SET "runner_id" 
= ?, "status" = ?, "started_at" 
= ?, "updated_at" = ?, "lock_version" 
= ? WHERE "ci_builds"."type" IN 
(?) AND "ci_builds"."lock_version" 
= ? AND "ci_builds"."id" = ?
Full query 		12,721
6.46/sec
1.00/call
0.09%		 18,469.00 ms
9.378 ms/sec
1.452 ms/call
0.91%		 5,390
2.74/sec
0.42/call
0.01%		 552,593 blks
280.59 blks/sec
43.44 blks/call
0.12%		 2,041 blks
1.04 blks/sec
0.16 blks/call
1.20%		 5,435 blks
2.76 blks/sec
0.43 blks/call
1.19%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 839.06 ms
0.426 ms/sec
0.066 ms/call
0.87%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		131
0.07/sec
1.00/call
0.00%		 16,964.54 ms
8.614 ms/sec
129.500 ms/call
0.84%		 131
0.07/sec
1.00/call
0.00%		 11,718,635 blks
5.96K blks/sec
89.46K blks/call
2.65%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 18 blks
0.01 blks/sec
0.14 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica servers:

Replica (10.220.16.101)

Start: 2020-01-05T13:05:31.853449+00:00
End: 2020-01-05T13:35:50.660149+00:00
Period seconds: 1818.8067
Period age: 00:30:18.8067

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(2154552891)		 SELECT  "events"."id" AS t0_r0, 
"events"."project_id" AS t0_r1,
 "events"."author_id" AS t0_r2,
 "events"."target_id" AS t0_r3,
 "events"."created_at" AS t0_r4,
 "events"."updated_at" AS t0_r5,
 "events"."action" AS t0_r6, "events"."target_type" 
AS t0_r7, "events"."group_id" AS 
t0_r8, "users"."id" AS t1_r0, "users"."email" 
AS t1_r1, "users"."encrypted_password" 
AS t1_r2, "users"."reset_password_token" 
AS t1_r3, "users"."reset_password_sent_at" 
AS t1_r4, "users"."remember_created_at" 
AS t1_r5, "users"."sign_in_count" 
AS t1_r6, "users"."current_sign_in_at" 
AS t1_r7, "users"."last_sign_in_at" 
AS t1_r8, "users"."current_sign_in_ip" 
AS t1_r9, "users"."last_sign_in_ip" 
AS t1_r10, "users"."created_at" 
AS t1_r11, "users"."updated_at" 
AS t1_r12, "users"."name" AS t1_r13,
 "users"."admin" AS t1_r14, "users"."projects_limit" 
AS t1_r15, "users"."skype" AS t1_r16,
 "users"."linkedin" AS t1_r17, 
"users"."twitter" AS t1_r18, "users"."bio" 
AS t1_r19, "users"."failed_attempts" 
AS t1_r20, "users"."locked_at" 
...
Full query 		84
0.05/sec
1.00/call
0.00%		 110,544.89 ms
60.779 ms/sec
1316.011 ms/call
12.60%		 1,675
0.92/sec
19.94/call
0.01%		 112,569,336 blks
61.90K blks/sec
1.35M blks/call
24.68%		 7,659 blks
4.21 blks/sec
91.18 blks/call
2.93%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 375.48 ms
0.206 ms/sec
4.470 ms/call
2.18%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(1032929533)		 SELECT  "projects".* FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?
Full query 		56
0.03/sec
1.00/call
0.00%		 104,935.58 ms
57.695 ms/sec
1873.850 ms/call
11.96%		 4,536
2.49/sec
81.00/call
0.02%		 45,570,795 blks
25.06K blks/sec
813.77K blks/call
9.99%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(116237109)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		347
0.19/sec
1.00/call
0.01%		 103,020.48 ms
56.642 ms/sec
296.889 ms/call
11.75%		 1,932
1.06/sec
5.57/call
0.01%		 78,232,591 blks
43.02K blks/sec
225.46K blks/call
17.15%		 28 blks
0.02 blks/sec
0.08 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.25 ms
0.001 ms/sec
0.004 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1687354761)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		304
0.17/sec
1.00/call
0.01%		 91,349.36 ms
50.225 ms/sec
300.491 ms/call
10.41%		 2,109
1.16/sec
6.94/call
0.01%		 68,538,427 blks
37.69K blks/sec
225.46K blks/call
15.03%		 28 blks
0.02 blks/sec
0.09 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.34 ms
0.001 ms/sec
0.004 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(1134924411)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  projects.id FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		28
0.02/sec
1.00/call
0.00%		 52,581.59 ms
28.910 ms/sec
1877.914 ms/call
5.99%		 0
0.00/sec
0.00/call
0.00%		 22,785,882 blks
12.53K blks/sec
813.79K blks/call
5.00%		 3 blks
0.00 blks/sec
0.11 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.05 ms
0.000 ms/sec
0.002 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(322652387)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  "projects"."namespace_id" 
FROM "projects" WHERE (EXISTS (
SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		23
0.01/sec
1.00/call
0.00%		 46,736.89 ms
25.696 ms/sec
2032.039 ms/call
5.33%		 0
0.00/sec
0.00/call
0.00%		 21,682,222 blks
11.93K blks/sec
942.71K blks/call
4.75%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		1,071
0.59/sec
1.00/call
0.02%		 22,282.38 ms
12.251 ms/sec
20.805 ms/call
2.54%		 1,071
0.59/sec
1.00/call
0.00%		 6,595,952 blks
3.63K blks/sec
6.16K blks/call
1.45%		 6,803 blks
3.74 blks/sec
6.35 blks/call
2.60%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 80.44 ms
0.044 ms/sec
0.075 ms/call
0.47%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		3,183
1.75/sec
1.00/call
0.06%		 20,322.53 ms
11.174 ms/sec
6.385 ms/call
2.32%		 320
0.18/sec
0.10/call
0.00%		 8,203,665 blks
4.52K blks/sec
2.58K blks/call
1.80%		 6,633 blks
3.65 blks/sec
2.08 blks/call
2.53%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 115.41 ms
0.063 ms/sec
0.036 ms/call
0.67%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(1488973140)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		293,257
161.24/sec
1.00/call
5.90%		 18,772.58 ms
10.321 ms/sec
0.064 ms/call
2.14%		 285,390
156.91/sec
0.97/call
1.03%		 2,604,653 blks
1.44K blks/sec
8.88 blks/call
0.57%		 2,170 blks
1.19 blks/sec
0.01 blks/call
0.83%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 82.43 ms
0.045 ms/sec
0.000 ms/call
0.48%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		121
0.07/sec
1.00/call
0.00%		 15,444.05 ms
8.491 ms/sec
127.637 ms/call
1.76%		 121
0.07/sec
1.00/call
0.00%		 11,078,776 blks
6.10K blks/sec
91.57K blks/call
2.43%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		121
0.07/sec
1.00/call
0.00%		 14,351.92 ms
7.891 ms/sec
118.611 ms/call
1.64%		 121
0.07/sec
1.00/call
0.00%		 10,939,651 blks
6.02K blks/sec
90.42K blks/call
2.40%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(2329378283)		 SELECT  "users".* FROM "users" WHERE 
"users"."id" = ? LIMIT ?
Full query 		293,423
161.33/sec
1.00/call
5.90%		 11,921.48 ms
6.555 ms/sec
0.041 ms/call
1.36%		 293,378
161.30/sec
1.00/call
1.06%		 1,173,841 blks
645.39 blks/sec
4.00 blks/call
0.26%		 1 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.01 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(4133028576)		 SELECT  "events"."id" AS t0_r0, 
"events"."project_id" AS t0_r1,
 "events"."author_id" AS t0_r2,
 "events"."target_id" AS t0_r3,
 "events"."created_at" AS t0_r4,
 "events"."updated_at" AS t0_r5,
 "events"."action" AS t0_r6, "events"."target_type" 
AS t0_r7, "events"."group_id" AS 
t0_r8, "users"."id" AS t1_r0, "users"."email" 
AS t1_r1, "users"."encrypted_password" 
AS t1_r2, "users"."reset_password_token" 
AS t1_r3, "users"."reset_password_sent_at" 
AS t1_r4, "users"."remember_created_at" 
AS t1_r5, "users"."sign_in_count" 
AS t1_r6, "users"."current_sign_in_at" 
AS t1_r7, "users"."last_sign_in_at" 
AS t1_r8, "users"."current_sign_in_ip" 
AS t1_r9, "users"."last_sign_in_ip" 
AS t1_r10, "users"."created_at" 
AS t1_r11, "users"."updated_at" 
AS t1_r12, "users"."name" AS t1_r13,
 "users"."admin" AS t1_r14, "users"."projects_limit" 
AS t1_r15, "users"."skype" AS t1_r16,
 "users"."linkedin" AS t1_r17, 
"users"."twitter" AS t1_r18, "users"."bio" 
AS t1_r19, "users"."failed_attempts" 
AS t1_r20, "users"."locked_at" 
...
Full query 		208
0.11/sec
1.00/call
0.00%		 9,881.84 ms
5.433 ms/sec
47.509 ms/call
1.13%		 2,794
1.54/sec
13.43/call
0.01%		 7,701,240 blks
4.24K blks/sec
37.03K blks/call
1.69%		 91,282 blks
50.19 blks/sec
438.86 blks/call
34.88%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,591.55 ms
0.875 ms/sec
7.652 ms/call
9.23%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(2033340823)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL)
Full query 		32,174
17.69/sec
1.00/call
0.65%		 8,788.40 ms
4.832 ms/sec
0.273 ms/call
1.00%		 442,017
243.03/sec
13.74/call
1.60%		 749,064 blks
411.84 blks/sec
23.28 blks/call
0.16%		 6,888 blks
3.79 blks/sec
0.21 blks/call
2.63%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,106.16 ms
0.608 ms/sec
0.034 ms/call
6.41%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(2364908498)		 SELECT relnamespace::regnamespace 
as schemaname,
       relname as 
relname,
       pg_total_relation_size(
oid) bytes
  FROM pg_class
 WHERE 
relkind = ?;

Full query 		243
0.13/sec
1.00/call
0.00%		 8,746.90 ms
4.809 ms/sec
35.995 ms/call
1.00%		 89,910
49.43/sec
370.00/call
0.32%		 24,953 blks
13.72 blks/sec
102.69 blks/call
0.01%		 1 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.03 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		293,245
161.23/sec
1.00/call
5.90%		 8,505.06 ms
4.676 ms/sec
0.029 ms/call
0.97%		 293,245
161.23/sec
1.00/call
1.06%		 1,467,234 blks
806.70 blks/sec
5.00 blks/call
0.32%		 12 blks
0.01 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.09 ms
0.001 ms/sec
0.000 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(1189808325)		 SELECT  "users".* FROM "users" WHERE 
("users"."state" IN (?)) AND (ghost 
IS NOT TRUE) AND "users"."bot_type" 
IS NULL AND (("users"."name" ILIKE 
? OR "users"."username" ILIKE ?) 
OR "users"."email" = ?) ORDER BY 
CASE
  WHEN users.name = ? THEN 
?
  WHEN users.username = ? THEN 
?
  WHEN users.email = ? THEN ?
 ELSE ?
END
, "users"."name" ASC 
LIMIT ?
Full query 		94
0.05/sec
1.00/call
0.00%		 7,979.07 ms
4.387 ms/sec
84.884 ms/call
0.91%		 883
0.49/sec
9.39/call
0.00%		 740,080 blks
406.90 blks/sec
7.88K blks/call
0.16%		 549 blks
0.30 blks/sec
5.84 blks/call
0.21%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 6.45 ms
0.004 ms/sec
0.069 ms/call
0.04%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(600568018)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_builds" WHERE "ci_builds"."type" 
IN (?) AND "ci_builds"."project_id" 
= ? AND "ci_builds"."status" = 
? LIMIT ?) subquery_for_count
Full query 		140
0.08/sec
1.00/call
0.00%		 7,736.84 ms
4.254 ms/sec
55.263 ms/call
0.88%		 140
0.08/sec
1.00/call
0.00%		 2,289,501 blks
1.26K blks/sec
16.36K blks/call
0.50%		 5,069 blks
2.79 blks/sec
36.21 blks/call
1.94%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 47.16 ms
0.026 ms/sec
0.337 ms/call
0.27%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		271,150
149.08/sec
1.00/call
5.46%		 7,658.55 ms
4.211 ms/sec
0.028 ms/call
0.87%		 267,080
146.84/sec
0.98/call
0.96%		 1,437,011 blks
790.08 blks/sec
5.30 blks/call
0.32%		 3,599 blks
1.98 blks/sec
0.01 blks/call
1.38%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 157.06 ms
0.086 ms/sec
0.001 ms/call
0.91%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2856151481)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."type" IN (?) 
AND "namespaces"."id" = ? AND "namespaces"."type" 
= ? LIMIT ?
Full query 		263,152
144.68/sec
1.00/call
5.29%		 7,619.92 ms
4.190 ms/sec
0.029 ms/call
0.87%		 206,332
113.44/sec
0.78/call
0.75%		 1,053,130 blks
579.02 blks/sec
4.00 blks/call
0.23%		 5 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.09 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.102)

Start: 2020-01-05T13:05:32.987076+00:00
End: 2020-01-05T13:36:18.559256+00:00
Period seconds: 1845.57218
Period age: 00:30:45.57218

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(1687354761)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		326
0.18/sec
1.00/call
0.01%		 96,233.43 ms
52.143 ms/sec
295.195 ms/call
13.36%		 1,622
0.88/sec
4.98/call
0.01%		 73,411,064 blks
39.78K blks/sec
225.19K blks/call
22.56%		 20 blks
0.01 blks/sec
0.06 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.71 ms
0.001 ms/sec
0.005 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(116237109)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		325
0.18/sec
1.00/call
0.01%		 93,539.16 ms
50.683 ms/sec
287.813 ms/call
12.98%		 1,712
0.93/sec
5.27/call
0.01%		 73,116,066 blks
39.62K blks/sec
224.98K blks/call
22.47%		 30 blks
0.02 blks/sec
0.09 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.48 ms
0.000 ms/sec
0.001 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(1032929533)		 SELECT  "projects".* FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?
Full query 		46
0.02/sec
1.00/call
0.00%		 93,458.85 ms
50.639 ms/sec
2031.714 ms/call
12.97%		 3,846
2.08/sec
83.61/call
0.01%		 41,252,734 blks
22.36K blks/sec
896.80K blks/call
12.68%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1134924411)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  projects.id FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		23
0.01/sec
1.00/call
0.00%		 49,679.57 ms
26.918 ms/sec
2159.981 ms/call
6.90%		 0
0.00/sec
0.00/call
0.00%		 20,624,707 blks
11.18K blks/sec
896.73K blks/call
6.34%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(322652387)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  "projects"."namespace_id" 
FROM "projects" WHERE (EXISTS (
SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		21
0.01/sec
1.00/call
0.00%		 41,749.65 ms
22.622 ms/sec
1988.079 ms/call
5.79%		 0
0.00/sec
0.00/call
0.00%		 19,474,027 blks
10.56K blks/sec
927.34K blks/call
5.98%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(1488973140)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		297,788
161.35/sec
1.00/call
4.82%		 18,172.74 ms
9.847 ms/sec
0.061 ms/call
2.52%		 289,695
156.97/sec
0.97/call
1.04%		 2,644,674 blks
1.44K blks/sec
8.88 blks/call
0.81%		 2,262 blks
1.23 blks/sec
0.01 blks/call
0.84%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 86.20 ms
0.047 ms/sec
0.000 ms/call
0.43%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		1,126
0.61/sec
1.00/call
0.02%		 16,897.10 ms
9.155 ms/sec
15.006 ms/call
2.35%		 1,126
0.61/sec
1.00/call
0.00%		 4,690,640 blks
2.55K blks/sec
4.17K blks/call
1.44%		 13,146 blks
7.12 blks/sec
11.67 blks/call
4.85%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 106.58 ms
0.058 ms/sec
0.095 ms/call
0.53%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		3,050
1.65/sec
1.00/call
0.05%		 16,564.56 ms
8.975 ms/sec
5.431 ms/call
2.30%		 304
0.16/sec
0.10/call
0.00%		 5,986,706 blks
3.25K blks/sec
1.97K blks/call
1.84%		 5,054 blks
2.74 blks/sec
1.66 blks/call
1.87%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 938.34 ms
0.508 ms/sec
0.308 ms/call
4.68%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= true
AND project_mirror_data.retry_count 
<= 14
AND (projects.visibility_level 
= 20 OR plans.name IN ('early_adopter',
 'bronze', 'silver', 'gold'))
AND 
project_mirror_data.status NOT 
IN ('scheduled', 'started')
AND 
project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		123
0.07/sec
1.00/call
0.00%		 14,606.55 ms
7.914 ms/sec
118.752 ms/call
2.03%		 123
0.07/sec
1.00/call
0.00%		 11,170,153 blks
6.06K blks/sec
90.82K blks/call
3.43%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		123
0.07/sec
1.00/call
0.00%		 13,772.38 ms
7.462 ms/sec
111.971 ms/call
1.91%		 123
0.07/sec
1.00/call
0.00%		 11,025,764 blks
5.98K blks/sec
89.65K blks/call
3.39%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(2033340823)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL)
Full query 		33,420
18.11/sec
1.00/call
0.54%		 13,613.19 ms
7.376 ms/sec
0.407 ms/call
1.89%		 480,962
260.60/sec
14.39/call
1.73%		 802,400 blks
434.77 blks/sec
24.01 blks/call
0.25%		 12,511 blks
6.78 blks/sec
0.37 blks/call
4.62%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 5,504.36 ms
2.982 ms/sec
0.165 ms/call
27.45%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(2329378283)		 SELECT  "users".* FROM "users" WHERE 
"users"."id" = ? LIMIT ?
Full query 		296,796
160.82/sec
1.00/call
4.81%		 11,859.30 ms
6.426 ms/sec
0.040 ms/call
1.65%		 296,744
160.79/sec
1.00/call
1.07%		 1,187,345 blks
643.35 blks/sec
4.00 blks/call
0.36%		 2 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.05 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(4133028576)		 SELECT  "events"."id" AS t0_r0, 
"events"."project_id" AS t0_r1,
 "events"."author_id" AS t0_r2,
 "events"."target_id" AS t0_r3,
 "events"."created_at" AS t0_r4,
 "events"."updated_at" AS t0_r5,
 "events"."action" AS t0_r6, "events"."target_type" 
AS t0_r7, "events"."group_id" AS 
t0_r8, "users"."id" AS t1_r0, "users"."email" 
AS t1_r1, "users"."encrypted_password" 
AS t1_r2, "users"."reset_password_token" 
AS t1_r3, "users"."reset_password_sent_at" 
AS t1_r4, "users"."remember_created_at" 
AS t1_r5, "users"."sign_in_count" 
AS t1_r6, "users"."current_sign_in_at" 
AS t1_r7, "users"."last_sign_in_at" 
AS t1_r8, "users"."current_sign_in_ip" 
AS t1_r9, "users"."last_sign_in_ip" 
AS t1_r10, "users"."created_at" 
AS t1_r11, "users"."updated_at" 
AS t1_r12, "users"."name" AS t1_r13,
 "users"."admin" AS t1_r14, "users"."projects_limit" 
AS t1_r15, "users"."skype" AS t1_r16,
 "users"."linkedin" AS t1_r17, 
"users"."twitter" AS t1_r18, "users"."bio" 
AS t1_r19, "users"."failed_attempts" 
AS t1_r20, "users"."locked_at" 
...
Full query 		250
0.14/sec
1.00/call
0.00%		 10,643.65 ms
5.767 ms/sec
42.575 ms/call
1.48%		 3,493
1.89/sec
13.97/call
0.01%		 8,014,422 blks
4.35K blks/sec
32.06K blks/call
2.46%		 108,903 blks
59.01 blks/sec
435.61 blks/call
40.20%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2,424.78 ms
1.314 ms/sec
9.699 ms/call
12.09%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(1074971172)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "issues" INNER JOIN 
"projects" ON "projects"."id" = 
"issues"."project_id" LEFT JOIN 
project_features ON projects.id 
= project_features.project_id WHERE 
"issues"."confidential" = ? AND 
(EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id) AND (project_authorizations.access_level 
>= ?)) OR projects.visibility_level 
IN (?,?)) AND ("project_features"."issues_access_level" 
IS NULL OR "project_features"."issues_access_level" 
IN (?,?) OR ("project_features"."issues_access_level" 
= ? AND EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id) AND (project_authorizations.access_level 
>= ?)))) AND ("issues"."title" 
ILIKE ? OR "issues"."description" 
ILIKE ?) LIMIT ?) subquery_for_count
Full query 		3
0.00/sec
1.00/call
0.00%		 9,670.81 ms
5.240 ms/sec
3223.603 ms/call
1.34%		 3
0.00/sec
1.00/call
0.00%		 202,192 blks
109.56 blks/sec
67.40K blks/call
0.06%		 14,115 blks
7.65 blks/sec
4.71K blks/call
5.21%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 362.91 ms
0.197 ms/sec
120.971 ms/call
1.81%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(600568018)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_builds" WHERE "ci_builds"."type" 
IN (?) AND "ci_builds"."project_id" 
= ? AND "ci_builds"."status" = 
? LIMIT ?) subquery_for_count
Full query 		141
0.08/sec
1.00/call
0.00%		 9,634.24 ms
5.220 ms/sec
68.328 ms/call
1.34%		 141
0.08/sec
1.00/call
0.00%		 3,036,519 blks
1.65K blks/sec
21.54K blks/call
0.93%		 807 blks
0.44 blks/sec
5.72 blks/call
0.30%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 123.60 ms
0.067 ms/sec
0.877 ms/call
0.62%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(1189808325)		 SELECT  "users".* FROM "users" WHERE 
("users"."state" IN (?)) AND (ghost 
IS NOT TRUE) AND "users"."bot_type" 
IS NULL AND (("users"."name" ILIKE 
? OR "users"."username" ILIKE ?) 
OR "users"."email" = ?) ORDER BY 
CASE
  WHEN users.name = ? THEN 
?
  WHEN users.username = ? THEN 
?
  WHEN users.email = ? THEN ?
 ELSE ?
END
, "users"."name" ASC 
LIMIT ?
Full query 		90
0.05/sec
1.00/call
0.00%		 8,820.29 ms
4.779 ms/sec
98.003 ms/call
1.22%		 913
0.49/sec
10.14/call
0.00%		 741,220 blks
401.62 blks/sec
8.24K blks/call
0.23%		 621 blks
0.34 blks/sec
6.90 blks/call
0.23%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 5.83 ms
0.003 ms/sec
0.065 ms/call
0.03%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		297,623
161.26/sec
1.00/call
4.82%		 8,363.62 ms
4.532 ms/sec
0.028 ms/call
1.16%		 297,623
161.26/sec
1.00/call
1.07%		 1,488,993 blks
806.79 blks/sec
5.00 blks/call
0.46%		 70 blks
0.04 blks/sec
0.00 blks/call
0.03%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.84 ms
0.001 ms/sec
0.000 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(2856151481)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."type" IN (?) 
AND "namespaces"."id" = ? AND "namespaces"."type" 
= ? LIMIT ?
Full query 		268,249
145.35/sec
1.00/call
4.34%		 7,721.37 ms
4.184 ms/sec
0.029 ms/call
1.07%		 210,525
114.07/sec
0.78/call
0.76%		 1,073,549 blks
581.69 blks/sec
4.00 blks/call
0.33%		 7 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.17 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		279,252
151.31/sec
1.00/call
4.52%		 7,426.48 ms
4.024 ms/sec
0.027 ms/call
1.03%		 275,103
149.06/sec
0.99/call
0.99%		 1,479,438 blks
801.61 blks/sec
5.30 blks/call
0.45%		 3,817 blks
2.07 blks/sec
0.01 blks/call
1.41%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 113.95 ms
0.062 ms/sec
0.000 ms/call
0.57%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2418469088)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."id" = ? LIMIT 
?
Full query 		261,749
141.83/sec
1.00/call
4.24%		 6,961.98 ms
3.772 ms/sec
0.027 ms/call
0.97%		 261,749
141.83/sec
1.00/call
0.94%		 1,047,482 blks
567.56 blks/sec
4.00 blks/call
0.32%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.103)

Start: 2020-01-05T13:05:34.092382+00:00
End: 2020-01-05T13:36:46.554508+00:00
Period seconds: 1872.46213
Period age: 00:31:12.462126

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(1032929533)		 SELECT  "projects".* FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?
Full query 		72
0.04/sec
1.00/call
0.00%		 132,424.69 ms
70.722 ms/sec
1839.232 ms/call
17.35%		 5,672
3.03/sec
78.78/call
0.02%		 56,904,164 blks
30.40K blks/sec
790.34K blks/call
16.62%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(1687354761)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		340
0.18/sec
1.00/call
0.00%		 103,445.00 ms
55.245 ms/sec
304.250 ms/call
13.55%		 2,009
1.07/sec
5.91/call
0.01%		 76,624,067 blks
40.93K blks/sec
225.37K blks/call
22.38%		 49 blks
0.03 blks/sec
0.14 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3.33 ms
0.002 ms/sec
0.010 ms/call
0.02%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(116237109)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		291
0.16/sec
1.00/call
0.00%		 83,180.36 ms
44.423 ms/sec
285.843 ms/call
10.90%		 1,154
0.62/sec
3.97/call
0.00%		 65,591,422 blks
35.03K blks/sec
225.41K blks/call
19.15%		 12 blks
0.01 blks/sec
0.04 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.19 ms
0.000 ms/sec
0.001 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1134924411)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  projects.id FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		36
0.02/sec
1.00/call
0.00%		 64,202.49 ms
34.288 ms/sec
1783.403 ms/call
8.41%		 0
0.00/sec
0.00/call
0.00%		 28,452,763 blks
15.20K blks/sec
790.36K blks/call
8.31%		 2 blks
0.00 blks/sec
0.06 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.04 ms
0.000 ms/sec
0.001 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(322652387)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  "projects"."namespace_id" 
FROM "projects" WHERE (EXISTS (
SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		32
0.02/sec
1.00/call
0.00%		 60,625.24 ms
32.377 ms/sec
1894.539 ms/call
7.94%		 0
0.00/sec
0.00/call
0.00%		 28,451,238 blks
15.20K blks/sec
889.11K blks/call
8.31%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		1,157
0.62/sec
1.00/call
0.01%		 20,065.82 ms
10.716 ms/sec
17.343 ms/call
2.63%		 1,157
0.62/sec
1.00/call
0.00%		 5,984,400 blks
3.20K blks/sec
5.18K blks/call
1.75%		 11,749 blks
6.27 blks/sec
10.15 blks/call
5.49%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 479.86 ms
0.256 ms/sec
0.415 ms/call
2.89%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(1488973140)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		301,359
160.94/sec
1.00/call
3.75%		 19,095.40 ms
10.198 ms/sec
0.063 ms/call
2.50%		 293,303
156.64/sec
0.97/call
1.08%		 2,677,281 blks
1.43K blks/sec
8.88 blks/call
0.78%		 2,086 blks
1.11 blks/sec
0.01 blks/call
0.97%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 149.69 ms
0.080 ms/sec
0.000 ms/call
0.90%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		3,183
1.70/sec
1.00/call
0.04%		 16,394.38 ms
8.756 ms/sec
5.151 ms/call
2.15%		 294
0.16/sec
0.09/call
0.00%		 7,491,298 blks
4.01K blks/sec
2.36K blks/call
2.19%		 4,329 blks
2.31 blks/sec
1.36 blks/call
2.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 254.11 ms
0.136 ms/sec
0.080 ms/call
1.53%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		125
0.07/sec
1.00/call
0.00%		 15,491.55 ms
8.273 ms/sec
123.932 ms/call
2.03%		 125
0.07/sec
1.00/call
0.00%		 11,405,366 blks
6.10K blks/sec
91.25K blks/call
3.33%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		125
0.07/sec
1.00/call
0.00%		 14,606.93 ms
7.801 ms/sec
116.855 ms/call
1.91%		 125
0.07/sec
1.00/call
0.00%		 11,261,677 blks
6.02K blks/sec
90.10K blks/call
3.29%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(2329378283)		 SELECT  "users".* FROM "users" WHERE 
"users"."id" = ? LIMIT ?
Full query 		303,764
162.23/sec
1.00/call
3.78%		 12,380.10 ms
6.612 ms/sec
0.041 ms/call
1.62%		 303,724
162.21/sec
1.00/call
1.12%		 1,215,233 blks
649.00 blks/sec
4.00 blks/call
0.35%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(1189808325)		 SELECT  "users".* FROM "users" WHERE 
("users"."state" IN (?)) AND (ghost 
IS NOT TRUE) AND "users"."bot_type" 
IS NULL AND (("users"."name" ILIKE 
? OR "users"."username" ILIKE ?) 
OR "users"."email" = ?) ORDER BY 
CASE
  WHEN users.name = ? THEN 
?
  WHEN users.username = ? THEN 
?
  WHEN users.email = ? THEN ?
 ELSE ?
END
, "users"."name" ASC 
LIMIT ?
Full query 		139
0.07/sec
1.00/call
0.00%		 10,336.18 ms
5.520 ms/sec
74.361 ms/call
1.35%		 1,529
0.82/sec
11.00/call
0.01%		 936,054 blks
499.91 blks/sec
6.74K blks/call
0.27%		 562 blks
0.30 blks/sec
4.04 blks/call
0.26%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 23.09 ms
0.012 ms/sec
0.166 ms/call
0.14%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(4133028576)		 SELECT  "events"."id" AS t0_r0, 
"events"."project_id" AS t0_r1,
 "events"."author_id" AS t0_r2,
 "events"."target_id" AS t0_r3,
 "events"."created_at" AS t0_r4,
 "events"."updated_at" AS t0_r5,
 "events"."action" AS t0_r6, "events"."target_type" 
AS t0_r7, "events"."group_id" AS 
t0_r8, "users"."id" AS t1_r0, "users"."email" 
AS t1_r1, "users"."encrypted_password" 
AS t1_r2, "users"."reset_password_token" 
AS t1_r3, "users"."reset_password_sent_at" 
AS t1_r4, "users"."remember_created_at" 
AS t1_r5, "users"."sign_in_count" 
AS t1_r6, "users"."current_sign_in_at" 
AS t1_r7, "users"."last_sign_in_at" 
AS t1_r8, "users"."current_sign_in_ip" 
AS t1_r9, "users"."last_sign_in_ip" 
AS t1_r10, "users"."created_at" 
AS t1_r11, "users"."updated_at" 
AS t1_r12, "users"."name" AS t1_r13,
 "users"."admin" AS t1_r14, "users"."projects_limit" 
AS t1_r15, "users"."skype" AS t1_r16,
 "users"."linkedin" AS t1_r17, 
"users"."twitter" AS t1_r18, "users"."bio" 
AS t1_r19, "users"."failed_attempts" 
AS t1_r20, "users"."locked_at" 
...
Full query 		240
0.13/sec
1.00/call
0.00%		 10,297.14 ms
5.499 ms/sec
42.905 ms/call
1.35%		 3,118
1.67/sec
12.99/call
0.01%		 4,655,467 blks
2.49K blks/sec
19.40K blks/call
1.36%		 92,762 blks
49.54 blks/sec
386.51 blks/call
43.35%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 5,076.24 ms
2.711 ms/sec
21.151 ms/call
30.60%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		305,070
162.92/sec
1.00/call
3.80%		 8,739.93 ms
4.668 ms/sec
0.029 ms/call
1.14%		 305,070
162.92/sec
1.00/call
1.12%		 1,526,307 blks
815.13 blks/sec
5.00 blks/call
0.45%		 17 blks
0.01 blks/sec
0.00 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.27 ms
0.001 ms/sec
0.000 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		282,245
150.73/sec
1.00/call
3.51%		 8,452.34 ms
4.514 ms/sec
0.030 ms/call
1.11%		 278,204
148.58/sec
0.99/call
1.02%		 1,495,290 blks
798.57 blks/sec
5.30 blks/call
0.44%		 3,412 blks
1.82 blks/sec
0.01 blks/call
1.59%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 695.81 ms
0.372 ms/sec
0.002 ms/call
4.19%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(2856151481)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."type" IN (?) 
AND "namespaces"."id" = ? AND "namespaces"."type" 
= ? LIMIT ?
Full query 		272,436
145.50/sec
1.00/call
3.39%		 7,994.63 ms
4.270 ms/sec
0.029 ms/call
1.05%		 213,967
114.27/sec
0.79/call
0.79%		 1,090,323 blks
582.29 blks/sec
4.00 blks/call
0.32%		 3 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.08 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(600568018)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_builds" WHERE "ci_builds"."type" 
IN (?) AND "ci_builds"."project_id" 
= ? AND "ci_builds"."status" = 
? LIMIT ?) subquery_for_count
Full query 		142
0.08/sec
1.00/call
0.00%		 7,347.26 ms
3.924 ms/sec
51.741 ms/call
0.96%		 142
0.08/sec
1.00/call
0.00%		 2,249,044 blks
1.21K blks/sec
15.84K blks/call
0.66%		 1,371 blks
0.73 blks/sec
9.65 blks/call
0.64%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 31.30 ms
0.017 ms/sec
0.220 ms/call
0.19%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(2418469088)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."id" = ? LIMIT 
?
Full query 		264,891
141.47/sec
1.00/call
3.30%		 7,251.76 ms
3.873 ms/sec
0.027 ms/call
0.95%		 264,890
141.47/sec
1.00/call
0.97%		 1,059,981 blks
566.09 blks/sec
4.00 blks/call
0.31%		 2 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.03 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(1543342767)		 SELECT  "project_features".* FROM 
"project_features" WHERE "project_features"."project_id" 
= ? LIMIT ?
Full query 		252,878
135.05/sec
1.00/call
3.15%		 6,887.24 ms
3.678 ms/sec
0.027 ms/call
0.90%		 252,878
135.05/sec
1.00/call
0.93%		 1,010,043 blks
539.42 blks/sec
3.99 blks/call
0.29%		 1,949 blks
1.04 blks/sec
0.01 blks/call
0.91%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 104.22 ms
0.056 ms/sec
0.000 ms/call
0.63%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2747454572)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND (project_mirror_data.next_execution_timestamp 
- project_mirror_data.last_update_at) 
<= ?::interval
AND project_mirror_data.last_update_at 
< NOW() - ?::interval
;
Full query 		125
0.07/sec
1.00/call
0.00%		 6,642.83 ms
3.548 ms/sec
53.143 ms/call
0.87%		 125
0.07/sec
1.00/call
0.00%		 3,771,381 blks
2.02K blks/sec
30.18K blks/call
1.10%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.104)

Start: 2020-01-05T13:05:35.201287+00:00
End: 2020-01-05T13:37:15.175732+00:00
Period seconds: 1899.97444
Period age: 00:31:39.974445

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(1032929533)		 SELECT  "projects".* FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?
Full query 		48
0.03/sec
1.00/call
0.00%		 116,159.39 ms
61.137 ms/sec
2419.987 ms/call
15.36%		 4,208
2.21/sec
87.67/call
0.02%		 46,173,675 blks
24.31K blks/sec
961.96K blks/call
13.85%		 14 blks
0.01 blks/sec
0.29 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.14 ms
0.001 ms/sec
0.024 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(116237109)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		358
0.19/sec
1.00/call
0.00%		 109,772.29 ms
57.776 ms/sec
306.627 ms/call
14.51%		 1,913
1.01/sec
5.34/call
0.01%		 80,720,479 blks
42.49K blks/sec
225.48K blks/call
24.21%		 34 blks
0.02 blks/sec
0.09 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2.28 ms
0.001 ms/sec
0.006 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(1687354761)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		341
0.18/sec
1.00/call
0.00%		 103,890.02 ms
54.680 ms/sec
304.663 ms/call
13.73%		 2,655
1.40/sec
7.79/call
0.01%		 76,912,449 blks
40.49K blks/sec
225.55K blks/call
23.07%		 20 blks
0.01 blks/sec
0.06 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.36 ms
0.001 ms/sec
0.004 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1134924411)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  projects.id FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		24
0.01/sec
1.00/call
0.00%		 55,026.32 ms
28.962 ms/sec
2292.764 ms/call
7.27%		 0
0.00/sec
0.00/call
0.00%		 23,087,306 blks
12.16K blks/sec
961.98K blks/call
6.93%		 3 blks
0.00 blks/sec
0.12 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.05 ms
0.000 ms/sec
0.002 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(322652387)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  "projects"."namespace_id" 
FROM "projects" WHERE (EXISTS (
SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		22
0.01/sec
1.00/call
0.00%		 49,867.28 ms
26.246 ms/sec
2266.694 ms/call
6.59%		 0
0.00/sec
0.00/call
0.00%		 23,086,190 blks
12.16K blks/sec
1.05M blks/call
6.92%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(1488973140)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		306,215
161.17/sec
1.00/call
3.88%		 19,934.53 ms
10.492 ms/sec
0.065 ms/call
2.64%		 297,978
156.83/sec
0.97/call
1.09%		 2,719,709 blks
1.44K blks/sec
8.88 blks/call
0.82%		 2,214 blks
1.17 blks/sec
0.01 blks/call
1.25%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 169.89 ms
0.089 ms/sec
0.001 ms/call
0.88%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		3,155
1.66/sec
1.00/call
0.04%		 16,205.75 ms
8.529 ms/sec
5.137 ms/call
2.14%		 282
0.15/sec
0.09/call
0.00%		 6,388,376 blks
3.37K blks/sec
2.03K blks/call
1.92%		 4,802 blks
2.53 blks/sec
1.52 blks/call
2.72%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 262.76 ms
0.138 ms/sec
0.083 ms/call
1.37%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		127
0.07/sec
1.00/call
0.00%		 15,708.35 ms
8.268 ms/sec
123.688 ms/call
2.08%		 127
0.07/sec
1.00/call
0.00%		 11,549,651 blks
6.08K blks/sec
90.95K blks/call
3.46%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		127
0.07/sec
1.00/call
0.00%		 14,950.94 ms
7.869 ms/sec
117.724 ms/call
1.98%		 127
0.07/sec
1.00/call
0.00%		 11,401,289 blks
6.01K blks/sec
89.78K blks/call
3.42%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(2329378283)		 SELECT  "users".* FROM "users" WHERE 
"users"."id" = ? LIMIT ?
Full query 		303,937
159.97/sec
1.00/call
3.85%		 12,562.13 ms
6.612 ms/sec
0.041 ms/call
1.66%		 303,892
159.95/sec
1.00/call
1.11%		 1,215,893 blks
639.95 blks/sec
4.00 blks/call
0.36%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		1,188
0.63/sec
1.00/call
0.02%		 12,135.01 ms
6.387 ms/sec
10.215 ms/call
1.60%		 1,188
0.63/sec
1.00/call
0.00%		 3,255,130 blks
1.72K blks/sec
2.75K blks/call
0.98%		 3,259 blks
1.72 blks/sec
2.74 blks/call
1.85%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 166.50 ms
0.088 ms/sec
0.140 ms/call
0.87%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		305,298
160.69/sec
1.00/call
3.87%		 9,024.82 ms
4.750 ms/sec
0.030 ms/call
1.19%		 305,298
160.69/sec
1.00/call
1.12%		 1,527,401 blks
803.91 blks/sec
5.00 blks/call
0.46%		 32 blks
0.02 blks/sec
0.00 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.58 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(1189808325)		 SELECT  "users".* FROM "users" WHERE 
("users"."state" IN (?)) AND (ghost 
IS NOT TRUE) AND "users"."bot_type" 
IS NULL AND (("users"."name" ILIKE 
? OR "users"."username" ILIKE ?) 
OR "users"."email" = ?) ORDER BY 
CASE
  WHEN users.name = ? THEN 
?
  WHEN users.username = ? THEN 
?
  WHEN users.email = ? THEN ?
 ELSE ?
END
, "users"."name" ASC 
LIMIT ?
Full query 		100
0.05/sec
1.00/call
0.00%		 8,851.21 ms
4.659 ms/sec
88.512 ms/call
1.17%		 902
0.47/sec
9.02/call
0.00%		 741,625 blks
390.33 blks/sec
7.42K blks/call
0.22%		 543 blks
0.29 blks/sec
5.43 blks/call
0.31%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 27.88 ms
0.015 ms/sec
0.279 ms/call
0.15%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(4133028576)		 SELECT  "events"."id" AS t0_r0, 
"events"."project_id" AS t0_r1,
 "events"."author_id" AS t0_r2,
 "events"."target_id" AS t0_r3,
 "events"."created_at" AS t0_r4,
 "events"."updated_at" AS t0_r5,
 "events"."action" AS t0_r6, "events"."target_type" 
AS t0_r7, "events"."group_id" AS 
t0_r8, "users"."id" AS t1_r0, "users"."email" 
AS t1_r1, "users"."encrypted_password" 
AS t1_r2, "users"."reset_password_token" 
AS t1_r3, "users"."reset_password_sent_at" 
AS t1_r4, "users"."remember_created_at" 
AS t1_r5, "users"."sign_in_count" 
AS t1_r6, "users"."current_sign_in_at" 
AS t1_r7, "users"."last_sign_in_at" 
AS t1_r8, "users"."current_sign_in_ip" 
AS t1_r9, "users"."last_sign_in_ip" 
AS t1_r10, "users"."created_at" 
AS t1_r11, "users"."updated_at" 
AS t1_r12, "users"."name" AS t1_r13,
 "users"."admin" AS t1_r14, "users"."projects_limit" 
AS t1_r15, "users"."skype" AS t1_r16,
 "users"."linkedin" AS t1_r17, 
"users"."twitter" AS t1_r18, "users"."bio" 
AS t1_r19, "users"."failed_attempts" 
AS t1_r20, "users"."locked_at" 
...
Full query 		249
0.13/sec
1.00/call
0.00%		 8,267.25 ms
4.351 ms/sec
33.202 ms/call
1.09%		 3,355
1.77/sec
13.47/call
0.01%		 4,019,989 blks
2.12K blks/sec
16.15K blks/call
1.21%		 42,313 blks
22.27 blks/sec
169.93 blks/call
23.98%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 4,015.63 ms
2.114 ms/sec
16.127 ms/call
20.91%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(600568018)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_builds" WHERE "ci_builds"."type" 
IN (?) AND "ci_builds"."project_id" 
= ? AND "ci_builds"."status" = 
? LIMIT ?) subquery_for_count
Full query 		131
0.07/sec
1.00/call
0.00%		 8,243.29 ms
4.339 ms/sec
62.926 ms/call
1.09%		 131
0.07/sec
1.00/call
0.00%		 2,625,936 blks
1.39K blks/sec
20.05K blks/call
0.79%		 3,893 blks
2.05 blks/sec
29.72 blks/call
2.21%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 38.63 ms
0.020 ms/sec
0.295 ms/call
0.20%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(2856151481)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."type" IN (?) 
AND "namespaces"."id" = ? AND "namespaces"."type" 
= ? LIMIT ?
Full query 		275,591
145.05/sec
1.00/call
3.49%		 8,204.57 ms
4.318 ms/sec
0.030 ms/call
1.08%		 216,355
113.87/sec
0.79/call
0.79%		 1,102,908 blks
580.49 blks/sec
4.00 blks/call
0.33%		 7 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.10 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		280,221
147.49/sec
1.00/call
3.55%		 8,133.13 ms
4.281 ms/sec
0.029 ms/call
1.08%		 276,116
145.33/sec
0.99/call
1.01%		 1,484,588 blks
781.37 blks/sec
5.30 blks/call
0.45%		 3,615 blks
1.90 blks/sec
0.01 blks/call
2.05%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 302.87 ms
0.159 ms/sec
0.001 ms/call
1.58%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(2418469088)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."id" = ? LIMIT 
?
Full query 		267,880
140.99/sec
1.00/call
3.39%		 7,440.03 ms
3.916 ms/sec
0.028 ms/call
0.98%		 267,880
140.99/sec
1.00/call
0.98%		 1,072,050 blks
564.24 blks/sec
4.00 blks/call
0.32%		 5 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.12 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(1543342767)		 SELECT  "project_features".* FROM 
"project_features" WHERE "project_features"."project_id" 
= ? LIMIT ?
Full query 		255,793
134.63/sec
1.00/call
3.24%		 7,070.30 ms
3.721 ms/sec
0.028 ms/call
0.93%		 255,793
134.63/sec
1.00/call
0.94%		 1,021,695 blks
537.74 blks/sec
3.99 blks/call
0.31%		 1,956 blks
1.03 blks/sec
0.01 blks/call
1.11%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 95.43 ms
0.050 ms/sec
0.000 ms/call
0.50%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2747454572)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND (project_mirror_data.next_execution_timestamp 
- project_mirror_data.last_update_at) 
<= ?::interval
AND project_mirror_data.last_update_at 
< NOW() - ?::interval
;
Full query 		127
0.07/sec
1.00/call
0.00%		 6,708.44 ms
3.531 ms/sec
52.822 ms/call
0.89%		 127
0.07/sec
1.00/call
0.00%		 3,882,709 blks
2.05K blks/sec
30.58K blks/call
1.16%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.105)

Start: 2020-01-05T13:05:36.33304+00:00
End: 2020-01-05T13:37:43.269132+00:00
Period seconds: 1926.93609
Period age: 00:32:06.936092

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(1032929533)		 SELECT  "projects".* FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?
Full query 		76
0.04/sec
1.00/call
0.00%		 131,256.41 ms
68.117 ms/sec
1727.058 ms/call
15.92%		 5,116
2.65/sec
67.32/call
0.02%		 48,429,427 blks
25.14K blks/sec
637.23K blks/call
14.21%		 3 blks
0.00 blks/sec
0.04 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.12 ms
0.000 ms/sec
0.002 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(1687354761)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		354
0.18/sec
1.00/call
0.00%		 114,266.14 ms
59.299 ms/sec
322.786 ms/call
13.86%		 1,649
0.86/sec
4.66/call
0.01%		 79,956,698 blks
41.50K blks/sec
225.87K blks/call
23.46%		 33 blks
0.02 blks/sec
0.09 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2.42 ms
0.001 ms/sec
0.007 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(116237109)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		337
0.17/sec
1.00/call
0.00%		 107,564.63 ms
55.822 ms/sec
319.183 ms/call
13.05%		 1,861
0.97/sec
5.52/call
0.01%		 76,048,451 blks
39.47K blks/sec
225.67K blks/call
22.31%		 29 blks
0.02 blks/sec
0.09 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3.12 ms
0.002 ms/sec
0.009 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1134924411)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  projects.id FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		38
0.02/sec
1.00/call
0.00%		 60,340.22 ms
31.314 ms/sec
1587.900 ms/call
7.32%		 0
0.00/sec
0.00/call
0.00%		 24,215,331 blks
12.57K blks/sec
637.25K blks/call
7.10%		 2 blks
0.00 blks/sec
0.05 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.04 ms
0.000 ms/sec
0.001 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(322652387)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  "projects"."namespace_id" 
FROM "projects" WHERE (EXISTS (
SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		32
0.02/sec
1.00/call
0.00%		 56,994.76 ms
29.578 ms/sec
1781.086 ms/call
6.91%		 0
0.00/sec
0.00/call
0.00%		 24,212,695 blks
12.57K blks/sec
756.65K blks/call
7.10%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		1,157
0.60/sec
1.00/call
0.01%		 24,270.61 ms
12.595 ms/sec
20.977 ms/call
2.94%		 1,157
0.60/sec
1.00/call
0.00%		 6,782,864 blks
3.53K blks/sec
5.87K blks/call
1.99%		 1,375 blks
0.71 blks/sec
1.19 blks/call
0.67%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 52.66 ms
0.027 ms/sec
0.046 ms/call
0.24%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(1488973140)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		310,942
161.37/sec
1.00/call
3.83%		 20,913.98 ms
10.853 ms/sec
0.067 ms/call
2.54%		 302,459
156.96/sec
0.97/call
1.08%		 2,761,412 blks
1.44K blks/sec
8.88 blks/call
0.81%		 2,305 blks
1.20 blks/sec
0.01 blks/call
1.13%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 200.33 ms
0.104 ms/sec
0.001 ms/call
0.90%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		3,117
1.62/sec
1.00/call
0.04%		 20,526.38 ms
10.652 ms/sec
6.585 ms/call
2.49%		 332
0.17/sec
0.11/call
0.00%		 7,632,598 blks
3.97K blks/sec
2.45K blks/call
2.24%		 9,381 blks
4.87 blks/sec
3.01 blks/call
4.58%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 439.65 ms
0.228 ms/sec
0.141 ms/call
1.97%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		128
0.07/sec
1.00/call
0.00%		 16,555.37 ms
8.592 ms/sec
129.339 ms/call
2.01%		 128
0.07/sec
1.00/call
0.00%		 11,735,542 blks
6.10K blks/sec
91.69K blks/call
3.44%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		128
0.07/sec
1.00/call
0.00%		 15,353.14 ms
7.968 ms/sec
119.946 ms/call
1.86%		 128
0.07/sec
1.00/call
0.00%		 11,589,780 blks
6.02K blks/sec
90.55K blks/call
3.40%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(2329378283)		 SELECT  "users".* FROM "users" WHERE 
"users"."id" = ? LIMIT ?
Full query 		308,884
160.30/sec
1.00/call
3.80%		 13,000.76 ms
6.747 ms/sec
0.042 ms/call
1.58%		 308,849
160.28/sec
1.00/call
1.10%		 1,235,723 blks
641.29 blks/sec
4.00 blks/call
0.36%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(1189808325)		 SELECT  "users".* FROM "users" WHERE 
("users"."state" IN (?)) AND (ghost 
IS NOT TRUE) AND "users"."bot_type" 
IS NULL AND (("users"."name" ILIKE 
? OR "users"."username" ILIKE ?) 
OR "users"."email" = ?) ORDER BY 
CASE
  WHEN users.name = ? THEN 
?
  WHEN users.username = ? THEN 
?
  WHEN users.email = ? THEN ?
 ELSE ?
END
, "users"."name" ASC 
LIMIT ?
Full query 		114
0.06/sec
1.00/call
0.00%		 11,253.62 ms
5.840 ms/sec
98.716 ms/call
1.37%		 1,237
0.64/sec
10.85/call
0.00%		 945,405 blks
490.63 blks/sec
8.30K blks/call
0.28%		 533 blks
0.28 blks/sec
4.68 blks/call
0.26%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 51.16 ms
0.027 ms/sec
0.449 ms/call
0.23%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = 3503181 
AND "routes"."source_type" = 'Project' 
LIMIT 1
Full query 		309,604
160.67/sec
1.00/call
3.81%		 9,576.28 ms
4.970 ms/sec
0.031 ms/call
1.16%		 309,604
160.67/sec
1.00/call
1.11%		 1,548,926 blks
803.83 blks/sec
5.00 blks/call
0.45%		 65 blks
0.03 blks/sec
0.00 blks/call
0.03%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 2.13 ms
0.001 ms/sec
0.000 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(2856151481)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."type" IN (?) 
AND "namespaces"."id" = ? AND "namespaces"."type" 
= ? LIMIT ?
Full query 		279,533
145.07/sec
1.00/call
3.44%		 8,621.30 ms
4.474 ms/sec
0.031 ms/call
1.05%		 219,092
113.70/sec
0.78/call
0.78%		 1,118,700 blks
580.56 blks/sec
4.00 blks/call
0.33%		 8 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.14 ms
0.001 ms/sec
0.000 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		283,815
147.29/sec
1.00/call
3.50%		 8,603.27 ms
4.465 ms/sec
0.030 ms/call
1.04%		 279,779
145.19/sec
0.99/call
1.00%		 1,504,510 blks
780.78 blks/sec
5.30 blks/call
0.44%		 3,802 blks
1.97 blks/sec
0.01 blks/call
1.86%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 309.20 ms
0.160 ms/sec
0.001 ms/call
1.38%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(4133028576)		 SELECT  "events"."id" AS t0_r0, 
"events"."project_id" AS t0_r1,
 "events"."author_id" AS t0_r2,
 "events"."target_id" AS t0_r3,
 "events"."created_at" AS t0_r4,
 "events"."updated_at" AS t0_r5,
 "events"."action" AS t0_r6, "events"."target_type" 
AS t0_r7, "events"."group_id" AS 
t0_r8, "users"."id" AS t1_r0, "users"."email" 
AS t1_r1, "users"."encrypted_password" 
AS t1_r2, "users"."reset_password_token" 
AS t1_r3, "users"."reset_password_sent_at" 
AS t1_r4, "users"."remember_created_at" 
AS t1_r5, "users"."sign_in_count" 
AS t1_r6, "users"."current_sign_in_at" 
AS t1_r7, "users"."last_sign_in_at" 
AS t1_r8, "users"."current_sign_in_ip" 
AS t1_r9, "users"."last_sign_in_ip" 
AS t1_r10, "users"."created_at" 
AS t1_r11, "users"."updated_at" 
AS t1_r12, "users"."name" AS t1_r13,
 "users"."admin" AS t1_r14, "users"."projects_limit" 
AS t1_r15, "users"."skype" AS t1_r16,
 "users"."linkedin" AS t1_r17, 
"users"."twitter" AS t1_r18, "users"."bio" 
AS t1_r19, "users"."failed_attempts" 
AS t1_r20, "users"."locked_at" 
...
Full query 		254
0.13/sec
1.00/call
0.00%		 8,270.51 ms
4.292 ms/sec
32.561 ms/call
1.00%		 3,635
1.89/sec
14.31/call
0.01%		 4,732,957 blks
2.46K blks/sec
18.64K blks/call
1.39%		 35,089 blks
18.21 blks/sec
138.15 blks/call
17.14%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3,406.83 ms
1.768 ms/sec
13.413 ms/call
15.23%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(600568018)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_builds" WHERE "ci_builds"."type" 
IN (?) AND "ci_builds"."project_id" 
= ? AND "ci_builds"."status" = 
? LIMIT ?) subquery_for_count
Full query 		141
0.07/sec
1.00/call
0.00%		 8,072.69 ms
4.189 ms/sec
57.253 ms/call
0.98%		 141
0.07/sec
1.00/call
0.00%		 2,233,540 blks
1.16K blks/sec
15.85K blks/call
0.66%		 3,025 blks
1.57 blks/sec
21.45 blks/call
1.48%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 33.70 ms
0.017 ms/sec
0.239 ms/call
0.15%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(2418469088)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."id" = ? LIMIT 
?
Full query 		271,225
140.75/sec
1.00/call
3.34%		 7,762.92 ms
4.029 ms/sec
0.029 ms/call
0.94%		 271,225
140.75/sec
1.00/call
0.97%		 1,085,309 blks
563.23 blks/sec
4.00 blks/call
0.32%		 1 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.01 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(1543342767)		 SELECT  "project_features".* FROM 
"project_features" WHERE "project_features"."project_id" 
= ? LIMIT ?
Full query 		260,193
135.03/sec
1.00/call
3.20%		 7,571.60 ms
3.929 ms/sec
0.029 ms/call
0.92%		 260,193
135.03/sec
1.00/call
0.93%		 1,039,133 blks
539.27 blks/sec
3.99 blks/call
0.30%		 2,153 blks
1.12 blks/sec
0.01 blks/call
1.05%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 110.17 ms
0.057 ms/sec
0.000 ms/call
0.49%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2364908498)		 SELECT relnamespace::regnamespace 
as schemaname,
       relname as 
relname,
       pg_total_relation_size(
oid) bytes
  FROM pg_class
 WHERE 
relkind = ?;

Full query 		257
0.13/sec
1.00/call
0.00%		 6,939.57 ms
3.601 ms/sec
27.002 ms/call
0.84%		 95,090
49.35/sec
370.00/call
0.34%		 26,591 blks
13.80 blks/sec
103.47 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.107)

Start: 2020-01-05T13:05:38.544364+00:00
End: 2020-01-05T13:38:57.033548+00:00
Period seconds: 1998.48919
Period age: 00:33:18.489184

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(1032929533)		 SELECT  "projects".* FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?
Full query 		52
0.03/sec
1.00/call
0.00%		 117,464.73 ms
58.777 ms/sec
2258.937 ms/call
14.62%		 4,612
2.31/sec
88.69/call
0.02%		 49,803,956 blks
24.93K blks/sec
957.77K blks/call
13.83%		 93 blks
0.05 blks/sec
1.79 blks/call
0.04%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.02 ms
0.001 ms/sec
0.020 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(116237109)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		347
0.17/sec
1.00/call
0.01%		 105,986.10 ms
53.033 ms/sec
305.435 ms/call
13.19%		 847
0.42/sec
2.44/call
0.00%		 78,282,962 blks
39.18K blks/sec
225.60K blks/call
21.73%		 28 blks
0.01 blks/sec
0.08 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.42 ms
0.001 ms/sec
0.004 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(1687354761)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		335
0.17/sec
1.00/call
0.01%		 95,258.62 ms
47.665 ms/sec
284.354 ms/call
11.86%		 2,009
1.01/sec
6.00/call
0.01%		 75,697,905 blks
37.88K blks/sec
225.97K blks/call
21.02%		 33 blks
0.02 blks/sec
0.10 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.59 ms
0.000 ms/sec
0.002 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(322652387)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  "projects"."namespace_id" 
FROM "projects" WHERE (EXISTS (
SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		25
0.01/sec
1.00/call
0.00%		 58,652.96 ms
29.349 ms/sec
2346.118 ms/call
7.30%		 0
0.00/sec
0.00/call
0.00%		 24,901,946 blks
12.47K blks/sec
996.08K blks/call
6.91%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(1134924411)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  projects.id FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		26
0.01/sec
1.00/call
0.00%		 56,850.47 ms
28.447 ms/sec
2186.557 ms/call
7.08%		 0
0.00/sec
0.00/call
0.00%		 24,902,545 blks
12.47K blks/sec
957.80K blks/call
6.91%		 2 blks
0.00 blks/sec
0.08 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.03 ms
0.000 ms/sec
0.001 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(1488973140)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		322,077
161.16/sec
1.00/call
5.23%		 19,829.77 ms
9.922 ms/sec
0.062 ms/call
2.47%		 313,259
156.75/sec
0.97/call
1.04%		 2,859,617 blks
1.44K blks/sec
8.88 blks/call
0.79%		 2,669 blks
1.34 blks/sec
0.01 blks/call
1.28%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 100.19 ms
0.050 ms/sec
0.000 ms/call
0.65%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		1,230
0.62/sec
1.00/call
0.02%		 19,133.48 ms
9.574 ms/sec
15.556 ms/call
2.38%		 1,230
0.62/sec
1.00/call
0.00%		 5,322,218 blks
2.67K blks/sec
4.33K blks/call
1.48%		 5,352 blks
2.68 blks/sec
4.35 blks/call
2.58%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 55.69 ms
0.028 ms/sec
0.045 ms/call
0.36%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		3,193
1.60/sec
1.00/call
0.05%		 18,855.44 ms
9.435 ms/sec
5.905 ms/call
2.35%		 318
0.16/sec
0.10/call
0.00%		 6,924,098 blks
3.47K blks/sec
2.17K blks/call
1.92%		 1,842 blks
0.92 blks/sec
0.58 blks/call
0.89%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 64.53 ms
0.032 ms/sec
0.020 ms/call
0.42%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= true
AND project_mirror_data.retry_count 
<= 14
AND (projects.visibility_level 
= 20 OR plans.name IN ('early_adopter',
 'bronze', 'silver', 'gold'))
AND 
project_mirror_data.status NOT 
IN ('scheduled', 'started')
AND 
project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		134
0.07/sec
1.00/call
0.00%		 16,424.39 ms
8.218 ms/sec
122.570 ms/call
2.04%		 134
0.07/sec
1.00/call
0.00%		 12,240,850 blks
6.13K blks/sec
91.35K blks/call
3.40%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		134
0.07/sec
1.00/call
0.00%		 15,279.22 ms
7.645 ms/sec
114.024 ms/call
1.90%		 134
0.07/sec
1.00/call
0.00%		 12,082,894 blks
6.05K blks/sec
90.18K blks/call
3.35%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(2329378283)		 SELECT  "users".* FROM "users" WHERE 
"users"."id" = ? LIMIT ?
Full query 		322,664
161.45/sec
1.00/call
5.24%		 12,842.88 ms
6.426 ms/sec
0.040 ms/call
1.60%		 322,610
161.43/sec
1.00/call
1.07%		 1,290,834 blks
645.90 blks/sec
4.00 blks/call
0.36%		 5 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.11 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(1189808325)		 SELECT  "users".* FROM "users" WHERE 
("users"."state" IN (?)) AND (ghost 
IS NOT TRUE) AND "users"."bot_type" 
IS NULL AND (("users"."name" ILIKE 
? OR "users"."username" ILIKE ?) 
OR "users"."email" = ?) ORDER BY 
CASE
  WHEN users.name = ? THEN 
?
  WHEN users.username = ? THEN 
?
  WHEN users.email = ? THEN ?
 ELSE ?
END
, "users"."name" ASC 
LIMIT ?
Full query 		123
0.06/sec
1.00/call
0.00%		 12,088.55 ms
6.049 ms/sec
98.281 ms/call
1.50%		 1,375
0.69/sec
11.18/call
0.00%		 1,100,459 blks
550.65 blks/sec
8.95K blks/call
0.31%		 674 blks
0.34 blks/sec
5.48 blks/call
0.32%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 9.14 ms
0.005 ms/sec
0.074 ms/call
0.06%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(2364908498)		 SELECT relnamespace::regnamespace 
as schemaname,
       relname as 
relname,
       pg_total_relation_size(
oid) bytes
  FROM pg_class
 WHERE 
relkind = ?;

Full query 		267
0.13/sec
1.00/call
0.00%		 10,861.35 ms
5.435 ms/sec
40.679 ms/call
1.35%		 98,790
49.43/sec
370.00/call
0.33%		 27,515 blks
13.77 blks/sec
103.05 blks/call
0.01%		 1 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.02 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(2033340823)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL)
Full query 		35,588
17.81/sec
1.00/call
0.58%		 10,412.76 ms
5.210 ms/sec
0.293 ms/call
1.30%		 502,568
251.47/sec
14.12/call
1.67%		 833,252 blks
416.94 blks/sec
23.41 blks/call
0.23%		 9,212 blks
4.61 blks/sec
0.26 blks/call
4.43%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,735.49 ms
0.868 ms/sec
0.049 ms/call
11.30%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		322,579
161.41/sec
1.00/call
5.24%		 9,025.61 ms
4.516 ms/sec
0.028 ms/call
1.12%		 322,579
161.41/sec
1.00/call
1.07%		 1,613,935 blks
807.58 blks/sec
5.00 blks/call
0.45%		 6 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.13 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(600568018)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "ci_builds" WHERE "ci_builds"."type" 
IN (?) AND "ci_builds"."project_id" 
= ? AND "ci_builds"."status" = 
? LIMIT ?) subquery_for_count
Full query 		132
0.07/sec
1.00/call
0.00%		 8,209.27 ms
4.108 ms/sec
62.191 ms/call
1.02%		 132
0.07/sec
1.00/call
0.00%		 2,623,898 blks
1.32K blks/sec
19.88K blks/call
0.73%		 4,505 blks
2.25 blks/sec
34.13 blks/call
2.17%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 40.77 ms
0.020 ms/sec
0.309 ms/call
0.27%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(2856151481)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."type" IN (?) 
AND "namespaces"."id" = ? AND "namespaces"."type" 
= ? LIMIT ?
Full query 		289,933
145.08/sec
1.00/call
4.71%		 8,137.50 ms
4.072 ms/sec
0.028 ms/call
1.01%		 227,172
113.67/sec
0.78/call
0.76%		 1,160,342 blks
580.61 blks/sec
4.00 blks/call
0.32%		 7 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.19 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(4133028576)		 SELECT  "events"."id" AS t0_r0, 
"events"."project_id" AS t0_r1,
 "events"."author_id" AS t0_r2,
 "events"."target_id" AS t0_r3,
 "events"."created_at" AS t0_r4,
 "events"."updated_at" AS t0_r5,
 "events"."action" AS t0_r6, "events"."target_type" 
AS t0_r7, "events"."group_id" AS 
t0_r8, "users"."id" AS t1_r0, "users"."email" 
AS t1_r1, "users"."encrypted_password" 
AS t1_r2, "users"."reset_password_token" 
AS t1_r3, "users"."reset_password_sent_at" 
AS t1_r4, "users"."remember_created_at" 
AS t1_r5, "users"."sign_in_count" 
AS t1_r6, "users"."current_sign_in_at" 
AS t1_r7, "users"."last_sign_in_at" 
AS t1_r8, "users"."current_sign_in_ip" 
AS t1_r9, "users"."last_sign_in_ip" 
AS t1_r10, "users"."created_at" 
AS t1_r11, "users"."updated_at" 
AS t1_r12, "users"."name" AS t1_r13,
 "users"."admin" AS t1_r14, "users"."projects_limit" 
AS t1_r15, "users"."skype" AS t1_r16,
 "users"."linkedin" AS t1_r17, 
"users"."twitter" AS t1_r18, "users"."bio" 
AS t1_r19, "users"."failed_attempts" 
AS t1_r20, "users"."locked_at" 
...
Full query 		251
0.13/sec
1.00/call
0.00%		 8,026.52 ms
4.016 ms/sec
31.978 ms/call
1.00%		 3,432
1.72/sec
13.67/call
0.01%		 6,788,178 blks
3.40K blks/sec
27.05K blks/call
1.88%		 46,150 blks
23.09 blks/sec
183.86 blks/call
22.21%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,216.22 ms
0.609 ms/sec
4.846 ms/call
7.92%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		297,500
148.86/sec
1.00/call
4.83%		 8,009.28 ms
4.008 ms/sec
0.027 ms/call
1.00%		 293,074
146.65/sec
0.99/call
0.97%		 1,576,308 blks
788.75 blks/sec
5.30 blks/call
0.44%		 4,268 blks
2.14 blks/sec
0.01 blks/call
2.05%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 168.64 ms
0.084 ms/sec
0.001 ms/call
1.10%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2418469088)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."id" = ? LIMIT 
?
Full query 		282,369
141.29/sec
1.00/call
4.58%		 7,413.33 ms
3.709 ms/sec
0.026 ms/call
0.92%		 282,369
141.29/sec
1.00/call
0.94%		 1,129,911 blks
565.38 blks/sec
4.00 blks/call
0.31%		 1 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.03 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.108)

Start: 2020-01-05T13:05:39.654466+00:00
End: 2020-01-05T13:39:27.213604+00:00
Period seconds: 2027.55913
Period age: 00:33:47.559138

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(1032929533)		 SELECT  "projects".* FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?
Full query 		54
0.03/sec
1.00/call
0.00%		 115,417.41 ms
56.924 ms/sec
2137.359 ms/call
14.33%		 4,494
2.22/sec
83.22/call
0.01%		 47,624,676 blks
23.49K blks/sec
881.94K blks/call
13.31%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(1687354761)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		375
0.18/sec
1.00/call
0.00%		 108,612.46 ms
53.568 ms/sec
289.633 ms/call
13.49%		 1,576
0.78/sec
4.20/call
0.01%		 84,612,902 blks
41.74K blks/sec
225.64K blks/call
23.65%		 46 blks
0.02 blks/sec
0.12 blks/call
0.02%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.64 ms
0.001 ms/sec
0.004 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(116237109)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		360
0.18/sec
1.00/call
0.00%		 104,243.60 ms
51.413 ms/sec
289.566 ms/call
12.94%		 1,943
0.96/sec
5.40/call
0.01%		 81,064,910 blks
39.99K blks/sec
225.19K blks/call
22.65%		 19 blks
0.01 blks/sec
0.05 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.33 ms
0.000 ms/sec
0.001 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(1134924411)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  projects.id FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		27
0.01/sec
1.00/call
0.00%		 53,195.11 ms
26.236 ms/sec
1970.189 ms/call
6.60%		 0
0.00/sec
0.00/call
0.00%		 23,812,816 blks
11.75K blks/sec
881.96K blks/call
6.65%		 3 blks
0.00 blks/sec
0.11 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.04 ms
0.000 ms/sec
0.002 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(322652387)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  "projects"."namespace_id" 
FROM "projects" WHERE (EXISTS (
SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		25
0.01/sec
1.00/call
0.00%		 51,329.66 ms
25.316 ms/sec
2053.186 ms/call
6.37%		 0
0.00/sec
0.00/call
0.00%		 22,661,642 blks
11.18K blks/sec
906.47K blks/call
6.33%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(1488973140)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		326,327
160.95/sec
1.00/call
3.85%		 20,133.38 ms
9.930 ms/sec
0.062 ms/call
2.50%		 317,520
156.60/sec
0.97/call
1.05%		 2,898,271 blks
1.43K blks/sec
8.88 blks/call
0.81%		 2,425 blks
1.20 blks/sec
0.01 blks/call
0.87%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 93.87 ms
0.046 ms/sec
0.000 ms/call
0.57%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(1189808325)		 SELECT  "users".* FROM "users" WHERE 
("users"."state" IN (?)) AND (ghost 
IS NOT TRUE) AND "users"."bot_type" 
IS NULL AND (("users"."name" ILIKE 
? OR "users"."username" ILIKE ?) 
OR "users"."email" = ?) ORDER BY 
CASE
  WHEN users.name = ? THEN 
?
  WHEN users.username = ? THEN 
?
  WHEN users.email = ? THEN ?
 ELSE ?
END
, "users"."name" ASC 
LIMIT ?
Full query 		122
0.06/sec
1.00/call
0.00%		 18,887.05 ms
9.315 ms/sec
154.812 ms/call
2.34%		 1,250
0.62/sec
10.25/call
0.00%		 1,431,278 blks
705.91 blks/sec
11.74K blks/call
0.40%		 27,029 blks
13.33 blks/sec
221.55 blks/call
9.70%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 237.57 ms
0.117 ms/sec
1.947 ms/call
1.44%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		3,215
1.59/sec
1.00/call
0.04%		 17,669.06 ms
8.714 ms/sec
5.496 ms/call
2.19%		 320
0.16/sec
0.10/call
0.00%		 7,792,603 blks
3.85K blks/sec
2.43K blks/call
2.18%		 4,877 blks
2.41 blks/sec
1.52 blks/call
1.75%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 122.78 ms
0.061 ms/sec
0.038 ms/call
0.74%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		134
0.07/sec
1.00/call
0.00%		 16,343.06 ms
8.060 ms/sec
121.963 ms/call
2.03%		 134
0.07/sec
1.00/call
0.00%		 12,117,602 blks
5.98K blks/sec
90.43K blks/call
3.39%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		134
0.07/sec
1.00/call
0.00%		 15,186.70 ms
7.490 ms/sec
113.334 ms/call
1.89%		 134
0.07/sec
1.00/call
0.00%		 11,965,795 blks
5.91K blks/sec
89.30K blks/call
3.34%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(2329378283)		 SELECT  "users".* FROM "users" WHERE 
"users"."id" = ? LIMIT ?
Full query 		327,544
161.55/sec
1.00/call
3.86%		 13,043.71 ms
6.433 ms/sec
0.040 ms/call
1.62%		 327,497
161.52/sec
1.00/call
1.08%		 1,310,326 blks
646.26 blks/sec
4.00 blks/call
0.37%		 1 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.02 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		1,194
0.59/sec
1.00/call
0.01%		 12,271.42 ms
6.052 ms/sec
10.278 ms/call
1.52%		 1,194
0.59/sec
1.00/call
0.00%		 3,401,273 blks
1.68K blks/sec
2.85K blks/call
0.95%		 9,910 blks
4.89 blks/sec
8.30 blks/call
3.56%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 97.69 ms
0.048 ms/sec
0.082 ms/call
0.59%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(4133028576)		 SELECT  "events"."id" AS t0_r0, 
"events"."project_id" AS t0_r1,
 "events"."author_id" AS t0_r2,
 "events"."target_id" AS t0_r3,
 "events"."created_at" AS t0_r4,
 "events"."updated_at" AS t0_r5,
 "events"."action" AS t0_r6, "events"."target_type" 
AS t0_r7, "events"."group_id" AS 
t0_r8, "users"."id" AS t1_r0, "users"."email" 
AS t1_r1, "users"."encrypted_password" 
AS t1_r2, "users"."reset_password_token" 
AS t1_r3, "users"."reset_password_sent_at" 
AS t1_r4, "users"."remember_created_at" 
AS t1_r5, "users"."sign_in_count" 
AS t1_r6, "users"."current_sign_in_at" 
AS t1_r7, "users"."last_sign_in_at" 
AS t1_r8, "users"."current_sign_in_ip" 
AS t1_r9, "users"."last_sign_in_ip" 
AS t1_r10, "users"."created_at" 
AS t1_r11, "users"."updated_at" 
AS t1_r12, "users"."name" AS t1_r13,
 "users"."admin" AS t1_r14, "users"."projects_limit" 
AS t1_r15, "users"."skype" AS t1_r16,
 "users"."linkedin" AS t1_r17, 
"users"."twitter" AS t1_r18, "users"."bio" 
AS t1_r19, "users"."failed_attempts" 
AS t1_r20, "users"."locked_at" 
...
Full query 		258
0.13/sec
1.00/call
0.00%		 11,600.33 ms
5.721 ms/sec
44.962 ms/call
1.44%		 3,642
1.80/sec
14.12/call
0.01%		 8,180,130 blks
4.04K blks/sec
31.71K blks/call
2.29%		 116,507 blks
57.46 blks/sec
451.58 blks/call
41.80%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 3,148.65 ms
1.553 ms/sec
12.204 ms/call
19.03%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(2364908498)		 SELECT relnamespace::regnamespace 
as schemaname,
       relname as 
relname,
       pg_total_relation_size(
oid) bytes
  FROM pg_class
 WHERE 
relkind = ?;

Full query 		270
0.13/sec
1.00/call
0.00%		 10,866.68 ms
5.359 ms/sec
40.247 ms/call
1.35%		 99,900
49.27/sec
370.00/call
0.33%		 27,891 blks
13.76 blks/sec
103.30 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(2033340823)		 SELECT "ci_builds".* FROM "ci_builds" 
WHERE "ci_builds"."commit_id" = 
? AND ("ci_builds"."retried" = 
? OR "ci_builds"."retried" IS NULL)
Full query 		36,144
17.83/sec
1.00/call
0.43%		 10,538.68 ms
5.198 ms/sec
0.292 ms/call
1.31%		 492,543
242.92/sec
13.63/call
1.63%		 806,825 blks
397.93 blks/sec
22.32 blks/call
0.23%		 9,167 blks
4.52 blks/sec
0.25 blks/call
3.29%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,996.42 ms
0.985 ms/sec
0.055 ms/call
12.06%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		330,857
163.18/sec
1.00/call
3.90%		 9,223.16 ms
4.549 ms/sec
0.028 ms/call
1.15%		 330,857
163.18/sec
1.00/call
1.10%		 1,655,297 blks
816.40 blks/sec
5.00 blks/call
0.46%		 17 blks
0.01 blks/sec
0.00 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.31 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(1074971172)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "issues" INNER JOIN 
"projects" ON "projects"."id" = 
"issues"."project_id" LEFT JOIN 
project_features ON projects.id 
= project_features.project_id WHERE 
"issues"."confidential" = ? AND 
(EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id) AND (project_authorizations.access_level 
>= ?)) OR projects.visibility_level 
IN (?,?)) AND ("project_features"."issues_access_level" 
IS NULL OR "project_features"."issues_access_level" 
IN (?,?) OR ("project_features"."issues_access_level" 
= ? AND EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id) AND (project_authorizations.access_level 
>= ?)))) AND ("issues"."title" 
ILIKE ? OR "issues"."description" 
ILIKE ?) LIMIT ?) subquery_for_count
Full query 		2
0.00/sec
1.00/call
0.00%		 9,129.74 ms
4.503 ms/sec
4564.869 ms/call
1.13%		 2
0.00/sec
1.00/call
0.00%		 154,232 blks
76.07 blks/sec
77.12K blks/call
0.04%		 7,792 blks
3.84 blks/sec
3.90K blks/call
2.80%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 548.14 ms
0.270 ms/sec
274.070 ms/call
3.31%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(2856151481)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."type" IN (?) 
AND "namespaces"."id" = ? AND "namespaces"."type" 
= ? LIMIT ?
Full query 		294,971
145.48/sec
1.00/call
3.48%		 8,329.47 ms
4.108 ms/sec
0.028 ms/call
1.03%		 231,459
114.16/sec
0.78/call
0.77%		 1,180,459 blks
582.21 blks/sec
4.00 blks/call
0.33%		 8 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.16 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		308,465
152.14/sec
1.00/call
3.64%		 8,296.94 ms
4.092 ms/sec
0.027 ms/call
1.03%		 304,056
149.96/sec
0.99/call
1.01%		 1,635,470 blks
806.62 blks/sec
5.30 blks/call
0.46%		 4,031 blks
1.99 blks/sec
0.01 blks/call
1.45%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 197.77 ms
0.098 ms/sec
0.001 ms/call
1.20%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2418469088)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."id" = ? LIMIT 
?
Full query 		285,771
140.94/sec
1.00/call
3.37%		 7,520.20 ms
3.709 ms/sec
0.026 ms/call
0.93%		 285,771
140.94/sec
1.00/call
0.95%		 1,143,584 blks
564.02 blks/sec
4.00 blks/call
0.32%		 3 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.06 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Replica (10.220.16.109)

Start: 2020-01-05T13:05:40.779676+00:00
End: 2020-01-05T13:39:56.951447+00:00
Period seconds: 2056.17177
Period age: 00:34:16.171771

Top 50 rows

#
(query id)		 Query Calls ▼ Total time Rows shared_blks_hit shared_blks_read shared_blks_dirtied shared_blks_written blk_read_time blk_write_time kcache_reads kcache_writes kcache_user_time_ms kcache_system_time
1
(1032929533)		 SELECT  "projects".* FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?
Full query 		63
0.03/sec
1.00/call
0.00%		 145,515.45 ms
70.770 ms/sec
2309.769 ms/call
16.25%		 5,723
2.78/sec
90.84/call
0.02%		 60,550,557 blks
29.45K blks/sec
961.12K blks/call
15.67%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
2
(116237109)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		369
0.18/sec
1.00/call
0.00%		 113,710.51 ms
55.302 ms/sec
308.159 ms/call
12.70%		 2,203
1.07/sec
5.97/call
0.01%		 83,216,486 blks
40.48K blks/sec
225.52K blks/call
21.54%		 24 blks
0.01 blks/sec
0.07 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.49 ms
0.000 ms/sec
0.001 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
3
(1687354761)		 SELECT "ci_builds".* FROM "ci_builds" 
INNER JOIN "projects" ON "projects"."id" 
= "ci_builds"."project_id" LEFT 
JOIN project_features ON ci_builds.project_id 
= project_features.project_id LEFT 
JOIN (SELECT "ci_builds"."project_id",
 count(*) AS running_builds FROM 
"ci_builds" WHERE "ci_builds"."type" 
IN (?) AND ("ci_builds"."status" 
IN (?)) AND "ci_builds"."runner_id" 
IN (SELECT "ci_runners"."id" FROM 
"ci_runners" WHERE "ci_runners"."runner_type" 
= ?) GROUP BY "ci_builds"."project_id") 
AS project_builds ON ci_builds.project_id=project_builds.project_id 
WHERE "ci_builds"."type" IN (?) 
AND ("ci_builds"."status" IN (?)) 
AND "ci_builds"."runner_id" IS 
NULL AND "projects"."shared_runners_enabled" 
= ? AND "projects"."pending_delete" 
= ? AND (project_features.builds_access_level 
IS NULL or project_features.builds_access_level 
> ?) AND (projects.visibility_level=? 
OR (WITH RECURSIVE "base_and_ancestors" 
AS ((SELECT "namespaces".* FROM 
"namespaces" WHERE (namespaces.id 
= projects.namespace_id))
...
Full query 		391
0.19/sec
1.00/call
0.00%		 112,604.30 ms
54.764 ms/sec
287.991 ms/call
12.57%		 2,018
0.98/sec
5.16/call
0.01%		 88,169,785 blks
42.89K blks/sec
225.50K blks/call
22.82%		 28 blks
0.01 blks/sec
0.07 blks/call
0.01%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1.54 ms
0.001 ms/sec
0.004 ms/call
0.01%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
4
(322652387)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  "projects"."namespace_id" 
FROM "projects" WHERE (EXISTS (
SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		30
0.01/sec
1.00/call
0.00%		 71,413.61 ms
34.731 ms/sec
2380.454 ms/call
7.97%		 0
0.00/sec
0.00/call
0.00%		 29,671,600 blks
14.44K blks/sec
989.06K blks/call
7.68%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
5
(1134924411)		 SELECT "members".* FROM "members" 
WHERE "members"."source_type" = 
? AND "members"."type" IN (?) AND 
"members"."user_id" = ? AND "members"."requested_at" 
IS NULL AND "members"."source_id" 
IN (SELECT  projects.id FROM "projects" 
WHERE (EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?)) AND "projects"."pending_delete" 
= ? ORDER BY "projects"."created_at" 
DESC, "projects"."id" DESC LIMIT 
? OFFSET ?)
Full query 		32
0.02/sec
1.00/call
0.00%		 69,042.35 ms
33.578 ms/sec
2157.573 ms/call
7.71%		 0
0.00/sec
0.00/call
0.00%		 30,773,900 blks
14.97K blks/sec
961.69K blks/call
7.97%		 3 blks
0.00 blks/sec
0.09 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.17 ms
0.000 ms/sec
0.005 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
6
(1682658247)		 SELECT "environments".* FROM "environments" 
WHERE "environments"."project_id" 
= ? AND ("environments"."state" 
IN (?)) AND "environments"."id" 
IN (SELECT "deployments"."environment_id" 
FROM "deployments" WHERE "deployments"."project_id" 
= ? AND "deployments"."sha" = ? 
GROUP BY "deployments"."environment_id") 
ORDER BY (SELECT MAX("deployments"."id") 
FROM "deployments" WHERE "deployments"."environment_id" 
= "environments"."id") DESC NULLS 
LAST
Full query 		3,382
1.64/sec
1.00/call
0.04%		 21,277.36 ms
10.348 ms/sec
6.291 ms/call
2.38%		 379
0.18/sec
0.11/call
0.00%		 7,980,193 blks
3.89K blks/sec
2.36K blks/call
2.07%		 5,810 blks
2.83 blks/sec
1.72 blks/call
1.86%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 142.62 ms
0.069 ms/sec
0.042 ms/call
0.78%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
7
(1189808325)		 SELECT  "users".* FROM "users" WHERE 
("users"."state" IN (?)) AND (ghost 
IS NOT TRUE) AND "users"."bot_type" 
IS NULL AND (("users"."name" ILIKE 
? OR "users"."username" ILIKE ?) 
OR "users"."email" = ?) ORDER BY 
CASE
  WHEN users.name = ? THEN 
?
  WHEN users.username = ? THEN 
?
  WHEN users.email = ? THEN ?
 ELSE ?
END
, "users"."name" ASC 
LIMIT ?
Full query 		122
0.06/sec
1.00/call
0.00%		 21,140.52 ms
10.281 ms/sec
173.283 ms/call
2.36%		 1,385
0.67/sec
11.35/call
0.00%		 1,580,773 blks
768.79 blks/sec
12.96K blks/call
0.41%		 25,591 blks
12.45 blks/sec
209.76 blks/call
8.20%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 220.70 ms
0.107 ms/sec
1.809 ms/call
1.20%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
8
(1488973140)		 SELECT  "projects"."id" AS t0_r0,
 "projects"."name" AS t0_r1, "projects"."path" 
AS t0_r2, "projects"."description" 
AS t0_r3, "projects"."created_at" 
AS t0_r4, "projects"."updated_at" 
AS t0_r5, "projects"."creator_id" 
AS t0_r6, "projects"."namespace_id" 
AS t0_r7, "projects"."last_activity_at" 
AS t0_r8, "projects"."import_url" 
AS t0_r9, "projects"."visibility_level" 
AS t0_r10, "projects"."archived" 
AS t0_r11, "projects"."merge_requests_template" 
AS t0_r12, "projects"."star_count" 
AS t0_r13, "projects"."merge_requests_rebase_enabled" 
AS t0_r14, "projects"."import_type" 
AS t0_r15, "projects"."import_source" 
AS t0_r16, "projects"."avatar" 
AS t0_r17, "projects"."approvals_before_merge" 
AS t0_r18, "projects"."reset_approvals_on_push" 
AS t0_r19, "projects"."merge_requests_ff_only_enabled" 
AS t0_r20, "projects"."issues_template" 
AS t0_r21, "projects"."mirror" 
AS t0_r22, "projects"."mirror_user_id" 
AS t0_r23, "projects"."shared_runners_enabled" 
AS t0_r24, "projects"."runners_token" 
AS t0_r25, "...
Full query 		331,283
161.12/sec
1.00/call
3.87%		 20,802.31 ms
10.117 ms/sec
0.063 ms/call
2.32%		 322,372
156.78/sec
0.97/call
1.10%		 2,942,458 blks
1.44K blks/sec
8.88 blks/call
0.76%		 2,385 blks
1.16 blks/sec
0.01 blks/call
0.76%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 111.52 ms
0.054 ms/sec
0.000 ms/call
0.61%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
9
(1074971172)		 SELECT COUNT(*) FROM (SELECT  ? 
AS one FROM "issues" INNER JOIN 
"projects" ON "projects"."id" = 
"issues"."project_id" LEFT JOIN 
project_features ON projects.id 
= project_features.project_id WHERE 
"issues"."confidential" = ? AND 
(EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id) AND (project_authorizations.access_level 
>= ?)) OR projects.visibility_level 
IN (?,?)) AND ("project_features"."issues_access_level" 
IS NULL OR "project_features"."issues_access_level" 
IN (?,?) OR ("project_features"."issues_access_level" 
= ? AND EXISTS (SELECT ? FROM "project_authorizations" 
WHERE "project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id) AND (project_authorizations.access_level 
>= ?)))) AND ("issues"."title" 
ILIKE ? OR "issues"."description" 
ILIKE ?) LIMIT ?) subquery_for_count
Full query 		5
0.00/sec
1.00/call
0.00%		 20,242.42 ms
9.845 ms/sec
4048.484 ms/call
2.26%		 5
0.00/sec
1.00/call
0.00%		 355,037 blks
172.67 blks/sec
71.01K blks/call
0.09%		 23,981 blks
11.66 blks/sec
4.80K blks/call
7.68%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 1,434.87 ms
0.698 ms/sec
286.975 ms/call
7.82%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
10
(2576098393)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW()
;
Full query 		137
0.07/sec
1.00/call
0.00%		 17,815.74 ms
8.665 ms/sec
130.042 ms/call
1.99%		 137
0.07/sec
1.00/call
0.00%		 12,527,048 blks
6.10K blks/sec
91.44K blks/call
3.24%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
11
(746170934)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND project_mirror_data.next_execution_timestamp 
<= NOW() - ?::interval
;
Full query 		137
0.07/sec
1.00/call
0.00%		 16,502.47 ms
8.026 ms/sec
120.456 ms/call
1.84%		 137
0.07/sec
1.00/call
0.00%		 12,372,332 blks
6.02K blks/sec
90.31K blks/call
3.20%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
12
(2329378283)		 SELECT  "users".* FROM "users" WHERE 
"users"."id" = ? LIMIT ?
Full query 		331,203
161.08/sec
1.00/call
3.87%		 13,307.27 ms
6.472 ms/sec
0.040 ms/call
1.49%		 331,154
161.05/sec
1.00/call
1.13%		 1,324,976 blks
644.39 blks/sec
4.00 blks/call
0.34%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
13
(3938502189)		 SELECT COUNT(DISTINCT "events"."id") 
FROM "events" INNER JOIN "projects" 
ON "projects"."id" = "events"."project_id" 
LEFT OUTER JOIN "project_features" 
ON "project_features"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"project_import_data" ON "project_import_data"."project_id" 
= "projects"."id" LEFT OUTER JOIN 
"namespaces" ON "namespaces"."id" 
= "projects"."namespace_id" LEFT 
OUTER JOIN "users" ON "users"."id" 
= "events"."author_id" WHERE "events"."project_id" 
= ? AND (EXISTS (SELECT ? FROM 
"project_authorizations" WHERE 
"project_authorizations"."user_id" 
= ? AND (project_authorizations.project_id 
= projects.id)) OR projects.visibility_level 
IN (?,?))
Full query 		1,227
0.60/sec
1.00/call
0.01%		 13,130.53 ms
6.386 ms/sec
10.701 ms/call
1.47%		 1,227
0.60/sec
1.00/call
0.00%		 3,170,368 blks
1.55K blks/sec
2.59K blks/call
0.82%		 12,211 blks
5.94 blks/sec
9.95 blks/call
3.91%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 111.48 ms
0.054 ms/sec
0.091 ms/call
0.61%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
14
(4133028576)		 SELECT  "events"."id" AS t0_r0, 
"events"."project_id" AS t0_r1,
 "events"."author_id" AS t0_r2,
 "events"."target_id" AS t0_r3,
 "events"."created_at" AS t0_r4,
 "events"."updated_at" AS t0_r5,
 "events"."action" AS t0_r6, "events"."target_type" 
AS t0_r7, "events"."group_id" AS 
t0_r8, "users"."id" AS t1_r0, "users"."email" 
AS t1_r1, "users"."encrypted_password" 
AS t1_r2, "users"."reset_password_token" 
AS t1_r3, "users"."reset_password_sent_at" 
AS t1_r4, "users"."remember_created_at" 
AS t1_r5, "users"."sign_in_count" 
AS t1_r6, "users"."current_sign_in_at" 
AS t1_r7, "users"."last_sign_in_at" 
AS t1_r8, "users"."current_sign_in_ip" 
AS t1_r9, "users"."last_sign_in_ip" 
AS t1_r10, "users"."created_at" 
AS t1_r11, "users"."updated_at" 
AS t1_r12, "users"."name" AS t1_r13,
 "users"."admin" AS t1_r14, "users"."projects_limit" 
AS t1_r15, "users"."skype" AS t1_r16,
 "users"."linkedin" AS t1_r17, 
"users"."twitter" AS t1_r18, "users"."bio" 
AS t1_r19, "users"."failed_attempts" 
AS t1_r20, "users"."locked_at" 
...
Full query 		274
0.13/sec
1.00/call
0.00%		 12,981.86 ms
6.314 ms/sec
47.379 ms/call
1.45%		 3,987
1.94/sec
14.55/call
0.01%		 7,910,858 blks
3.85K blks/sec
28.88K blks/call
2.05%		 120,335 blks
58.52 blks/sec
439.18 blks/call
38.56%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 4,373.59 ms
2.127 ms/sec
15.962 ms/call
23.83%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
15
(2364908498)		 SELECT relnamespace::regnamespace 
as schemaname,
       relname as 
relname,
       pg_total_relation_size(
oid) bytes
  FROM pg_class
 WHERE 
relkind = ?;

Full query 		274
0.13/sec
1.00/call
0.00%		 11,054.02 ms
5.376 ms/sec
40.343 ms/call
1.23%		 101,380
49.31/sec
370.00/call
0.34%		 28,354 blks
13.79 blks/sec
103.48 blks/call
0.01%		 1 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.03 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
16
(1850364306)		 SELECT  "routes".* FROM "routes" 
WHERE "routes"."source_id" = ? 
AND "routes"."source_type" = ? 
LIMIT ?
Full query 		329,877
160.43/sec
1.00/call
3.86%		 9,528.88 ms
4.634 ms/sec
0.029 ms/call
1.06%		 329,877
160.43/sec
1.00/call
1.12%		 1,649,920 blks
802.42 blks/sec
5.00 blks/call
0.43%		 597 blks
0.29 blks/sec
0.00 blks/call
0.19%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 11.01 ms
0.005 ms/sec
0.000 ms/call
0.06%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
17
(2856151481)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."type" IN (?) 
AND "namespaces"."id" = ? AND "namespaces"."type" 
= ? LIMIT ?
Full query 		297,000
144.44/sec
1.00/call
3.47%		 8,507.27 ms
4.137 ms/sec
0.029 ms/call
0.95%		 232,304
112.98/sec
0.78/call
0.79%		 1,188,587 blks
578.06 blks/sec
4.00 blks/call
0.31%		 4 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.09 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
18
(1180769399)		 SELECT MAX("project_authorizations"."access_level") 
AS maximum_access_level, "project_authorizations"."user_id" 
AS project_authorizations_user_id 
FROM "project_authorizations" WHERE 
"project_authorizations"."project_id" 
= ? AND "project_authorizations"."user_id" 
= ? GROUP BY "project_authorizations"."user_id"
Full query 		303,386
147.55/sec
1.00/call
3.55%		 8,423.47 ms
4.097 ms/sec
0.028 ms/call
0.94%		 298,843
145.34/sec
0.99/call
1.02%		 1,608,618 blks
782.34 blks/sec
5.30 blks/call
0.42%		 3,743 blks
1.82 blks/sec
0.01 blks/call
1.20%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 180.41 ms
0.088 ms/sec
0.001 ms/call
0.98%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
19
(2418469088)		 SELECT  "namespaces".* FROM "namespaces" 
WHERE "namespaces"."id" = ? LIMIT 
?
Full query 		290,327
141.20/sec
1.00/call
3.39%		 7,742.08 ms
3.765 ms/sec
0.027 ms/call
0.86%		 290,327
141.20/sec
1.00/call
0.99%		 1,161,682 blks
564.97 blks/sec
4.00 blks/call
0.30%		 2 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.05 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%
20
(2747454572)		 SELECT COUNT(*) FROM projects INNER 
JOIN project_mirror_data ON project_mirror_data.project_id 
= projects.id
INNER JOIN namespaces 
ON projects.namespace_id = namespaces.id
LEFT 
JOIN plans ON namespaces.plan_id 
= plans.id
 WHERE projects.mirror 
= ?
AND project_mirror_data.retry_count 
<= ?
AND (projects.visibility_level 
= ? OR plans.name IN (?, ?, ?, 
?))
AND project_mirror_data.status 
NOT IN (?, ?)
AND (project_mirror_data.next_execution_timestamp 
- project_mirror_data.last_update_at) 
<= ?::interval
AND project_mirror_data.last_update_at 
< NOW() - ?::interval
;
Full query 		137
0.07/sec
1.00/call
0.00%		 7,496.61 ms
3.646 ms/sec
54.720 ms/call
0.84%		 137
0.07/sec
1.00/call
0.00%		 4,156,348 blks
2.03K blks/sec
30.34K blks/call
1.08%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0 blks
0.00 blks/sec
0.00 blks/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 bytes
0.00 bytes/sec
0.00 bytes/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%		 0.00 ms
0.000 ms/sec
0.000 ms/call
0.00%

Conclusions

Recommendations

All good, no recommendations here.

  Table of contents

L001 Table Sizes

Observations

Data collected: 2020-01-05 13:38:45 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

The list is limited to 20 items. Total: 363.

# Table Rows ▼ Total size Table size Index(es) Size TOAST Size
  ===== TOTAL ===== ~10B 4.59 TiB 2.45 TiB 1.24 TiB 926.03 GiB
1 merge_request_diff_files ~2B 1.52 TiB (33.02%) 792.47 GiB (31.65%) 30.59 GiB (2.41%) 728.41 GiB (78.66%)
2 ci_builds ~372M 748.13 GiB (15.92%) 374.42 GiB (14.95%) 250.99 GiB (19.78%) 122.73 GiB (13.25%)
3 merge_request_diff_commits ~3B 734.86 GiB (15.64%) 563.73 GiB (22.51%) 156.64 GiB (12.34%) 14.50 GiB (1.57%)
4 notes ~228M 292.45 GiB (6.22%) 148.53 GiB (5.93%) 134.38 GiB (10.59%) 9.55 GiB (1.03%)
5 ci_build_trace_sections ~2B 263.13 GiB (5.60%) 127.65 GiB (5.10%) 135.48 GiB (10.68%)
6 events ~519M 136.66 GiB (2.91%) 34.42 GiB (1.37%) 102.25 GiB (8.06%) 8.00 KiB (0.00%)
7 web_hook_logs ~54M 104.86 GiB (2.23%) 65.51 GiB (2.62%) 4.58 GiB (0.36%) 34.79 GiB (3.76%)
8 ci_job_artifacts ~316M 88.63 GiB (1.89%) 47.51 GiB (1.90%) 41.12 GiB (3.24%) 8.00 KiB (0.00%)
9 ci_pipelines ~101M 72.50 GiB (1.54%) 20.76 GiB (0.83%) 51.71 GiB (4.07%) 33.58 MiB (0.00%)
10 audit_events ~222M 71.12 GiB (1.51%) 51.96 GiB (2.07%) 19.15 GiB (1.51%) 6.54 MiB (0.00%)
11 sent_notifications ~268M 59.48 GiB (1.27%) 36.93 GiB (1.47%) 22.56 GiB (1.78%) 8.00 KiB (0.00%)
12 push_event_payloads ~390M 55.22 GiB (1.18%) 46.85 GiB (1.87%) 8.37 GiB (0.66%) 8.00 KiB (0.00%)
13 issues ~23M 54.53 GiB (1.16%) 14.53 GiB (0.58%) 33.91 GiB (2.67%) 6.11 GiB (0.66%)
14 merge_requests ~38M 51.92 GiB (1.10%) 16.63 GiB (0.66%) 33.85 GiB (2.67%) 1.45 GiB (0.16%)
15 projects ~12M 43.22 GiB (0.92%) 3.30 GiB (0.13%) 39.65 GiB (3.12%) 277.99 MiB (0.03%)
16 ci_stages ~181M 40.11 GiB (0.85%) 15.01 GiB (0.60%) 25.11 GiB (1.98%) 8.00 KiB (0.00%)
17 ci_builds_metadata ~303M 38.75 GiB (0.82%) 15.43 GiB (0.62%) 23.33 GiB (1.84%) 8.00 KiB (0.00%)
18 taggings ~153M 37.51 GiB (0.80%) 9.76 GiB (0.39%) 27.75 GiB (2.19%) 8.00 KiB (0.00%)
19 deployments ~44M 25.67 GiB (0.55%) 6.94 GiB (0.28%) 18.74 GiB (1.48%) 8.00 KiB (0.00%)
20 merge_request_diffs ~61M 19.77 GiB (0.42%) 12.90 GiB (0.51%) 4.78 GiB (0.38%) 2.10 GiB (0.23%)

Conclusions

Recommendations

  Table of contents

L003 Integer (int2, int4) Out-of-range Risks in PKs

Observations

Data collected: 2020-01-05 13:38:46 +0000 UTC
Current database: gitlabhq_production

Master (10.220.16.106)

Table PK Type Current max value ▼ Capacity used, %
events id int4 607,471,464 28.29
ci_job_artifacts id int4 413,446,131 19.25
ci_builds id int4 394,163,917 18.35
ci_builds_metadata id int4 316,506,247 14.74
notes id int4 267,284,218 12.45
sent_notifications id int4 267,274,736 12.45
web_hook_logs id int4 243,303,211 11.33
audit_events id int4 222,771,944 10.37

Conclusions

  • [P1] High risks of out-of-range errors for an integer column. The columns listed below, being part of a primary key, have high risks to reach 100% of the integer capacity (2^31-1, or 2147483647 for int4 columns, and 2^15-1, or 32767 for int2 columns; see the documentation. If it happens, INSERTs of new rows will not be possible (unless they use some non-incremental values, such as some negative values) and fixing it will require a long downtime. 8 such columns are found:
    • events: reached value 607471464, or 28.29% of int4 capacity
    • ci_job_artifacts: reached value 413446131, or 19.25% of int4 capacity
    • ci_builds: reached value 394163917, or 18.35% of int4 capacity
    • ci_builds_metadata: reached value 316506247, or 14.74% of int4 capacity
    • notes: reached value 267284218, or 12.45% of int4 capacity
    • etc.

Recommendations

  • [P1] High risks of out-of-range errors for an integer column. Consider using int8 in all PK columns, always. To convert existing columns to int8, consider the following approaches:
    1. Blocking ALTER TABLE .. ALTER COLUMN: a straightforward solution requiring significant downtime (a maintenance window).
    2. "New column": create a new column, update it in batches (running not longer than a few seconds, to avoid blocking issues), and then switch to using it, redefining all the constraints. Notice, that to redefine a primary key constraint, ALTER TABLE .. ALTER COLUMN .. SET NOT NULL will be needed. It is a blocking operation in all Postgres versions up to 12 (where it might be lightweight if a proper CHECK constraint is defined first; such constraint can be defined in a non-blocking way). Since Postgres 11, it is possible to use a trick: when adding a column, use DEFAULT with NOT NULL, it will be a non-blocking operation. For all Postgres versions prior to 11, a specific downtime (maintenance window) will be needed anyway.
    3. "New table": create a new table with the same schema as the existing one, capture all ongoing changes to an additional "log" table, copy existing data from the old table to the new one, and switch. This method, as the previous one, is non-trivial and requires careful development and testing under load (consider using Nancy for database experiments developing this solution). This approach is non-blocking regardless of Postgres version, but it requires significantly more efforts to implement.