Docker: Upgrade Gitlab instance to 9.1 error - missing Postgres file .s.PGSQL.5432
Summary
Error upgrading from 8.17.3-ce.0 to 9.1.0-ce.0 by updating to the new version number in the the Dockerfile. Image builds and the container creates and runs but logs show error in the postgres-exporter after upgrade
Error:
gitlab_1 | ==> /var/log/gitlab/postgres-exporter/current <== gitlab_1 | 2017-04-22_20:22:58.70677 time="2017-04-22T20:22:58Z" level=warning msg="Proceeding with outdated query maps, as the Postgres version could not be determined: Error scanning version string: dial unix /var/opt/gitlab/postgresql/.s.PGSQL.5432: connect: no such file or directory\n" source="postgres_exporter.go:1002"
I recreated the old container and found the missing file: /var/opt/gitlab/postgresql/.s.PGSQL.5432
but file is missing in upgraded container.
Steps to reproduce
Have an existing instance of gitlab-ce:8.17.3 running in a Docker container. Upgrade to gitlab-ce:9.1.0 using new version tag Dockerfile
What is the current bug behavior?
Log files show error message shown above and below. Error repeats indefinitely.
What is the expected correct behavior?
Seamless upgrade to 9.1 by simply updating version number
Relevant logs and/or screenshots
gitlab_1 | Thank you for using GitLab Docker Image!
gitlab_1 | Current version: gitlab-ce=9.1.0-ce.0
gitlab_1 |
gitlab_1 | Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
gitlab_1 | And restart this container to reload settings.
gitlab_1 | To do it use docker exec:
gitlab_1 |
gitlab_1 | docker exec -it gitlab vim /etc/gitlab/gitlab.rb
gitlab_1 | docker restart gitlab
gitlab_1 |
gitlab_1 | For a comprehensive list of configuration options please see the Omnibus GitLab readme
gitlab_1 | https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md
gitlab_1 |
gitlab_1 | If this container fails to start due to permission problems try to fix it by executing:
gitlab_1 |
gitlab_1 | docker exec -it gitlab update-permissions
gitlab_1 | docker restart gitlab
gitlab_1 |
gitlab_1 | Preparing services...
gitlab_1 | Starting services...
gitlab_1 | Configuring GitLab package...
gitlab_1 | /opt/gitlab/embedded/bin/runsvdir-start: line 24: ulimit: pending signals: cannot modify limit: Operation not permitted
gitlab_1 | /opt/gitlab/embedded/bin/runsvdir-start: line 37: /proc/sys/fs/file-max: Read-only file system
gitlab_1 | Configuring GitLab...
... cert installation ...
gitlab_1 | gitlab Reconfigured!
Checking for an omnibus managed postgresql: OK
Checking if we already upgraded: NOT OK
Checking for a newer version of PostgreSQL to install: OK
gitlab_1 | Upgrading PostgreSQL to 9.6.1
Checking if PostgreSQL bin files are symlinked to the expected location: OK
gitlab_1 | cp /opt/gitlab/embedded/service/gitlab-rails/public/deploy.html /opt/gitlab/embedded/service/gitlab-rails/public/index.html
Toggling deploy page: OK
Toggling services:ok: down: gitaly: 0s, normally up
gitlab_1 | ok: down: gitlab-monitor: 0s, normally up
gitlab_1 | ok: down: logrotate: 1s, normally up
gitlab_1 | ok: down: node-exporter: 0s, normally up
gitlab_1 | ok: down: postgres-exporter: 0s, normally up
gitlab_1 | ok: down: prometheus: 1s, normally up
gitlab_1 | ok: down: redis-exporter: 0s, normally up
gitlab_1 | ok: down: sidekiq: 0s, normally up
Toggling services: OK
gitlab_1 | Waiting 30 seconds to ensure tasks complete before PostgreSQL upgrade
gitlab_1 | Please hit Ctrl-C now if you want to cancel the upgrade
Stopping the database:ok: down: postgresql: 0s, normally up
Stopping the database: OK
Update the symlinks: OK
Creating temporary data directory: OK
Initializing the new database: OK
Upgrading the data: OK
Move the old data directory out of the way: OK
Rename the new data directory: OK
gitlab_1 | Upgrade is complete, doing post configuration steps
Running reconfigure: OK
gitlab_1 | Database upgrade is complete, running analyze_new_cluster.sh
gitlab_1 | ==== Upgrade has completed ====
gitlab_1 | Please verify everything is working and run the following if so
gitlab_1 | rm -rf /var/opt/gitlab/postgresql/data.9.2.18
gitlab_1 | rm -f /opt/gitlab/embedded/service/gitlab-rails/public/index.html
Toggling deploy page: OK
Toggling services:ok: run: gitaly: (pid 1505) 1s
gitlab_1 | ok: run: gitlab-monitor: (pid 1511) 0s
gitlab_1 | ok: run: logrotate: (pid 1515) 0s
gitlab_1 | ok: run: node-exporter: (pid 1517) 0s
gitlab_1 | ok: run: postgres-exporter: (pid 1519) 0s
gitlab_1 | ok: run: prometheus: (pid 1528) 0s
gitlab_1 | ok: run: redis-exporter: (pid 1544) 1s
gitlab_1 | ok: run: sidekiq: (pid 1553) 0s
Toggling services: OK
gitlab_1 | ==> /var/log/gitlab/gitaly/current <==
gitlab_1 | 2017-04-22_20:09:59.14631 2017/04/22 20:09:59 Starting Gitaly v0.6.0-20170422.132310
gitlab_1 | 2017-04-22_20:09:59.15041 2017/04/22 20:09:59 listening on unix socket "/var/opt/gitlab/gitaly/gitaly.socket"
gitlab_1 | 2017-04-22_20:10:21.99145 2017/04/22 20:10:21 Starting Gitaly v0.6.0-20170422.132310
gitlab_1 | 2017-04-22_20:10:21.99597 2017/04/22 20:10:21 listening on unix socket "/var/opt/gitlab/gitaly/gitaly.socket"
gitlab_1 | 2017-04-22_20:11:40.61317 2017/04/22 20:11:40 Starting Gitaly v0.6.0-20170422.132310
gitlab_1 | 2017-04-22_20:11:40.61676 2017/04/22 20:11:40 listening on unix socket "/var/opt/gitlab/gitaly/gitaly.socket"
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/gitlab-monitor/current <==
gitlab_1 | 2017-04-22_20:10:23.62381 [2017-04-22 20:10:23] INFO WEBrick::HTTPServer#start: pid=663 port=9168
gitlab_1 | 2017-04-22_20:10:24.17142 == Sinatra has ended his set (crowd applauds)
gitlab_1 | 2017-04-22_20:10:24.17195 [2017-04-22 20:10:24] INFO going to shutdown ...
gitlab_1 | 2017-04-22_20:10:24.17225 [2017-04-22 20:10:24] INFO WEBrick::HTTPServer#start done.
gitlab_1 | 2017-04-22_20:11:41.29040 [2017-04-22 20:11:41] INFO WEBrick 1.3.1
gitlab_1 | 2017-04-22_20:11:41.29093 [2017-04-22 20:11:41] INFO ruby 2.3.3 (2016-11-21) [x86_64-linux]
gitlab_1 | 2017-04-22_20:11:41.29121 == Sinatra (v1.4.8) has taken the stage on 9168 for development with backup from WEBrick
gitlab_1 | 2017-04-22_20:11:41.29155 [2017-04-22 20:11:41] INFO WEBrick::HTTPServer#start: pid=1511 port=9168
gitlab_1 | 2017-04-22_20:11:42.22505 ::1 - - [22/Apr/2017:20:11:42 UTC] "GET /sidekiq HTTP/1.1" 200 3238
gitlab_1 | 2017-04-22_20:11:42.22549 - -> /sidekiq
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/gitlab-rails/application.log <==
gitlab_1 | # Logfile created on 2017-04-22 20:03:16 +0000 by logger.rb/56438
gitlab_1 | April 22, 2017 20:03: User "Administrator" (admin@example.com) was created
gitlab_1 | April 22, 2017 20:07: User "Archie Archbold" (archiea@xxx) was created
gitlab_1 | April 22, 2017 20:07: (OAuth) saving user archiea@xxx from login with extern_uid => archiea
gitlab_1 | April 22, 2017 20:07: Archie Archbold created a new project "Archie Archbold / tester-testy"
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2017-04-22-20-02-53.log <==
gitlab_1 | -> 0.0314s
gitlab_1 |
gitlab_1 | == Seed from /opt/gitlab/embedded/service/gitlab-rails/db/fixtures/production/001_admin.rb
gitlab_1 | Administrator account created:
gitlab_1 |
gitlab_1 | login: root
gitlab_1 | password: You'll be prompted to create one on your first visit.
gitlab_1 |
gitlab_1 |
gitlab_1 | == Seed from /opt/gitlab/embedded/service/gitlab-rails/db/fixtures/production/010_settings.rb
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2017-04-22-20-09-15.log <==
gitlab_1 |
gitlab_1 | == 20170419001229 AddIndexToSystemNoteMetadata: migrating =====================
gitlab_1 | -- transaction_open?()
gitlab_1 | -> 0.0000s
gitlab_1 | -- execute("SET statement_timeout TO 0")
gitlab_1 | -> 0.0003s
gitlab_1 | -- add_index(:system_note_metadata, :note_id, {:unique=>true, :algorithm=>:concurrently})
gitlab_1 | -> 0.0139s
gitlab_1 | == 20170419001229 AddIndexToSystemNoteMetadata: migrated (0.0143s) ============
gitlab_1 |
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/gitlab-rails/production.log <==
gitlab_1 | Migrating to AddActiveToCiTriggerSchedule (20170407122426)
gitlab_1 | Migrating to AddForeighKeyTriggerRequestsTrigger (20170407135259)
gitlab_1 | Migrating to AddIndexToNextRunAtAndActive (20170407140450)
gitlab_1 | Migrating to RemoveOldCacheDirectories (20170408033905)
gitlab_1 | Migrating to DeleteOrphanNotificationSettings (20170418103908)
gitlab_1 | Migrating to AddIndexToSystemNoteMetadata (20170419001229)
gitlab_1 | Raven 2.4.0 configured not to capture errors: DSN not set
gitlab_1 | Raven 2.4.0 configured not to capture errors: DSN not set
gitlab_1 | Raven 2.4.0 configured not to capture errors: DSN not set
gitlab_1 | Raven 2.4.0 configured not to capture errors: DSN not set
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/gitlab-shell/gitlab-shell.log <==
gitlab_1 | # Logfile created on 2017-04-22 20:02:26 +0000 by logger.rb/56438
gitlab_1 | I, [2017-04-22T20:07:16.716949 #1142] INFO -- : Adding project archiea/tester-testy.git at </var/opt/gitlab/git-data/repositories/archiea/tester-testy.git>.
gitlab_1 | I, [2017-04-22T20:07:16.831099 #1142] INFO -- : Moving existing hooks directory and symlinking global hooks directory for /var/opt/gitlab/git-data/repositories/archiea/tester-testy.git.
gitlab_1 | I, [2017-04-22T20:07:17.691095 #1147] INFO -- : Adding project archiea/tester-testy.wiki.git at </var/opt/gitlab/git-data/repositories/archiea/tester-testy.wiki.git>.
gitlab_1 | I, [2017-04-22T20:07:17.784275 #1147] INFO -- : Moving existing hooks directory and symlinking global hooks directory for /var/opt/gitlab/git-data/repositories/archiea/tester-testy.wiki.git.
gitlab_1 | I, [2017-04-22T20:08:04.960655 #1208] INFO -- : POST http://127.0.0.1:8080/api/v3/internal/allowed 1.79332
gitlab_1 | I, [2017-04-22T20:08:06.862169 #1223] INFO -- : GET http://127.0.0.1:8080/api/v3/internal/broadcast_message 1.50698
gitlab_1 | I, [2017-04-22T20:08:06.917178 #1223] INFO -- : GET http://127.0.0.1:8080/api/v3/internal/merge_request_urls?project=/var/opt/gitlab/git-data/repositories/archiea/tester-testy.git&changes=0000000000000000000000000000000000000000%208befd2dbd07380f005b742ecb7677dbfb87a1837%20refs/heads/master%0A 0.05414
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/gitlab-workhorse/current <==
gitlab_1 | 2017-04-22_20:10:00.06844 2017/04/22 20:10:00 Starting gitlab-workhorse v1.4.3-20170422.132251
gitlab_1 | 2017-04-22_20:10:00.07124 2017/04/22 20:10:00 Can not load config file "config.toml": open config.toml: no such file or directory
gitlab_1 | 2017-04-22_20:10:01.09838 2017/04/22 20:10:01 Starting gitlab-workhorse v1.4.3-20170422.132251
gitlab_1 | 2017-04-22_20:10:01.10598 2017/04/22 20:10:01 keywatcher: starting process loop
gitlab_1 | 2017-04-22_20:10:01.11058 2017/04/22 20:10:01 redis: dialing "tcp", "redis:6379"
gitlab_1 | 2017-04-22_20:10:22.03287 2017/04/22 20:10:22 Starting gitlab-workhorse v1.4.3-20170422.132251
gitlab_1 | 2017-04-22_20:10:22.03688 2017/04/22 20:10:22 keywatcher: starting process loop
gitlab_1 | 2017-04-22_20:10:22.03746 2017/04/22 20:10:22 redis: dialing "tcp", "redis:6379"
gitlab_1 | 2017-04-22_20:10:57.62078 localhost @ - - [2017-04-22 20:10:57.620530657 +0000 UTC] "GET /help HTTP/1.1" 200 2576 "" "curl/7.53.0" 0.000192
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/logrotate/current <==
gitlab_1 | 2017-04-22_20:10:24.59263 Received TERM from runit, sending to process group (-PID)
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/nginx/access.log <==
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/nginx/current <==
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/nginx/error.log <==
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/nginx/gitlab_access.log <==
gitlab_1 | 172.18.0.1 - - [22/Apr/2017:20:07:06 +0000] "GET / HTTP/1.1" 200 4824 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
gitlab_1 | 172.18.0.1 - - [22/Apr/2017:20:07:10 +0000] "GET /projects/new HTTP/1.1" 200 7299 "http://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
gitlab_1 | 127.0.0.1 - - [22/Apr/2017:20:07:17 +0000] "GET /help HTTP/1.1" 200 18571 "-" "curl/7.52.0"
gitlab_1 | 172.18.0.1 - - [22/Apr/2017:20:07:18 +0000] "POST /projects HTTP/1.1" 302 103 "http://localhost/projects/new" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
gitlab_1 | 172.18.0.1 - - [22/Apr/2017:20:07:18 +0000] "GET /archiea/tester-testy HTTP/1.1" 200 8360 "http://localhost/projects/new" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
gitlab_1 | 172.18.0.1 - - [22/Apr/2017:20:07:26 +0000] "GET /archiea/tester-testy/new/master?commit_message=Add+readme.md&file_name=README.md HTTP/1.1" 200 8559 "http://localhost/archiea/tester-testy" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
gitlab_1 | 172.18.0.1 - - [22/Apr/2017:20:08:06 +0000] "POST /archiea/tester-testy/create/master HTTP/1.1" 302 125 "http://localhost/archiea/tester-testy/new/master?commit_message=Add+readme.md&file_name=README.md" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
gitlab_1 | 172.18.0.1 - - [22/Apr/2017:20:08:07 +0000] "GET /archiea/tester-testy/blob/master/README.md HTTP/1.1" 200 7505 "http://localhost/archiea/tester-testy/new/master?commit_message=Add+readme.md&file_name=README.md" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
gitlab_1 | 127.0.0.1 - - [22/Apr/2017:20:08:17 +0000] "GET /help HTTP/1.1" 200 18556 "-" "curl/7.52.0"
gitlab_1 | 127.0.0.1 - - [22/Apr/2017:20:10:57 +0000] "GET /help HTTP/1.1" 200 2588 "-" "curl/7.53.0"
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/nginx/gitlab_error.log <==
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/node-exporter/current <==
gitlab_1 | 2017-04-22_20:11:41.48724 time="2017-04-22T20:11:41Z" level=info msg=" - netstat" source="node_exporter.go:162"
gitlab_1 | 2017-04-22_20:11:41.48749 time="2017-04-22T20:11:41Z" level=info msg=" - stat" source="node_exporter.go:162"
gitlab_1 | 2017-04-22_20:11:41.48772 time="2017-04-22T20:11:41Z" level=info msg=" - uname" source="node_exporter.go:162"
gitlab_1 | 2017-04-22_20:11:41.48797 time="2017-04-22T20:11:41Z" level=info msg=" - vmstat" source="node_exporter.go:162"
gitlab_1 | 2017-04-22_20:11:41.48827 time="2017-04-22T20:11:41Z" level=info msg=" - wifi" source="node_exporter.go:162"
gitlab_1 | 2017-04-22_20:11:41.48852 time="2017-04-22T20:11:41Z" level=info msg=" - entropy" source="node_exporter.go:162"
gitlab_1 | 2017-04-22_20:11:41.48875 time="2017-04-22T20:11:41Z" level=info msg=" - edac" source="node_exporter.go:162"
gitlab_1 | 2017-04-22_20:11:41.48901 time="2017-04-22T20:11:41Z" level=info msg=" - mdadm" source="node_exporter.go:162"
gitlab_1 | 2017-04-22_20:11:41.48926 time="2017-04-22T20:11:41Z" level=info msg=" - netdev" source="node_exporter.go:162"
gitlab_1 | 2017-04-22_20:11:41.48950 time="2017-04-22T20:11:41Z" level=info msg="Listening on localhost:9100" source="node_exporter.go:186"
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/postgres-exporter/current <==
gitlab_1 | 2017-04-22_20:10:06.26333 time="2017-04-22T20:10:06Z" level=info msg="Semantic Version Changed: 0.0.0 -> 9.2.18" source="postgres_exporter.go:945"
gitlab_1 | 2017-04-22_20:10:06.33825 time="2017-04-22T20:10:06Z" level=info msg="Starting Server: localhost:9187" source="postgres_exporter.go:1042"
gitlab_1 | 2017-04-22_20:10:22.95371 time="2017-04-22T20:10:22Z" level=info msg="Semantic Version Changed: 0.0.0 -> 9.2.18" source="postgres_exporter.go:945"
gitlab_1 | 2017-04-22_20:10:23.02910 time="2017-04-22T20:10:23Z" level=info msg="Starting Server: localhost:9187" source="postgres_exporter.go:1042"
gitlab_1 | 2017-04-22_20:11:41.46647 time="2017-04-22T20:11:41Z" level=warning msg="Proceeding with outdated query maps, as the Postgres version could not be determined: Error scanning version string: dial unix /var/opt/gitlab/postgresql/.s.PGSQL.5432: connect: no such file or directory\n" source="postgres_exporter.go:1002"
gitlab_1 | 2017-04-22_20:11:41.46716 time="2017-04-22T20:11:41Z" level=info msg="Starting Server: localhost:9187" source="postgres_exporter.go:1042"
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/postgresql/current <==
gitlab_1 | 2017-04-22_20:09:14.90923 FATAL: configuration file "/var/opt/gitlab/postgresql/data/postgresql.conf" contains errors
gitlab_1 | 2017-04-22_20:09:16.04183 LOG: database system was shut down at 2017-04-22 20:08:50 GMT
gitlab_1 | 2017-04-22_20:09:16.06291 LOG: database system is ready to accept connections
gitlab_1 | 2017-04-22_20:09:16.06333 LOG: autovacuum launcher started
gitlab_1 | 2017-04-22_20:10:59.78150 received TERM from runit, sending INT instead to force quit connections
gitlab_1 | 2017-04-22_20:10:59.78232 LOG: received fast shutdown request
gitlab_1 | 2017-04-22_20:10:59.78264 LOG: aborting any active transactions
gitlab_1 | 2017-04-22_20:10:59.78290 LOG: autovacuum launcher shutting down
gitlab_1 | 2017-04-22_20:10:59.78346 LOG: shutting down
gitlab_1 | 2017-04-22_20:11:00.31828 LOG: database system is shut down
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/prometheus/current <==
gitlab_1 | 2017-04-22_20:10:25.92685 time="2017-04-22T20:10:25Z" level=info msg="Local storage stopped." source="storage.go:421"
gitlab_1 | 2017-04-22_20:11:41.54705 time="2017-04-22T20:11:41Z" level=info msg="Starting prometheus (version=, branch=, revision=)" source="main.go:75"
gitlab_1 | 2017-04-22_20:11:41.54739 time="2017-04-22T20:11:41Z" level=info msg="Build context (go=go1.7.5, user=, date=)" source="main.go:76"
gitlab_1 | 2017-04-22_20:11:41.55255 time="2017-04-22T20:11:41Z" level=info msg="Loading configuration file /var/opt/gitlab/prometheus/prometheus.yml" source="main.go:248"
gitlab_1 | 2017-04-22_20:11:41.62690 time="2017-04-22T20:11:41Z" level=info msg="Loading series map and head chunks..." source="storage.go:373"
gitlab_1 | 2017-04-22_20:11:41.64660 time="2017-04-22T20:11:41Z" level=info msg="1977 series loaded." source="storage.go:378"
gitlab_1 | 2017-04-22_20:11:41.64842 time="2017-04-22T20:11:41Z" level=info msg="Starting target manager..." source="targetmanager.go:61"
gitlab_1 | 2017-04-22_20:11:41.64872 time="2017-04-22T20:11:41Z" level=info msg="Listening on localhost:9090" source="web.go:259"
gitlab_1 | 2017-04-22_20:11:41.64903 time="2017-04-22T20:11:41Z" level=error msg="Error creating HTTP client for job \"kubernetes-nodes\": unable to use specified CA cert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory" source="scrape.go:119"
gitlab_1 | 2017-04-22_20:11:41.64926 time="2017-04-22T20:11:41Z" level=error msg="Cannot create Kubernetes discovery: open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory" source="discovery.go:85"
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/redis/current <==
gitlab_1 | 2017-04-22_20:09:12.79892 `-._ `-._`-.__.-'_.-' _.-'
gitlab_1 | 2017-04-22_20:09:12.79919 `-._ `-.__.-' _.-'
gitlab_1 | 2017-04-22_20:09:12.79945 `-._ _.-'
gitlab_1 | 2017-04-22_20:09:12.79972 `-.__.-'
gitlab_1 | 2017-04-22_20:09:12.79999
gitlab_1 | 2017-04-22_20:09:12.80025 337:M 22 Apr 20:09:12.793 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
gitlab_1 | 2017-04-22_20:09:12.80051 337:M 22 Apr 20:09:12.793 # Server started, Redis version 3.2.5
gitlab_1 | 2017-04-22_20:09:12.80075 337:M 22 Apr 20:09:12.794 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
gitlab_1 | 2017-04-22_20:09:12.80102 337:M 22 Apr 20:09:12.796 * DB loaded from disk: 0.003 seconds
gitlab_1 | 2017-04-22_20:09:12.80127 337:M 22 Apr 20:09:12.796 * The server is now ready to accept connections at /var/opt/gitlab/redis/redis.socket
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/redis-exporter/current <==
gitlab_1 | 2017-04-22_20:10:05.21151 time="2017-04-22T20:10:05Z" level=info msg="Redis Metrics Exporter <<< filled in by build >>> build date: <<< filled in by build >>> sha1: <<< filled in by build >>>\n"
gitlab_1 | 2017-04-22_20:10:05.21192 time="2017-04-22T20:10:05Z" level=info msg="Providing metrics at localhost:9121/metrics"
gitlab_1 | 2017-04-22_20:10:05.21224 time="2017-04-22T20:10:05Z" level=info msg="Connecting to redis hosts: []string{\"unix:///var/opt/gitlab/redis/redis.socket\"}"
gitlab_1 | 2017-04-22_20:10:05.21256 time="2017-04-22T20:10:05Z" level=info msg="Using alias: []string{\"\"}"
gitlab_1 | 2017-04-22_20:11:41.89064 time="2017-04-22T20:11:41Z" level=info msg="Redis Metrics Exporter <<< filled in by build >>> build date: <<< filled in by build >>> sha1: <<< filled in by build >>>\n"
gitlab_1 | 2017-04-22_20:11:41.89105 time="2017-04-22T20:11:41Z" level=info msg="Providing metrics at localhost:9121/metrics"
gitlab_1 | 2017-04-22_20:11:41.89133 time="2017-04-22T20:11:41Z" level=info msg="Connecting to redis hosts: []string{\"unix:///var/opt/gitlab/redis/redis.socket\"}"
gitlab_1 | 2017-04-22_20:11:41.89159 time="2017-04-22T20:11:41Z" level=info msg="Using alias: []string{\"\"}"
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/sidekiq/current <==
gitlab_1 | 2017-04-22_20:10:11.85836 2017-04-22T20:10:11.858Z 436 TID-ory75cjzc INFO: Running in ruby 2.3.3p222 (2016-11-21 revision 56859) [x86_64-linux]
gitlab_1 | 2017-04-22_20:10:11.85896 2017-04-22T20:10:11.858Z 436 TID-ory75cjzc INFO: See LICENSE and the LGPL-3.0 for licensing details.
gitlab_1 | 2017-04-22_20:10:11.85935 2017-04-22T20:10:11.858Z 436 TID-ory75cjzc INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org
gitlab_1 | 2017-04-22_20:10:11.85979 2017-04-22T20:10:11.859Z 436 TID-ory75cjzc INFO: Starting processing, hit Ctrl-C to stop
gitlab_1 | 2017-04-22_20:10:26.71857 2017-04-22T20:10:26.718Z 436 TID-ory75cjzc INFO: Shutting down
gitlab_1 | 2017-04-22_20:10:26.71911 2017-04-22T20:10:26.718Z 436 TID-ory3q3zdk INFO: Scheduler exiting...
gitlab_1 | 2017-04-22_20:10:26.71948 2017-04-22T20:10:26.718Z 436 TID-ory75cjzc INFO: Terminating quiet workers
gitlab_1 | 2017-04-22_20:10:26.71989 2017-04-22T20:10:26.719Z 436 TID-ory4h585g INFO: Scheduler exiting...
gitlab_1 | 2017-04-22_20:10:27.22012 2017-04-22T20:10:27.219Z 436 TID-ory75cjzc INFO: Pausing to allow workers to finish...
gitlab_1 | 2017-04-22_20:10:29.22367 2017-04-22T20:10:29.223Z 436 TID-ory75cjzc INFO: Bye!
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/sshd/current <==
gitlab_1 | 2017-04-22_20:09:01.81958 Server listening on 0.0.0.0 port 22.
gitlab_1 | 2017-04-22_20:09:01.81990 Server listening on :: port 22.
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/unicorn/current <==
gitlab_1 | 2017-04-22_20:09:50.96947 starting new unicorn master
gitlab_1 | 2017-04-22_20:10:11.02148 adopted new unicorn master 426
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/unicorn/unicorn_stderr.log <==
gitlab_1 | I, [2017-04-22T20:08:54.598299 #768] INFO -- : reaped #<Process::Status: pid 829 exit 0> worker=2
gitlab_1 | I, [2017-04-22T20:08:54.598827 #768] INFO -- : master complete
gitlab_1 | I, [2017-04-22T20:09:51.558106 #426] INFO -- : Refreshing Gem list
gitlab_1 | I, [2017-04-22T20:10:09.969313 #426] INFO -- : listening on addr=127.0.0.1:8080 fd=17
gitlab_1 | I, [2017-04-22T20:10:09.971661 #426] INFO -- : unlinking existing socket=/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket
gitlab_1 | I, [2017-04-22T20:10:09.974034 #426] INFO -- : listening on addr=/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket fd=18
gitlab_1 | I, [2017-04-22T20:10:09.982587 #529] INFO -- : worker=0 ready
gitlab_1 | I, [2017-04-22T20:10:09.990912 #532] INFO -- : worker=1 ready
gitlab_1 | I, [2017-04-22T20:10:09.994966 #426] INFO -- : master process ready
gitlab_1 | I, [2017-04-22T20:10:09.996458 #535] INFO -- : worker=2 ready
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/unicorn/unicorn_stdout.log <==
gitlab_1 | I, [2017-04-22T20:06:58.193123 #826] INFO -- omniauth: (cas3) Request phase initiated.
gitlab_1 | I, [2017-04-22T20:07:05.066658 #829] INFO -- omniauth: (cas3) Callback phase initiated.
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/gitlab-monitor/current <==
gitlab_1 | 2017-04-22_20:11:43.99137 ::1 - - [22/Apr/2017:20:11:43 UTC] "GET /database HTTP/1.1" 200 0
gitlab_1 | 2017-04-22_20:11:43.99166 - -> /database
gitlab_1 |
gitlab_1 | ==> /var/log/gitlab/postgres-exporter/current <==
** gitlab_1 | 2017-04-22_20:11:43.70134 time="2017-04-22T20:11:43Z" level=warning msg="Proceeding with outdated query maps, as the Postgres version could not be determined: Error scanning version string: dial unix /var/opt/gitlab/postgresql/.s.PGSQL.5432: connect: no such file or directory\n" source="postgres_exporter.go:1002" **
Results of GitLab environment info
`System information System: Current User: git Using RVM: no Ruby Version: 2.3.3p222 Gem Version: 2.6.6 Bundler Version:1.13.7 Rake Version: 10.5.0 Redis Version: 3.2.5 Git Version: 2.11.1 Sidekiq Version:4.2.7
GitLab information Version: 9.1.0 Revision: c3f0f14 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: http://localhost HTTP Clone URL: http://localhost/some-group/some-project.git SSH Clone URL: ssh://git@localhost:9022/some-group/some-project.git Using LDAP: no Using Omniauth: yes Omniauth Providers: cas3
GitLab Shell Version: 5.0.2 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git`
Results of GitLab application Check
`Checking GitLab Shell ...
GitLab Shell version >= 5.0.2 ? ... OK (5.0.2) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 2/1 ... ok Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Access to /var/opt/gitlab/.ssh/authorized_keys: OK Send ping to redis server: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Reply by email ...
Reply by email is disabled in config/gitlab.yml
Checking Reply by email ... Finished
Checking LDAP ...
LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab ...
Git configured with autocrlf=input? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config outdated? ... no Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory setup correctly? ... skipped (no tmp uploads folder yet) Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) projects have namespace: ... 2/1 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.1.0 ? ... yes (2.3.3) Your git bin path is "/opt/gitlab/embedded/bin/git" Git version >= 2.7.3 ? ... yes (2.11.1) Active users: 2
Checking GitLab ... Finished`