### Summary When running GitLab in docker on macOS, GitLabs raises multiple errors due to `too many open files`. Not possible to push to repositories. Nothing is possible only restart solve problem. ``` 503 Whoops, GitLab is currently unavailable. ``` Docker process `com.docker.osxfx` open files reaches limit 10245 and then error appears. Docker Desktop (macOS): Version 2.0.0.3 (31259) macOS: 10.13.6 Restarting GitLab container results in immediate lowering opened files by `com.docker.osxfx` -> Docker correctly release file descriptors ### Steps to reproduce 1. Install Docker Desktop 2.0.0.3 on macOS 2. `docker-compose up -d` with `docker-compose.yml` 3. Work with GitLab ``` version: '2' services: gitlab: image: 'gitlab/gitlab-ce:latest' restart: always container_name: gitlab-latest hostname: imac.local environment: GITLAB_OMNIBUS_CONFIG: | external_url 'http://imac.local:9080' gitlab_rails['gitlab_shell_ssh_port'] = 9022 ports: - '9443:443' # https port - '9080:9080' # http port - '9022:22' # docker:host volumes: - '~/Docker/GitLab/config:/etc/gitlab:cached' - '~/Docker/GitLab/logs:/var/log/gitlab:cached' - '~/Docker/GitLab/data:/var/opt/gitlab:cached' ``` ### What is the current *bug* behavior? Open files per `com.docker.osxfs` grows up to `10245` when Postgresql starts to complain about `too many open files` ``` PID=`pgrep com.docker.osxfs` lsof -n -p $PID | wc ``` ### What is the expected *correct* behavior? Opened files per process should not reach more then 2000 ### Relevant logs <details> <summary> Relevant logs </summary> <pre> Completed 200 OK in 55ms (ActiveRecord: 7.1ms) Sent file /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/system/project/avatar/19/home.png (0.8ms) Completed 200 OK in 65ms (ActiveRecord: 7.5ms) Sent file /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/system/project/avatar/8/icon.png (0.9ms) Completed 200 OK in 41ms (ActiveRecord: 6.0ms) Started GET "/inventloop/other" for 172.18.0.1 at 2019-05-07 16:44:10 +0000 Processing by GroupsController#show as HTML Parameters: {"id"=>"inventloop/other"} Completed 200 OK in 148ms (Views: 91.9ms | ActiveRecord: 26.5ms) Started GET "/groups/inventloop/other/-/children.json" for 172.18.0.1 at 2019-05-07 16:44:10 +0000 Processing by Groups::ChildrenController#index as JSON Parameters: {"group_id"=>"inventloop/other"} Started GET "/uploads/-/system/appearance/header_logo/1/inventloop-logo-tiny_2x.png" for 172.18.0.1 at 2019-05-07 16:44:10 +0000 Started GET "/uploads/-/system/group/avatar/3/inventloop.png?width=15" for 172.18.0.1 at 2019-05-07 16:44:10 +0000 Started GET "/uploads/-/system/user/avatar/2/avatar.png?width=23" for 172.18.0.1 at 2019-05-07 16:44:10 +0000 Processing by UploadsController#show as HTML Parameters: {"model"=>"appearance", "mounted_as"=>"header_logo", "id"=>"1", "filename"=>"inventloop-logo-tiny_2x.png"} Processing by UploadsController#show as HTML Processing by UploadsController#show as HTML Parameters: {"width"=>"15", "model"=>"group", "mounted_as"=>"avatar", "id"=>"3", "filename"=>"inventloop.png"} Parameters: {"width"=>"23", "model"=>"user", "mounted_as"=>"avatar", "id"=>"2", "filename"=>"avatar.png"} Sent file /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/system/group/avatar/3/inventloop.png (1.2ms) Completed 200 OK in 33ms (ActiveRecord: 4.8ms) Sent file /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/system/appearance/header_logo/1/inventloop-logo-tiny_2x.png (0.7ms) Completed 200 OK in 54ms (ActiveRecord: 30.2ms) Sent file /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/system/user/avatar/2/avatar.png (2.7ms) Completed 200 OK in 49ms (ActiveRecord: 8.4ms) Completed 200 OK in 579ms (Views: 0.5ms | ActiveRecord: 38.8ms) Started GET "/-/metrics" for 127.0.0.1 at 2019-05-07 16:44:14 +0000 Processing by MetricsController#index as HTML Completed 200 OK in 30ms (Views: 0.6ms | ActiveRecord: 0.0ms) Started GET "/fbucek/invsdk.git/info/refs?service=git-upload-pack" for 172.18.0.1 at 2019-05-07 16:44:27 +0000 Started GET "/fbucek/fileork.git/info/refs?service=git-upload-pack" for 172.18.0.1 at 2019-05-07 16:44:27 +0000 Processing by Projects::GitHttpController#info_refs as */* Parameters: {"service"=>"git-upload-pack", "namespace_id"=>"fbucek", "project_id"=>"invsdk.git"} Processing by Projects::GitHttpController#info_refs as */* Parameters: {"service"=>"git-upload-pack", "namespace_id"=>"fbucek", "project_id"=>"fileork.git"} Completed 200 OK in 29ms (Views: 1.1ms | ActiveRecord: 3.6ms) Completed 200 OK in 39ms (Views: 0.6ms | ActiveRecord: 4.2ms) Started GET "/fbucek/code-management.git/info/refs?service=git-upload-pack" for 172.18.0.1 at 2019-05-07 16:44:27 +0000 Started GET "/jobs/job-tasks.git/info/refs?service=git-upload-pack" for 172.18.0.1 at 2019-05-07 16:44:27 +0000 Processing by Projects::GitHttpController#info_refs as */* Processing by Projects::GitHttpController#info_refs as */* Parameters: {"service"=>"git-upload-pack", "namespace_id"=>"fbucek", "project_id"=>"code-management.git"} Parameters: {"service"=>"git-upload-pack", "namespace_id"=>"jobs", "project_id"=>"job-tasks.git"} Filter chain halted as :authenticate_user rendered or redirected Completed 401 Unauthorized in 20ms (Views: 0.6ms | ActiveRecord: 4.1ms) Filter chain halted as :authenticate_user rendered or redirected Completed 401 Unauthorized in 62ms (Views: 1.1ms | ActiveRecord: 3.1ms) Started GET "/fbucek/code-management.git/info/refs?service=git-upload-pack" for 172.18.0.1 at 2019-05-07 16:44:27 +0000 Processing by Projects::GitHttpController#info_refs as */* Parameters: {"service"=>"git-upload-pack", "namespace_id"=>"fbucek", "project_id"=>"code-management.git"} Started GET "/jobs/job-tasks.git/info/refs?service=git-upload-pack" for 172.18.0.1 at 2019-05-07 16:44:27 +0000 Processing by Projects::GitHttpController#info_refs as */* Parameters: {"service"=>"git-upload-pack", "namespace_id"=>"jobs", "project_id"=>"job-tasks.git"} Completed 200 OK in 255ms (Views: 0.4ms | ActiveRecord: 20.5ms) Completed 200 OK in 446ms (Views: 0.4ms | ActiveRecord: 39.6ms) Started GET "/-/metrics" for 127.0.0.1 at 2019-05-07 16:44:29 +0000 Processing by MetricsController#index as HTML Completed 200 OK in 32ms (Views: 0.7ms | ActiveRecord: 0.0ms) Started GET "/-/metrics" for 127.0.0.1 at 2019-05-07 16:44:44 +0000 Processing by MetricsController#index as HTML Completed 200 OK in 129ms (Views: 0.8ms | ActiveRecord: 0.0ms) tail: '/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2019-04-29-14-06-10.log' has become inaccessible: Too many open files tail: '/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2019-01-24-07-19-39.log' has become accessible ==> /var/log/gitlab/unicorn/unicorn_stderr.log <== caught error of type Gitlab::Git::CommandError in after callback inside Grape::Middleware::Formatter : 2:read info line: EOF caught error of type Gitlab::Git::CommandError in after callback inside Grape::Middleware::Formatter : 2:read info line: EOF caught error of type Gitlab::Git::CommandError in after callback inside Grape::Middleware::Formatter : 2:read info line: EOF I, [2019-05-07T12:33:04.600828 #638] INFO -- : Refreshing Gem list I, [2019-05-07T12:34:12.609267 #638] INFO -- : listening on addr=127.0.0.1:8080 fd=21 I, [2019-05-07T12:34:12.610486 #638] INFO -- : unlinking existing socket=/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket I, [2019-05-07T12:34:12.613207 #638] INFO -- : listening on addr=/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket fd=22 I, [2019-05-07T12:34:12.734492 #638] INFO -- : master process ready I, [2019-05-07T12:34:12.836517 #855] INFO -- : worker=0 ready I, [2019-05-07T12:34:12.864967 #861] INFO -- : worker=2 ready I, [2019-05-07T12:34:12.885325 #858] INFO -- : worker=1 ready I, [2019-05-07T12:34:12.927121 #864] INFO -- : worker=3 ready W, [2019-05-07T12:37:45.732255 #864] WARN -- : #<Unicorn::HttpServer:0x00007f980beb7260>: worker (pid: 864) exceeds memory limit (448656384.0 bytes > 437936556 bytes) W, [2019-05-07T12:37:45.733867 #864] WARN -- : Unicorn::WorkerKiller send SIGQUIT (pid: 864) alive: 212 sec (trial 1) I, [2019-05-07T12:37:46.107825 #638] INFO -- : reaped #<Process::Status: pid 864 exit 0> worker=3 I, [2019-05-07T12:37:46.642380 #2085] INFO -- : worker=3 ready I, [2019-05-07T12:50:10.745083 #638] INFO -- : reaped #<Process::Status: pid 858 exit 0> worker=1 I, [2019-05-07T12:50:10.747053 #638] INFO -- : reaped #<Process::Status: pid 861 exit 0> worker=2 I, [2019-05-07T12:50:10.747986 #638] INFO -- : reaped #<Process::Status: pid 2085 exit 0> worker=3 I, [2019-05-07T12:50:52.562351 #770] INFO -- : Refreshing Gem list I, [2019-05-07T12:51:29.639729 #770] INFO -- : listening on addr=127.0.0.1:8080 fd=21 I, [2019-05-07T12:51:29.641049 #770] INFO -- : unlinking existing socket=/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket I, [2019-05-07T12:51:29.644441 #770] INFO -- : listening on addr=/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket fd=22 I, [2019-05-07T12:51:29.818334 #770] INFO -- : master process ready I, [2019-05-07T12:51:30.022666 #931] INFO -- : worker=2 ready I, [2019-05-07T12:51:30.048932 #934] INFO -- : worker=3 ready I, [2019-05-07T12:51:30.048932 #925] INFO -- : worker=0 ready I, [2019-05-07T12:51:30.077419 #928] INFO -- : worker=1 ready W, [2019-05-07T13:44:01.101729 #934] WARN -- : #<Unicorn::HttpServer:0x00007fcbd3c9f530>: worker (pid: 934) exceeds memory limit (506458624.0 bytes > 502553016 bytes) W, [2019-05-07T13:44:01.105212 #934] WARN -- : Unicorn::WorkerKiller send SIGQUIT (pid: 934) alive: 3127 sec (trial 1) I, [2019-05-07T13:44:01.390238 #770] INFO -- : reaped #<Process::Status: pid 934 exit 0> worker=3 I, [2019-05-07T13:44:01.692976 #8960] INFO -- : worker=3 ready caught error of type Gitlab::Git::CommandError in after callback inside Grape::Middleware::Formatter : 2:read info line: EOF caught error of type Gitlab::Git::CommandError in after callback inside Grape::Middleware::Formatter : 2:read info line: EOF caught error of type Gitlab::Git::CommandError in after callback inside Grape::Middleware::Formatter : 2:read info line: EOF caught error of type Gitlab::Git::CommandError in after callback inside Grape::Middleware::Formatter : 2:read info line: EOF caught error of type Gitlab::Git::CommandError in after callback inside Grape::Middleware::Formatter : 2:read info line: EOF caught error of type Gitlab::Git::CommandError in after callback inside Grape::Middleware::Formatter : 2:read info line: EOF caught error of type Gitlab::Git::CommandError in after callback inside Grape::Middleware::Formatter : 2:read info line: EOF caught error of type Gitlab::Git::CommandError in after callback inside Grape::Middleware::Formatter : 2:read info line: EOF tail: '/var/log/gitlab/nginx/state' has become inaccessible: Too many open files tail: '/var/log/gitlab/nginx/access.log' has become accessible </pre> </details> ### Details of package version <details> <summary>Provide the package version installation details</summary> <pre> Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==========================================-==========================-==========================-========================================================================================= ii gitlab-ce 11.10.4-ce.0 amd64 GitLab Community Edition (including NGINX, Postgres, Redis) un gitlab-ee <none> <none> (no description available) </pre> </details> ### Environment details * Operating System: macOS * Installation Target, remove incorrect values: * Other: Docker Desktop version: 2.0.0.3 * Installation Type, remove incorrect values: * Upgrade continuously from versions 10.10.0 / 10.9.0 / 10.8.0 ( always the same problem ) * Is there any other software running on the machine: Running alone in Docker * Is this a single or multiple node installation? Single * Resources * CPU: 2.5 GHZ Inter Core i5 - 4 core ( Docker assigned 3 cores ) * Memory total: 12 GB ( Docker 6GB ) ### Configuration details <details> <summary> Provide the relevant sections of `/etc/gitlab/gitlab.rb` </summary> <pre> external_url 'http://imac.local:9080' postgresql['max_files_per_process'] = 200 nginx['http2_enabled'] = true pages_external_url "http://pages.imac.dev/" gitlab_pages['enable'] = true gitlab_pages['redirect_http'] = false gitlab_pages['use_http2'] = true gitlab_pages['inplace_chroot'] = true </pre> </details>