Skip to content

gitlab spams /tmp dir in docker container with gitlab-pages

Summary

I have a GitLab CE instance running on an ubuntu server using the official gitlab-ce docker image. The usual data volumes are mounted to a host directory srv/gitlab, which resides on a separate disk.

volumes:
      - '/srv/gitlab/config:/etc/gitlab'
      - '/srv/gitlab/logs:/var/log/gitlab'
      - '/srv/gitlab/data:/var/opt/gitlab'

This morning I found the root drive 100% used, with the major amount beeing used by the running gitlab-ce docker container (10.1 GB). Digging into it I found the /tmp dir inside the container is 9.3 GB in size, being entirely made up of gitlab-pages-<sometimestamp> directories (996 apparently).

After clearing out the /tmp dir it immediately starts filling up again, at approx. 1-2 gitlab-pages-<sometimestamp> directories per second!

I currently have one single repository which uses pages to publish a Sandcastle built library documentation on tagged builds. Since this has been established only 2 tags were added to the project.

The container is stopped and restarted daily by an automated maintenance procedure. I'm using GitLab CE 10.7.0.

What is the current bug behavior?

GitLab fills up the docker containers' /tmp directory with gitlab-pages-<sometimestamp> directories until the host disk runs out of space.

What is the expected correct behavior?

GitLab must not repeately spam files into the /tmp dir.

The /tmp dir contents should not grow endlessly but instead be cleaned up on a regular basis, at least on a container restart.

Also /tmp files should be removed when they are no longer needed.

Relevant logs and/or screenshots

Results of docker system df -v

image

Results of du -hsx /* | sort -rh | head -n 40 on the docker container root

Expand for output
root@gitlab:/# du -hsx /* | sort -rh | head -n 40
9.3G /tmp
1.4G /opt
130M /usr
24M /var
24M /lib
7.4M /bin
3.5M /sbin
2.2M /etc
32K /run
28K /assets
12K /root
4.0K /srv
4.0K /mnt
4.0K /media
4.0K /lib64
4.0K /home
4.0K /boot
4.0K /RELEASE
0 /sys
0 /proc
0 /dev

Results of du -hsx /* | sort -rh | head -n 40 on containers' /tmp dir

Expand for output
root@gitlab:/# du -hsx /tmp/* | sort -rh | head -n 20
9.6M /tmp/gitlab-pages-1524465583158923969
9.6M /tmp/gitlab-pages-1524465582125403367
9.6M /tmp/gitlab-pages-1524454211109324575
9.6M /tmp/gitlab-pages-1524454209747360893
9.6M /tmp/gitlab-pages-1524454208716069855
9.6M /tmp/gitlab-pages-1524454207684050608
9.6M /tmp/gitlab-pages-1524454206651237250
9.6M /tmp/gitlab-pages-1524454205383355475
9.6M /tmp/gitlab-pages-1524454204350860122
9.6M /tmp/gitlab-pages-1524454203312405992
9.6M /tmp/gitlab-pages-1524454202279725437
9.6M /tmp/gitlab-pages-1524454201247414286
9.6M /tmp/gitlab-pages-1524454200216047447
9.6M /tmp/gitlab-pages-1524454199185104114
9.6M /tmp/gitlab-pages-1524454198153812276
9.6M /tmp/gitlab-pages-1524454196871811221
9.6M /tmp/gitlab-pages-1524454195836429231
9.6M /tmp/gitlab-pages-1524454194805022391
9.6M /tmp/gitlab-pages-1524454193771707728
9.6M /tmp/gitlab-pages-1524454192737619754

Results of GitLab environment info

Expand for output related to GitLab environment info
System information
System:
Current User: git
Using RVM: no
Ruby Version: 2.3.6p384
Gem Version: 2.6.13
Bundler Version:1.13.7
Rake Version: 12.3.0
Redis Version: 3.2.11
Git Version: 2.14.3
Sidekiq Version:5.0.5
Go Version: unknown

GitLab information Version: 10.7.0 Revision: 2011c9b Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: https://gitlab.ad.haprotec.de HTTP Clone URL: https://gitlab.ad.haprotec.de/some-group/some-project.git SSH Clone URL: git@gitlab.ad.haprotec.de:some-group/some-project.git Using LDAP: yes Using Omniauth: no

GitLab Shell Version: 7.1.2 Repository storage paths:

  • default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check
Checking GitLab ...

Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 3/3 ... yes 3/4 ... yes 6/8 ... yes 6/9 ... yes 6/10 ... yes 9/11 ... yes 10/12 ... yes 8/13 ... yes 8/16 ... yes 10/17 ... yes 6/18 ... yes 6/19 ... yes 8/21 ... yes 11/22 ... yes 8/23 ... yes 9/24 ... yes 9/25 ... yes 9/26 ... yes 11/27 ... yes 15/28 ... yes 11/29 ... yes 16/30 ... yes 8/31 ... yes 17/32 ... yes 6/34 ... yes 9/35 ... yes 9/36 ... yes 11/37 ... yes 16/38 ... yes 16/39 ... yes 8/40 ... yes 6/41 ... yes 15/42 ... yes 9/43 ... yes 9/44 ... yes 27/45 ... yes 11/46 ... yes 6/47 ... yes 9/48 ... yes 11/49 ... yes 9/51 ... yes 22/53 ... yes 27/55 ... yes 9/56 ... yes 9/57 ... yes 9/58 ... yes 6/59 ... yes 29/60 ... yes 30/62 ... yes 29/63 ... yes 31/64 ... yes 32/65 ... yes 16/68 ... yes 15/69 ... yes 67/70 ... yes 32/71 ... yes 8/72 ... yes 69/73 ... yes 9/74 ... yes 16/75 ... yes 11/76 ... yes 11/77 ... yes 9/78 ... yes 9/79 ... yes 72/81 ... yes 71/82 ... yes 9/84 ... yes 22/85 ... yes 73/86 ... yes 75/87 ... yes 74/88 ... yes 75/89 ... yes 75/90 ... yes 76/93 ... yes 11/95 ... yes 9/96 ... yes 73/97 ... yes 29/98 ... yes 82/99 ... yes 82/100 ... yes 9/101 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.5 ? ... yes (2.3.6) Git version >= 2.9.5 ? ... yes (2.14.3) Git user has default SSH configuration? ... yes Active users: ... 22

Checking GitLab ... Finished

Edited by Alexander Rose