Update Puma to 6.4.3 and patch PID reaper
What does this MR do and why?
Puma v6.4.1 was reverted in #437948 (closed) since it exposed a PID-reaping bug in Ruby 3.1.4 (https://bugs.ruby-lang.org/issues/19837). Full details are in https://github.com/puma/puma/issues/3313.
Unfortunately Ruby 3.1.5 and 3.2.5 still have a bug that prevents Puma from reaping child PIDs properly (https://bugs.ruby-lang.org/issues/20490), so let's pull in the patch in https://github.com/puma/puma/pull/3314 since a Puma release has not been forthcoming.
This update fixes a minor security issue and other bugs: https://my.diffend.io/gems/puma/6.4.0/6.4.3
Related incidents:
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
How to set up and validate locally
GDK
- Check out this branch.
gdk restart rails-web
- When Puma starts, kill one of the
cluster worker
jobs:
% ps -ef | grep "cluster worker"
501 65022 64509 0 10:08PM ?? 0:00.38 puma: cluster worker 0: 64509 [gitlab-puma-worker]
501 65378 64509 0 10:08PM ?? 0:00.40 puma: cluster worker 1: 64509 [gitlab-puma-worker]
501 79824 7003 0 10:17PM ttys004 0:00.00 grep cluster worker
% kill -9 65378
You should see a new PID take its place immediately:
% ps -ef | grep "cluster worker"
501 65022 64509 0 10:08PM ?? 0:00.39 puma: cluster worker 0: 64509 [gitlab-puma-worker]
501 80175 64509 0 10:18PM ?? 0:00.37 puma: cluster worker 1: 64509 [gitlab-puma-worker]
501 80213 7003 0 10:18PM ttys004 0:00.00 grep cluster worker
With Cloud Native GitLab
- Build a
webservice
container with Puma 6.4.3. I used thisDockerfile
:
FROM registry.gitlab.com/gitlab-org/build/cng/gitlab-webservice-ee:master
USER root
RUN apt update && apt install -y build-essential && cd /srv/gitlab && sed -i -e "s/= 6.4.0/= 6.4.3/" Gemfile && bundle config set frozen false && bundle install && bundle config set frozen true
COPY config/initializers/puma_patch.rb /srv/gitlab/config/initializers/puma_patch.rb
USER git
- Push this image to some public location. For example:
docker build -t registry.gitlab.com/stanhu/lfs-test/webservice-ee:latest .
docker push registry.gitlab.com/stanhu/lfs-test/webservice-ee:latest
If you are running this from your gitlab
dir, be aware that .dockerignore
may interfere with the above.
- Install the Helm Chart with this image:
helm upgrade --install gitlab . \
--set gitlab.webservice.image.pullPolicy=Always \
--set gitlab.webservice.image.repository=registry.gitlab.com/stanhu/lfs-test/webservice-ee \
--set gitlab.webservice.image.tag=latest
- When the
webservice
pod comes up, log into it and verify that Puma 6.4.3 is running:
git@gitlab-webservice-default-66f688f648-g6prb:/$ ps -ef
UID PID PPID C STIME TTY TIME CMD
git 1 0 87 20:09 ? 00:00:39 puma 6.4.2 (tcp://0.0.0.0:8080) [gitlab-puma-worker]
git 21 1 0 20:09 ? 00:00:00 /usr/local/bin/gitlab-logger /var/log/gitlab
git 68 0 0 20:09 pts/0 00:00:00 bash
git 81 1 65 20:09 ? 00:00:02 ruby /srv/gitlab/bin/metrics-server
git 84 1 22 20:09 ? 00:00:00 puma: cluster worker 0: 1 [gitlab-puma-worker]
git 86 1 16 20:09 ? 00:00:00 puma: cluster worker 1: 1 [gitlab-puma-worker]
git 121 68 0 20:09 pts/0 00:00:00 ps -ef
- Then start killing the
cluster worker
PIDs and verify they are reaped. Without the patch, you will see a zombie process:
git@gitlab-webservice-default-66f688f648-g6prb:/$ kill -9 86
git@gitlab-webservice-default-66f688f648-g6prb:/$ ps -ef
UID PID PPID C STIME TTY TIME CMD
git 1 0 45 20:09 ? 00:00:39 puma 6.4.2 (tcp://0.0.0.0:8080) [gitlab-puma-worker]
git 21 1 0 20:09 ? 00:00:00 /usr/local/bin/gitlab-logger /var/log/gitlab
git 68 0 0 20:09 pts/0 00:00:00 bash
git 81 1 4 20:09 ? 00:00:02 ruby /srv/gitlab/bin/metrics-server
git 84 1 1 20:09 ? 00:00:00 puma: cluster worker 0: 1 [gitlab-puma-worker]
git 86 1 1 20:09 ? 00:00:00 [ruby] <defunct>
git 127 68 0 20:10 pts/0 00:00:00 ps -ef
With the patch, it works:
git@gitlab-webservice-default-6887959876-mg9mb:/srv/gitlab/config/initializers$ ps -ef | grep puma
git 1 0 88 05:12 ? 00:00:32 puma 6.4.3 (tcp://0.0.0.0:8080) [gitlab-puma-worker]
git 53 1 19 05:13 ? 00:00:00 puma: cluster worker 0: 1 [gitlab-puma-worker]
git 55 1 19 05:13 ? 00:00:00 puma: cluster worker 1: 1 [gitlab-puma-worker]
git 89 35 0 05:13 pts/0 00:00:00 grep puma
git@gitlab-webservice-default-6887959876-mg9mb:/srv/gitlab/config/initializers$ kill 55
git@gitlab-webservice-default-6887959876-mg9mb:/srv/gitlab/config/initializers$ ps -ef
UID PID PPID C STIME TTY TIME CMD
git 1 0 80 05:12 ? 00:00:32 puma 6.4.3 (tcp://0.0.0.0:8080) [gitlab-puma-worker]
git 21 1 0 05:12 ? 00:00:00 /usr/local/bin/gitlab-logger /var/log/gitlab
git 35 0 0 05:13 pts/0 00:00:00 bash
git 50 1 29 05:13 ? 00:00:01 ruby /srv/gitlab/bin/metrics-server
git 53 1 7 05:13 ? 00:00:00 puma: cluster worker 0: 1 [gitlab-puma-worker]
git 92 1 29 05:13 ? 00:00:00 puma: cluster worker 1: 1 [gitlab-puma-worker]
git 109 35 0 05:13 pts/0 00:00:00 ps -ef
Merge request reports
Activity
assigned to @stanhu
added pipelinetier-1 label
- Resolved by Stan Hu
Dependency change review report
Please wait to merge until below tasks are completed by @gitlab-com/gl-security/appsec
- Review metadata report and resolve this thread. Review guidelines are at handbook page
Click to view metadata analysis report
Modified Dependency: puma (6.4.3) Location:
Gemfile.lock
Version diffsChecks passed: 6/6
-
️ Latest version: 6.4.3 released on: 2024-09-19. URL: https://rubygems.org/gems/puma -
Latest version is in use. -
Total downloads: 391600446 -
Reverse dependencies: 570 -
Total number of releases: 329 -
Latest version age (months): 0 -
Source project in Github not archived. -
️ Maintainer emails are private. Email domain check skipped.
This automation is under testing, please leave your feedback in the issue.
changed milestone to %17.5
- A deleted user
added backend label
Reviewer roulette
Category Reviewer Maintainer backend @tyleramos
(UTC-4, 3 hours ahead of author)
@jpcyiza
(UTC+2, 9 hours ahead of author)
Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.
Rubygems
This merge request adds, or changes a Rubygems dependency. Please review the Gemfile guidelines.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
DangerEdited by Ghost Useradded pipeline:skip-undercoverage label and removed backend label
- A deleted user
added backend label
- Resolved by 🤖 GitLab Bot 🤖
Proper labels assigned to this merge request. Please ignore me.
@stanhu
- please see the following guidance and update this merge request.1 Error Please add typebug typefeature, or typemaintenance label to this merge request. Edited by 🤖 GitLab Bot 🤖
added bugfunctional typebug labels
mentioned in issue #492463
requested review from @alejandro
mentioned in incident gitlab-com/gl-infra/production#18591 (closed)