Make Puma enabled by default for everyone
We continue battle testing Puma and figuring out the best defaults that is reflect in number of issues and epics:
- enable of Puma on dev.gitlab.org: gitlab-com/gl-infra&82 (closed),
- enable of Puma on gitlab.com: gitlab-com/gl-infra&78 (closed)
This issue is to track the effort to make Puma on by default for everyone.
-
Ensure that this runs on
gitlab.com
for at least one full release, - Allow to update db_pool: omnibus-gitlab#4631 (closed)
- Update and close benchmarks: https://gitlab.com/gitlab-org/gitlab-ce/issues/62878, https://gitlab.com/gitlab-org/gitlab-ce/issues/62876
- Handle zero-downtime deployments with Puma: omnibus-gitlab#4739 (closed)
- Chronic Duration is not thread-safe: https://gitlab.com/gitlab-org/gitlab-ce/issues/66637
- Enable Puma fleet-wide on GitLab.com: gitlab-com/gl-infra/production#1684 (closed)
- Improve Puma request handling in multi-threaded mode: #36858 (closed)
- Improve graceful restart handling in Omnibus: omnibus-gitlab#4756 (closed)
- Do full restart of Puma during deployment: omnibus-gitlab#4928 (closed)
- Add optional Puma support to GitLab Chart: gitlab-org/charts/gitlab#1313 (closed)
- Upstream Puma multi-threading patches: #196002 (closed)
- Update relevant documentation: https://gitlab.com/gitlab-org/gitlab-ce/issues/66947
- Update Omnibus defaults: omnibus-gitlab#3896 (closed)
- Disable Rugged in Puma: #35371 (closed)
- Update HA documentation: omnibus-gitlab#5091 (closed)
- Ensure that Puma is on by default (Omnibus change): omnibus-gitlab#4698 (closed)
- Ensure that Puma is on by default (GitLab Chart change): gitlab-org/charts/gitlab#1908 (closed)
All issues with Multithreaded App Server: https://gitlab.com/groups/gitlab-org/-/issues?scope=all&utf8=%E2%9C%93&state=opened&label_name[]=Multithreaded%20App%20Server
Designs
- Show closed items
- Issue#35247014.81Category:Fuzz Testing GitLab Ultimate Technical Writing UX devops application security testing feature enhancement frontend fuzzing coverage group dynamic analysis section sec tw finished type feature workflow production
- Issue#35246514.81auto updated devops application security testing frontend section sec
- Issue#35207314.8Category:Fuzz Testing GitLab Ultimate backend devops application security testing feature enhancement fuzzing coverage group dynamic analysis section sec type bug workflow verification
- Issue#35163214.8Category:Fuzz Testing GitLab Ultimate devops application security testing feature enhancement frontend fuzzing coverage group dynamic analysis section sec type feature workflow verification
- Issue#35153914.8Category:Fuzz Testing GitLab Ultimate devops application security testing feature enhancement frontend fuzzing coverage group dynamic analysis section sec type feature workflow verification
- Issue#35097514.82Category:Fuzz Testing GitLab Ultimate devops application security testing feature enhancement frontend fuzzing coverage group dynamic analysis section sec type feature workflow verification
- Issue#35097414.82Category:Fuzz Testing GitLab Ultimate devops application security testing feature enhancement frontend fuzzing coverage group dynamic analysis section sec type feature workflow in dev
- Issue#34901414.8Category:Fuzz Testing GitLab Ultimate devops application security testing feature flag frontend fuzzing coverage group dynamic analysis missed:14.7 section sec type feature workflow in dev
- Issue#34748314.7Category:Fuzz Testing GitLab Ultimate devops application security testing feature enhancement frontend fuzzing coverage group dynamic analysis missed:14.6 section sec type bug workflow verification
- Issue#34509014.73Category:Fuzz Testing GitLab Ultimate devops application security testing feature enhancement frontend fuzzing coverage group dynamic analysis section sec type feature workflow verification
- Issue#34508914.73Category:Fuzz Testing auto updated devops application security testing frontend group dynamic analysis section sec workflow verification
- Issue#34508814.63Category:Fuzz Testing GitLab Ultimate devops application security testing feature enhancement frontend fuzzing coverage group dynamic analysis missed:14.5 section sec type feature workflow verification
- Issue#34148614.55Category:Fuzz Testing backend devops application security testing feature addition fuzzing coverage group dynamic analysis section sec type feature
- Issue#34148514.55Category:Fuzz Testing backend devops application security testing feature addition fuzzing coverage group dynamic analysis section sec type feature
- Issue#34148914.82Category:Fuzz Testing Technical Writing backend devops application security testing documentation feature addition fuzzing coverage group dynamic analysis missed:14.6 missed:14.7 section sec tw-weight 8 type feature
- Issue#34148714.65Category:Fuzz Testing backend devops application security testing feature enhancement fuzzing coverage group dynamic analysis section sec type feature
- Issue#34258014.62Category:Fuzz Testing GitLab Ultimate devops application security testing feature enhancement frontend fuzzing coverage group dynamic analysis missed:14.5 section sec type feature workflow verification
- Issue#34258314.82Category:Fuzz Testing GitLab Ultimate devops application security testing feature enhancement frontend fuzzing coverage group dynamic analysis missed:14.6 missed:14.7 section sec type feature workflow verification
- Issue#34243314.63Category:Fuzz Testing GitLab Ultimate devops application security testing feature enhancement frontend fuzzing coverage group dynamic analysis missed:14.5 section sec type feature workflow verification
- Issue#34133814.4automation:ml devops package group package registry section ops
- Issue#34121814.6Category:Package Registry Generic Package Repository auto updated devops package fuzzing coverage group package registry missed:14.4 missed:14.5 section ops
- Epicgitlab-org#668323Dec 18, 2021 – Jan 17, 2022Category:Fuzz Testing backend devops create frontend fuzzing coverage group dynamic analysis missed:14.2 section sec
- Issue#33973214.5Category:Fuzz Testing GitLab Ultimate backend devops application security testing direction fuzzing coverage group dynamic analysis missed:14.3 missed:14.4 section sec type feature workflow in dev
- Epicgitlab-org#640647Jul 18 – Aug 17, 2021Category:Fuzz Testing Deliverable GitLab Ultimate devops application security testing frontend fuzzing coverage group dynamic analysis section sec type feature
- Issue#33494114.7Category:Fuzz Testing Technical Writing backend devops application security testing documentation fuzzing coverage group dynamic analysis missed:14.2 missed:14.3 missed:14.4 missed:14.5 missed:14.6 section sec tw-weight 5 workflow refinement
- Issue#27316914.2Category:Fuzz Testing devops application security testing fuzzing coverage group dynamic analysis missed:13.8 section sec
- Issue#32689513.113Category:Fuzz Testing Deliverable GitLab Ultimate devops application security testing direction frontend fuzzing coverage group dynamic analysis missed-deliverable missed:13.10 section sec type feature workflow in review
- Issue#32689613.112Category:Fuzz Testing Deliverable GitLab Ultimate devops application security testing direction frontend fuzzing coverage group dynamic analysis missed-deliverable missed:13.10 section sec type feature workflow in review
- Issue#32689413.111Category:Fuzz Testing Deliverable GitLab Ultimate devops application security testing direction frontend fuzzing coverage group dynamic analysis missed-deliverable missed:13.10 section sec type feature workflow verification
- Issue#28059213.112Category:Fuzz Testing Deliverable GitLab Ultimate backend devops application security testing direction fuzzing coverage group dynamic analysis section sec type feature workflow in dev
- Issue#32636113.113Category:Fuzz Testing Deliverable GitLab Ultimate backend devops application security testing direction fuzzing coverage group dynamic analysis section sec type feature workflow in dev
- Issue#32636213.112Category:Fuzz Testing Deliverable GitLab Ultimate backend devops application security testing direction fuzzing coverage group dynamic analysis section sec type feature workflow in dev
- Issue#29444413.113Category:Fuzz Testing Deliverable GitLab Ultimate devops application security testing direction frontend fuzzing coverage group dynamic analysis missed-deliverable missed:13.10 section sec type feature workflow verification
- Issue#32161814.3Category:Fuzz Testing backend devops create frontend fuzzing coverage group dynamic analysis missed:14.2 section sec
- Issue#32789714.52Category:Fuzz Testing GitLab Ultimate devops application security testing frontend fuzzing coverage group dynamic analysis section sec type feature workflow production
- Issue#32088613.103Category:Fuzz Testing GitLab Ultimate devops application security testing direction frontend fuzzing coverage group dynamic analysis section sec workflow verification
- Issue#29954413.101Category:Fuzz Testing Deferred UX Deliverable UX devops create frontend group dynamic analysis workflow in dev
- Issue#29442513.92Category:Fuzz Testing GitLab Ultimate backend devops application security testing direction frontend fuzzing coverage group dynamic analysis section sec type feature workflow production
- Issue#28059314.32Category:Fuzz Testing GitLab Ultimate backend devops application security testing direction frontend fuzzing coverage group dynamic analysis section sec type feature workflow production
- Issue#28059013.9Category:Fuzz Testing backend devops application security testing fuzzing coverage group dynamic analysis missed:13.7 section sec type feature workflow ready for development
- Issue#32841814.8Category:Fuzz Testing GitLab Ultimate devops application security testing feature flag frontend fuzzing coverage group dynamic analysis missed:14.7 section sec type feature
- Issue#29074113.10Category:Fuzz Testing Deliverable devops application security testing fuzzing coverage group dynamic analysis missed-deliverable missed:13.8 section sec workflow verification
- Issue#268004Category:Fuzz Testing GitLab Ultimate UX FY21-Q4 backend devops application security testing direction feature enhancement frontend fuzzing coverage group dynamic analysis section sec secure:refinement-backend secure:refinement-frontend type feature workflow refinement
- Issue#32847514.2Category:Fuzz Testing Deliverable GitLab Ultimate devops application security testing frontend fuzzing coverage group dynamic analysis section sec type feature workflow refinement
Relates to
- omnibus-gitlab #389612.9
Activity
-
Newest first Oldest first
-
Show all activity Show comments only Show history only
- Kamil Trzciński added to epic &954 (closed)
added to epic &954 (closed)
- Kamil Trzciński changed milestone to %12.6
changed milestone to %12.6
- Kamil Trzciński changed the description
changed the description
- Maintainer
Hi @ayufan,
Please add labels to your issue, this aids categorization and locating issues in the future.
Thanks for your help!
You are welcome to help improve this comment.
- 🤖 GitLab Bot 🤖 added auto updated + 1 deleted label
added auto updated + 1 deleted label
- 🤖 GitLab Bot 🤖 added [deprecated] Accepting merge requests + 1 deleted label
added [deprecated] Accepting merge requests + 1 deleted label
- Kamil Trzciński marked the checklist item Chronic Duration is not thread-safe: https://gitlab.com/gitlab-org/gitlab-ce/issues/66637 as completed
marked the checklist item Chronic Duration is not thread-safe: https://gitlab.com/gitlab-org/gitlab-ce/issues/66637 as completed
- Kamil Trzciński added Category:Omnibus Package Multithreaded App Server backend + 1 deleted label and removed [deprecated] Accepting merge requests + 1 deleted label
added Category:Omnibus Package Multithreaded App Server backend + 1 deleted label and removed [deprecated] Accepting merge requests + 1 deleted label
- 🤖 GitLab Bot 🤖 changed milestone to %12.6
changed milestone to %12.6
- 🤖 GitLab Bot 🤖 added [deprecated] Accepting merge requests + 1 deleted label
added [deprecated] Accepting merge requests + 1 deleted label
- Maintainer
Setting devopsenablement groupdistribution based on ~"Omnibus".
- 🤖 GitLab Bot 🤖 added devopssystems groupdistribution + 1 deleted label
added devopssystems groupdistribution + 1 deleted label
- Stan Hu marked this issue as related to #32562 (closed)
marked this issue as related to #32562 (closed)
- Kamil Trzciński changed the description
changed the description
- Kamil Trzciński marked this issue as related to omnibus-gitlab#4739 (closed)
marked this issue as related to omnibus-gitlab#4739 (closed)
- Kamil Trzciński marked this issue as related to omnibus-gitlab#4631 (closed)
marked this issue as related to omnibus-gitlab#4631 (closed)
- Kamil Trzciński marked this issue as related to omnibus-gitlab#3896 (closed)
marked this issue as related to omnibus-gitlab#3896 (closed)
- Author MaintainerEdited by Kamil Trzciński
Collapse replies - Contributor
@ayufan - WDYT about this timeline?
- Enable on gitlab.com in release X
- Set as opt-in in Omnibus/Charts in release X. Announce Unicorn as deprecated in X+3, will likely become the default next release.
- Provided gitlab.com is stable, set as opt-out for Omnibus/Charts in release X+1.
- Remove the opt-out flag in X+3, provided feedback has been positive.
cc @ljlane
Edited by Joshua Lambert - Author Maintainer
I invited you to Puma Sync meeting. Lets discuss the timeline, zero-downtime and steps to finalise that story tomorrow.
1 - Contributor
@joshlambert Where did we land on your proposed timeline above? We have Puma added to Omnibus and working on an MR to add it to the charts.
Set as opt-in in Omnibus/Charts in release X. Announce Unicorn as deprecated in X+3, will likely become the default next release.
If Puma becomes opt-in in the charts in 12.9, the above proposal will have it becoming the default by %13.2 rather than the original goal of %13.0
Provided gitlab.com is stable, set as opt-out for Omnibus/Charts in release X+1
Is gitlab.com considered stable now?
- Craig Gomes mentioned in epic gitlab-com/gl-infra&78 (closed)
mentioned in epic gitlab-com/gl-infra&78 (closed)
- Kamil Trzciński marked the checklist item Allow to update db_pool: omnibus-gitlab#4631 (closed), as completed
marked the checklist item Allow to update db_pool: omnibus-gitlab#4631 (closed), as completed
- Kamil Trzciński mentioned in issue #35371 (closed)
mentioned in issue #35371 (closed)
- Kamil Trzciński changed the description
changed the description
- Kamil Trzciński mentioned in issue omnibus-gitlab#3896 (closed)
mentioned in issue omnibus-gitlab#3896 (closed)
- Contributor
This epic currently has groupdistribution and %12.6 but it's unclear to me what needs to be done by the Distribution team for %12.6. @craig-gomes @ayufan can you comment?
Collapse replies - Author Maintainer
I don't think that %12.6 is valid, but it seems the wrap-up of
on by default
is for groupdistribution as it requires:- updating defaults in Omnibus,
- updating HA documentation how to use new health-checks,
- deprecate Unicorn,
- adapt Cloud-Native to offer and use Puma as well,
- likely this should be done in %13.0
- Contributor
Okay, we can get issues created and targeted for %13.0. Are you aware of anything that is blocking groupdistribution from moving forward with this besides the milestone?
- Contributor
@ayufan @twk3 @WarheadsSE What is the user impact of enabling Puma by default? I'm referring to negative impact as in any manual changes required when upgrading. And the reason I'm asking is to understand whether this change needs to happen in a major release of GitLab.
- Author Maintainer
If we change default unless user made their own modifications to
unicron['
it will not be noticeable change, as application will restart to usePuma
. There are the following problems:- If user made their own changes to
unicorn['
it will break, aspuma['
has it's own config, - It affects HA: we do not offer zero-downtime deployments for Puma for single node,
- To have HA: we require to configure health-check and perform rolling update of GitLab.
No, I don't think that we can make this change in minor release. It is too big of a change to do it like this. We would require to:
- Start using health-check for having HA config,
- Update
puma['
to reflectunicorn['
.
I think that it is simply too big of a change.
cc @craig-gomes
- If user made their own changes to
- Developer
@ljlane Effectively, very similar within the Helm charts. We'd have items in the CheckConfig pattern to watch for having Unicorn configured, but Puma in use. Most of the settings will be similar, and even translate without interaction. Some settings, like worker processes, threads, and resulting memory configuration.
- Contributor
When we talk about enabling Puma by default are we talking about just new installs or upgrades too? Maybe I'm not seeing the big picture here, but if it's an upgrade couldn't we just leave their unicorn settings in place and recommend they move to Puma some time in the future? Unicorn will be supported until 14.0.
In absence of previous settings, then it would be Puma by default.
- Contributor
@craig-gomes I think we need to enable it on upgrades as well, in an opt-in fashion in 13.0. Otherwise we have to wait an entire year, which means we will have to support two different architectures for a long long time.
But we if we are concerned about the timeline (it is definitely fast) then let's have that discussion about alternative plans.
- Larissa Lane changed milestone to %13.0
changed milestone to %13.0
- Larissa Lane added Deliverable + 1 deleted label
added Deliverable + 1 deleted label
- Larissa Lane added priority1 + 1 deleted label
added priority1 + 1 deleted label
- Craig Gomes marked the checklist item Update and close benchmarks: https://gitlab.com/gitlab-org/gitlab-ce/issues/62878, https://gitlab.com/gitlab-org/gitlab-ce/issues/62876. as completed
marked the checklist item Update and close benchmarks: https://gitlab.com/gitlab-org/gitlab-ce/issues/62878, https://gitlab.com/gitlab-org/gitlab-ce/issues/62876. as completed
- Craig Gomes marked the checklist item Handle zero-downtime deployments with Puma: omnibus-gitlab#4739 (closed) as completed
marked the checklist item Handle zero-downtime deployments with Puma: omnibus-gitlab#4739 (closed) as completed
- 🤖 GitLab Bot 🤖 removed Deliverable + 1 deleted label
removed Deliverable + 1 deleted label
- Kamil Trzciński marked the checklist item Ensure that this runs on
gitlab.com
for at least one full release, as completedmarked the checklist item Ensure that this runs on
gitlab.com
for at least one full release, as completed - Kamil Trzciński changed the description
Compare with previous version changed the description
- Kamil Trzciński marked the checklist item Improve graceful restart handling in Omnibus: omnibus-gitlab#4756 (closed) as completed
marked the checklist item Improve graceful restart handling in Omnibus: omnibus-gitlab#4756 (closed) as completed
- Kamil Trzciński marked the checklist item Improve graceful restart handling in Omnibus: omnibus-gitlab#4756 (closed) as incomplete
marked the checklist item Improve graceful restart handling in Omnibus: omnibus-gitlab#4756 (closed) as incomplete
- Kamil Trzciński marked the checklist item Improve Puma request handling in multi-threaded mode: #36858 (closed) as completed
marked the checklist item Improve Puma request handling in multi-threaded mode: #36858 (closed) as completed
- Kamil Trzciński marked the checklist item Do full restart of Puma during deployment: omnibus-gitlab#4928 (closed) as completed
marked the checklist item Do full restart of Puma during deployment: omnibus-gitlab#4928 (closed) as completed
- Kamil Trzciński marked the checklist item Improve graceful restart handling in Omnibus: omnibus-gitlab#4756 (closed) as completed
marked the checklist item Improve graceful restart handling in Omnibus: omnibus-gitlab#4756 (closed) as completed
- Kamil Trzciński changed the description
Compare with previous version changed the description
- Kamil Trzciński marked this issue as related to gitlab-com/gl-infra/production#1684 (closed)
marked this issue as related to gitlab-com/gl-infra/production#1684 (closed)
- Kamil Trzciński marked this issue as related to #36858 (closed)
marked this issue as related to #36858 (closed)
- Kamil Trzciński marked this issue as related to omnibus-gitlab#4756 (closed)
marked this issue as related to omnibus-gitlab#4756 (closed)
- Kamil Trzciński marked this issue as related to omnibus-gitlab#4928 (closed)
marked this issue as related to omnibus-gitlab#4928 (closed)
- Kamil Trzciński marked this issue as related to gitlab-org/charts/gitlab#1313 (closed)
marked this issue as related to gitlab-org/charts/gitlab#1313 (closed)
- Kamil Trzciński marked this issue as related to #196002 (closed)
marked this issue as related to #196002 (closed)
- Kamil Trzciński marked this issue as related to #35371 (closed)
marked this issue as related to #35371 (closed)
- Kamil Trzciński marked this issue as related to omnibus-gitlab#5091 (closed)
marked this issue as related to omnibus-gitlab#5091 (closed)
- Kamil Trzciński marked this issue as related to omnibus-gitlab#4698 (closed)
marked this issue as related to omnibus-gitlab#4698 (closed)
- Kamil Trzciński marked this issue as related to gitlab-org/charts/gitlab#1908 (closed)
marked this issue as related to gitlab-org/charts/gitlab#1908 (closed)
- Larissa Lane mentioned in issue gitlab-org/distribution/team-tasks#585 (closed)
mentioned in issue gitlab-org/distribution/team-tasks#585 (closed)
- Craig Gomes marked the checklist item Update relevant documentation: https://gitlab.com/gitlab-org/gitlab-ce/issues/66947 as completed
marked the checklist item Update relevant documentation: https://gitlab.com/gitlab-org/gitlab-ce/issues/66947 as completed
- Larissa Lane added Deliverable + 1 deleted label
added Deliverable + 1 deleted label
- 🤖 GitLab Bot 🤖 removed Deliverable + 1 deleted label
removed Deliverable + 1 deleted label
- Craig Gomes marked the checklist item Add optional Puma support to GitLab Chart: gitlab-org/charts/gitlab#1313 (closed) as completed
marked the checklist item Add optional Puma support to GitLab Chart: gitlab-org/charts/gitlab#1313 (closed) as completed
- Craig Gomes marked the checklist item Disable Rugged in Puma: #35371 (closed) as completed
marked the checklist item Disable Rugged in Puma: #35371 (closed) as completed
- Craig Gomes marked the checklist item Update Omnibus defaults: omnibus-gitlab#3896 (closed) as completed
marked the checklist item Update Omnibus defaults: omnibus-gitlab#3896 (closed) as completed
- Larissa Lane added Deliverable + 1 deleted label
added Deliverable + 1 deleted label
- Contributor
@ayufan I'm taking groupdistribution off this one because we are using omnibus-gitlab#4698 (closed) and gitlab-org/charts/gitlab#1908 (closed) to track our work for %13.0
Collapse replies - Author Maintainer
How we can monitor progress? This was more like a join of different issues. Instead of removing the label I would rather reconsider, as then this issue becomes orphaned and:
- keeping this as a SSOT on getting into
on by default
- close this one and mark it as irrelevant/duplicate-of if we have a better ways to track that
- @gitlab-bot seems to also disagree with you to remove the label, as it has Omnibus Package :)
- keeping this as a SSOT on getting into
- Contributor
One option - close this in favor of tracking under &954 (closed). All of the open issues listed in this issue are under the &954 (closed) epic or sub-epic.
- Contributor
Grouping everything under an epic that is a SSOT sounds like a good plan so that nothing gets orphaned. My main goal is to remove this one from our %13.0 board because the duplication causes some confusion and looks like we have more issues scheduled than we really do. We get it off our board by either removing groupdistribution or removing the milestone.
- Contributor
The other option is to promote this to a sub-epic of &954 (closed) since this has a specific theme where &954 (closed) is a more general epic for Puma roll out.
- Contributor
I'm going ahead with this last change you suggested so the bot will stop putting it back on our board
- Larissa Lane removed devopssystems groupdistribution + 1 deleted label
removed devopssystems groupdistribution + 1 deleted label
- Maintainer
Setting devopsenablement groupdistribution based on Category:Omnibus Package.
- 🤖 GitLab Bot 🤖 added devopssystems groupdistribution + 1 deleted label
added devopssystems groupdistribution + 1 deleted label
- Larissa Lane promoted to epic &3180 (closed)
promoted to epic &3180 (closed)
- Larissa Lane removed devopssystems groupdistribution labels
removed devopssystems groupdistribution labels
- Craig Gomes mentioned in issue omnibus-gitlab#5489 (closed)
mentioned in issue omnibus-gitlab#5489 (closed)
- Sam Goldstein mentioned in issue gitlab-com/www-gitlab-com#9140 (closed)
mentioned in issue gitlab-com/www-gitlab-com#9140 (closed)