Add custom pluralization to FastGettext
What does this MR do and why?
This speeds up pluralization when using n_(...)
by 17x.
It also side-steps any potential security concerns related to eval
in FastGettext
.
Example failure message
Each locales must have a pluralization mapping otherwise the specs will fail with the following error:
2) Gitlab::I18n::Pluralization.call with available locales locale: "de" supports pluralization
Failure/Error:
raise ArgumentError, <<~MESSAGE
Missing pluralization rule for locale #{locale.inspect}.
#{suggestion}
MESSAGE
ArgumentError:
Missing pluralization rule for locale "de".
Add the following line to /home/peter/devel/gitlab/gdk/gitlab/lib/gitlab/i18n/pluralization.rb:
MAP = {
...
"de" => ->(n) { (n != 1) },
...
}.freeze
This rule was extracted from /home/peter/devel/gitlab/gdk/gitlab/locale/de/gitlab.po.
# ./lib/gitlab/i18n/pluralization.rb:53:in `block in <module:Pluralization>'
# ./lib/gitlab/i18n/pluralization.rb:63:in `fetch'
# ./lib/gitlab/i18n/pluralization.rb:63:in `call'
# ./spec/lib/gitlab/i18n/pluralization_spec.rb:9:in `block (3 levels) in <top (required)>'
# ./spec/lib/gitlab/i18n/pluralization_spec.rb:22:in `block (5 levels) in <top (required)>'
# ./lib/gitlab/i18n.rb:107:in `with_locale'
# ./spec/lib/gitlab/i18n/pluralization_spec.rb:13:in `block (4 levels) in <top (required)>'
# ./spec/support/system_exit_detected.rb:7:in `block (2 levels) in <top (required)>'
Benchmark
Comparison:
plural fast: 717904.2 i/s
plural slow: 40457.3 i/s - 17.74x slower
Code + Results
# frozen_string_literal: true
require "benchmark/ips"
Gitlab::I18n::Pluralization.install! if ENV['FAST']
p FastGettext.pluralisation_rule
FastGettext.locale = "pl_PL"
p n_("%d day", "%d days", 0)
p n_("%d day", "%d days", 1)
p n_("%d day", "%d days", 5)
Benchmark.ips do |x|
name = "plural #{ENV['FAST'] ? 'fast' : 'slow'}"
x.report name do
n_("day", "days", 1)
end
x.compare!
x.save! 'fast_gettext'
end
$ bin/rails runner foo.rb
WARNING: This version of GitLab depends on gitlab-shell 14.17.0, but you're running 14.15.0. Please update gitlab-shell.
#<Proc:0x0000565246e3f6e0 /home/peter/.dotfiles/asdf/installs/ruby/3.0.5/lib/ruby/gems/3.0.0/gems/fast_gettext-2.1.0/lib/fast_gettext/storage.rb:17 (lambda)>
"%d dni"
"%d dzień"
"%d dni"
Warming up --------------------------------------
plural slow 4.125k i/100ms
Calculating -------------------------------------
plural slow 40.457k (±15.2%) i/s - 193.875k in 5.045078s
Comparison:
plural fast: 512635.9 i/s
plural slow: 40457.3 i/s - 12.67x slower
peter@happy ~/devel/gitlab/gdk/gitlab on (pl-gettext-pluralization ?M)
$ FAST=1 bin/rails runner foo.rb
WARNING: This version of GitLab depends on gitlab-shell 14.17.0, but you're running 14.15.0. Please update gitlab-shell.
Gitlab::I18n::Pluralization
"%d dni"
"%d dzień"
"%d dni"
Warming up --------------------------------------
plural fast 65.010k i/100ms
Calculating -------------------------------------
plural fast 717.904k (±11.2%) i/s - 3.576M in 5.040367s
Comparison:
plural fast: 717904.2 i/s
plural slow: 40457.3 i/s - 17.74x slower
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Peter Leitzen