Commit 5830d80b authored by Yorick Peterse's avatar Yorick Peterse

Pre-calculate Emoji digests

By pre-calculating the digests we can manually construct the emoji URLs,
removing the need for using Rails' asset URL helpers. The reason we
don't want to use these helpers for Emojis is two-fold:

1. Rails' image_url() method is slow, really slow. For one it _might_
   have to calculate digests but it also performs a lot of other
   intensive operations (judging by the source code and based on
   measuring timings).

2. We have a lot of Emoji which coupled with the above can result in it
   taking minutes to load Emoji autocomplete data.

Using this pre-calculation setup generating the digests takes around 7
seconds (including the time it takes to start Rails/Rake), and only
around 600 milliseconds to load _all_ the autocomplete data of a project
(measured locally).

This commit _does_ change the Emoji URLs from absolute to relative URLs
as these are much easier to generate.

To update the Emoji data simply run:

    rake gemojione:digests

Then commit any changes.

Fixes gitlab-org/gitlab-ce#14009
parent 8718acdf
Pipeline #1246194 passed with stage
...@@ -138,7 +138,7 @@ class ProjectsController < Projects::ApplicationController ...@@ -138,7 +138,7 @@ class ProjectsController < Projects::ApplicationController
participants = ::Projects::ParticipantsService.new(@project, current_user).execute(note_type, note_id) participants = ::Projects::ParticipantsService.new(@project, current_user).execute(note_type, note_id)
@suggestions = { @suggestions = {
emojis: autocomplete_emojis, emojis: AwardEmoji.urls,
issues: autocomplete.issues, issues: autocomplete.issues,
mergerequests: autocomplete.merge_requests, mergerequests: autocomplete.merge_requests,
members: participants members: participants
...@@ -235,17 +235,6 @@ class ProjectsController < Projects::ApplicationController ...@@ -235,17 +235,6 @@ class ProjectsController < Projects::ApplicationController
) )
end end
def autocomplete_emojis
Rails.cache.fetch("autocomplete-emoji-#{Gemojione::VERSION}") do
Emoji.emojis.map do |name, emoji|
{
name: name,
path: view_context.image_url("#{emoji["unicode"]}.png")
}
end
end
end
def repo_exists? def repo_exists?
project.repository_exists? && !project.empty_repo? project.repository_exists? && !project.empty_repo?
end end
......
This diff is collapsed.
...@@ -48,4 +48,23 @@ class AwardEmoji ...@@ -48,4 +48,23 @@ class AwardEmoji
JSON.parse(File.read(json_path)) JSON.parse(File.read(json_path))
end end
end end
# Returns an Array of Emoji names and their asset URLs.
def self.urls
@urls ||= begin
path = File.join(Rails.root, 'fixtures', 'emojis', 'digests.json')
prefix = Gitlab::Application.config.assets.prefix
digest = Gitlab::Application.config.assets.digest
JSON.parse(File.read(path)).map do |hash|
if digest
fname = "#{hash['unicode']}-#{hash['digest']}"
else
fname = hash['unicode']
end
{ name: hash['name'], path: "#{prefix}/#{fname}.png" }
end
end
end
end end
# This task will generate a standard and Retina sprite of all of the current
# Gemojione Emojis, with the accompanying SCSS map.
#
# It will not appear in `rake -T` output, and the dependent gems are not
# included in the Gemfile by default, because this task will only be needed
# occasionally, such as when new Emojis are added to Gemojione.
begin
require 'sprite_factory'
require 'rmagick'
rescue LoadError
# noop
end
namespace :gemojione do namespace :gemojione do
desc 'Generates Emoji SHA256 digests'
task digests: :environment do
require 'digest/sha2'
require 'json'
dir = Gemojione.index.images_path
digests = AwardEmoji.emojis.map do |name, emoji_hash|
fpath = File.join(dir, "#{emoji_hash['unicode']}.png")
digest = Digest::SHA256.file(fpath).hexdigest
{ name: name, unicode: emoji_hash['unicode'], digest: digest }
end
out = File.join(Rails.root, 'fixtures', 'emojis', 'digests.json')
File.open(out, 'w') do |handle|
handle.write(JSON.pretty_generate(digests))
end
end
# This task will generate a standard and Retina sprite of all of the current
# Gemojione Emojis, with the accompanying SCSS map.
#
# It will not appear in `rake -T` output, and the dependent gems are not
# included in the Gemfile by default, because this task will only be needed
# occasionally, such as when new Emojis are added to Gemojione.
task sprite: :environment do task sprite: :environment do
begin
require 'sprite_factory'
require 'rmagick'
rescue LoadError
# noop
end
check_requirements! check_requirements!
SIZE = 20 SIZE = 20
......
require 'spec_helper'
describe AwardEmoji do
describe '.urls' do
subject { AwardEmoji.urls }
it { is_expected.to be_an_instance_of(Array) }
it { is_expected.to_not be_empty }
context 'every Hash in the Array' do
it 'has the correct keys and values' do
subject.each do |hash|
expect(hash[:name]).to be_an_instance_of(String)
expect(hash[:path]).to be_an_instance_of(String)
end
end
end
end
end
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment