Draft: POC - Image Resizing: Resizing via Sidekiq

What does this MR do

This MR allows us to produce resized versions of the same avatar image in the background (sidekiq) process, instead of resizing images on request.

Carrierwave will handle versions, we don't need to store anything related to version paths in the database.

• It stores the original file with no processing/versioning, which is as fast as before
• It uploads the original file to remote storage using Direct upload (if object_storage is enabled) and Workhorse acceleration, the same as before.
• It process, and store version uploads in the background (sidekiq worker).
• It serves the resized image version automatically when the width parameter is provided.
• If the resized image version is not available, it will serve the original image, and it will recreate all versions in the background sidekiq job. Next time a resized image is requested, it will serve a resized image.
• It recreates image versions in the background when a new avatar is uploaded or when one of the supported image sizes is requested and not found.
• It introduces :until_executed deduplication strategy which locks until the previously scheduled job finishes (gitlab-com/gl-infra/scalability#195 (closed)). This will prevent resize background jobs to be running simultaneously multiple times.

Supported Image sizes

Model	sizes
User	[40, 24, 23, 20, 16, 26]
Project	[48, 15, 40, 64]
Group	[15, 38, 64]

I decided to support only the most popular image sizes based on: #227388 (comment 391901232)

Implementation details

How to add support for image versions for any existing image

In order to add support for various image sizes, we could simply include GitlabUploaderVersions, and provide version sizes we would like to support.

class CustomUploader < GitlabUploader
  include GitlabUploaderVersions

  AVATAR_SIZES = [15, 38, 64].freeze

  versions_for AVATAR_SIZES
end

And on the model level, we need to include Versions::BackgroundRecreate, which will add support for automatic background recreation of mounted versions after the file is stored.

include Versions::BackgroundRecreate

mount_uploader :image, CustomUploader

Feature Flags

For avatars, this feature is enabled behind the feature flag:

• :static_image_resizing, in the scope of user/project/group

Sidekiq queue changes

Based on: https://docs.gitlab.com/ee/development/sidekiq_style_guide.html#changing-a-queues-urgency

The increase in RPS and execution time we expect for the new shard for recreation background job is 0.0157% (less than 5%). This is the worst-case scenario, since the job is supposed to run only once, and we will likely "warm-up" frequently used avatars.

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Testing

MR was tested on the local environment and production-like environment using omnibus docker image.

Tested with:

object store	background_upload	direct_upload
enabled	true	false
enabled	false	true
disabled	false	false

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team

Closes #232616 (closed)