Buffer counters for DataTransfer models

What does this MR do and why?

Add counter attributes for Projects::DataTransfer

New table project_data_transfers can experience a significant pressure due to parallel requests updating the table.

https://gitlab.com/gitlab-org/gitlab/-/issues/355976#note_1210806834

This MR explores the use of use counter_attribute concern to prevent the database table overload. I created a data_transfer_counter_attribute tailor-made for data transfer feature.

How to use

Set up an egress counter attribute

Do the following in a class definition.

module Projects
  class DataTransfer < ApplicationRecord

    # 1) include DataTransferCounterAttribute class
    include DataTransferCounterAttribute

    # 2) assign the counter_attribute to the egresses
    egress_attribute :repository_egress

  end
end

See https://gitlab.com/gitlab-org/gitlab/-/blob/dd503a061042fa7a7e0a56dbf767a22a219ea6dd/app/models/projects/data_transfer.rb

Check the current state of the counter (buffered amount + db state)

project_data_transfer.repository_egress # => 10

Increase egress counter

project_data_transfer.increment_repository_egress(45) # => 55 (= 10+45)

or

project_data_transfer.increment_egress(:repository_egress, 45) # => 55

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #386835

Edited by Patrick Cyiza

Merge request reports

Loading