Org Mover: Implement selective sync scope for Upload

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Work on this issue
• Close this issue

Problem

We need to implement selective sync by organization: &17004. We can break up the work by data type.

Currently:

      override :selective_sync_scope
      def selective_sync_scope(node, **_params)
        if node.selective_sync?
          group_attachments(node).or(project_attachments(node)).or(other_attachments)
        else
          all
        end
      end

      # @return [ActiveRecord::Relation<Upload>] scope of Namespace-associated uploads observing selective sync settings of the given node
      def group_attachments(node)
        where(model_type: 'Namespace', model_id: node.namespaces_for_group_owned_replicables.select(:id))
      end

      # @return [ActiveRecord::Relation<Upload>] scope of Project-associated uploads observing selective sync settings of the given node
      def project_attachments(node)
        where(model_type: 'Project', model_id: ::Project.selective_sync_scope(node).select(:id))
      end

      # @return [ActiveRecord::Relation<Upload>] scope of uploads which are not associated with Namespace or Project
      def other_attachments
        where.not(model_type: %w[Namespace Project])
      end

Proposal

Implement the selective sync scope for the Upload model.

We need to expand on def other_attachments to handle all other model_types. We can ask for a list of all uploads model_types in production to be sure we cover .com cases.

Prerequisites

• uploads has a composite sharding key
• Index on each sharding key

Acceptance criteria

When selective sync by organization is selected, then uploads will be selectively synced on the secondary site, and selectively checksummed on the primary site.