VSD - Implement the members count

Overview

The members Value Stream Dashboard count metric was not implemented within #404402 (closed) because it requires a special way of iterating over the members table.

Additionally, I remember a thread where instead of implementing the member count we actually want the contributor count. This issue is for discussing and implementing the correct metric.

Metric specification for "Members count":

• Objective: track the usage of the platform.
• Source: based on PG data.
• Definition: same as the group member list.
• Aggregation Type: total count of members
• Data Refresh Frequency: monthly
• Calculate via the VSD count worker.
• Available only for the top-level namespace.
• Will be visualized in the Usage overview panel.

Implementation

Introduce a new metric enum value called direct_members in the Analytics::ValueStreamDashboard::Count class which is only collected for Groups.

In the TopLevelGroupCounterService, we perform the batch count for all metrics. Since direct members might have duplicates (no unique constraint in the members table), we need to perform a different counting logic.

For calculating the count for the direct_members, create a new batching function which uses distinct each batch.

Docs: https://docs.gitlab.com/ee/development/database/iterating_tables_in_batches.html#loose-index-scan-with-distinct_each_batch

def batch_count_items
  # this is the special case for member count
  if direct_member_count?
    count = 0

    # this scope can be probably specified in in COUNTS_TO_COLLECT
    GroupMember.where(id: group_id).distinct_each_batch(column: :user_id, of: 500) |scope|
      count += scope.count
      # break if runtime_limiter.over_time
    end
    [count, last_counted_member_id]
  else
    countable_config[:count_scope].call(namespace).each_batch_count(**arguments) do
      runtime_limiter.over_time?
    end
  end
end