Skip to content

Add a calculator for Group Activity analytics

Aakriti Gupta requested to merge ag-add-group-activity-calculator into master

What does this MR do?

This MR is one of the 4 MRs to build the backend for Group Level Activity Analytics MVC

It introduces a calculator class for gathering a count of the recent issues and MRs created in the given group.

This class has been used in !26460 (merged)

Database queries

issues_count (runtime ~363ms)

Query

SELECT
    COUNT(*)
FROM
    "issues"
    INNER JOIN "projects" ON "projects"."id" = "issues"."project_id"
    LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE (issues.confidential IS NOT TRUE
    OR (issues.confidential = TRUE
        AND (issues.author_id = 4158075
            OR EXISTS (
                SELECT
                    TRUE
                FROM
                    issue_assignees
                WHERE
                    user_id = 4158075
                    AND issue_id = issues.id)
                OR EXISTS (
                    SELECT
                        1
                    FROM
                        "project_authorizations"
                    WHERE
                        "project_authorizations"."user_id" = 4158075
                        AND (project_authorizations.project_id = issues.project_id)
                        AND (project_authorizations.access_level >= 20)))))
    AND "projects"."namespace_id" IN ( WITH RECURSIVE "base_and_descendants" AS (
(
                SELECT
                    "namespaces".*
                FROM
                    "namespaces"
                WHERE
                    "namespaces"."type" = 'B2vUGT9soQ'
                    AND "namespaces"."id" = 9970)
            UNION (
                SELECT
                    "namespaces".*
                FROM
                    "namespaces",
                    "base_and_descendants"
                WHERE
                    "namespaces"."type" = 'B2vUGT9soQ'
                    AND "namespaces"."parent_id" = "base_and_descendants"."id"))
        SELECT
            "namespaces"."id"
        FROM
            "base_and_descendants" AS "namespaces")
    AND (EXISTS (
            SELECT
                1
            FROM
                "project_authorizations"
            WHERE
                "project_authorizations"."user_id" = 4158075
                AND (project_authorizations.project_id = projects.id)
                AND (project_authorizations.access_level >= 10))
            OR projects.visibility_level IN (10, 20))
    AND ("project_features"."issues_access_level" IS NULL
        OR "project_features"."issues_access_level" IN (20, 30)
        OR ("project_features"."issues_access_level" = 10
            AND EXISTS (
                SELECT
                    1
                FROM
                    "project_authorizations"
                WHERE
                    "project_authorizations"."user_id" = 4158075
                    AND (project_authorizations.project_id = projects.id)
                    AND (project_authorizations.access_level >= 10))))
    AND "issues"."created_at" >= 'Z55oOfUZS.'

Query plan https://explain.depesz.com/s/u7WG

#database-lab link

#database-lab link for another shorter run with slightly changed created_at param ~61ms

Summary

Time: 363.289 ms
  - planning: 5.455 ms
  - execution: 357.834 ms
    - I/O read: 262.871 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 28620 (~223.60 MiB) from the buffer pool
  - reads: 159 (~1.20 MiB) from the OS file cache, including disk I/O
  - dirtied: 6 (~48.00 KiB)
  - writes: 0

merge_request_count (~72ms)

Query

SELECT
    COUNT(*)
FROM
    "merge_requests"
    INNER JOIN "projects" ON "projects"."id" = "merge_requests"."target_project_id"
    LEFT JOIN project_features ON projects.id = project_features.project_id
WHERE
    "projects"."namespace_id" IN ( WITH RECURSIVE "base_and_descendants" AS (
(
                SELECT
                    "namespaces".*
                FROM
                    "namespaces"
                WHERE
                    "namespaces"."type" = 'vHqgGT.fkG'
                    AND "namespaces"."id" = 9970)
            UNION (
                SELECT
                    "namespaces".*
                FROM
                    "namespaces",
                    "base_and_descendants"
                WHERE
                    "namespaces"."type" = 'vHqgGT.fkG'
                    AND "namespaces"."parent_id" = "base_and_descendants"."id"))
        SELECT
            "namespaces"."id"
        FROM
            "base_and_descendants" AS "namespaces")
    AND (EXISTS (
            SELECT
                1
            FROM
                "project_authorizations"
            WHERE
                "project_authorizations"."user_id" = 4158075
                AND (project_authorizations.project_id = projects.id)
                AND (project_authorizations.access_level >= 20))
            OR projects.visibility_level IN (10, 20))
    AND ("project_features"."merge_requests_access_level" IS NULL
        OR "project_features"."merge_requests_access_level" IN (20, 30)
        OR ("project_features"."merge_requests_access_level" = 10
            AND EXISTS (
                SELECT
                    1
                FROM
                    "project_authorizations"
                WHERE
                    "project_authorizations"."user_id" = 4158075
                    AND (project_authorizations.project_id = projects.id)
                    AND (project_authorizations.access_level >= 20))))
    AND "merge_requests"."created_at" >= '06_03kmzVi'

Query plan https://explain.depesz.com/s/5sQO

#database-lab link

Summary

    Time: 73.672 ms
  - planning: 4.527 ms
  - execution: 69.145 ms
    - I/O read: 0.000 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 59707 (~466.50 MiB) from the buffer pool
  - reads: 0 from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

Screenshots

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Aakriti Gupta

Merge request reports