Skip to content

Importer Metrics: Add the total number of projects imported

Kassio Borges requested to merge kassio/add-project-imports-total-metric into master

What does this MR do?

In order to start tracking the self-managed instances, we need to instrument usage pings to collect the total number of projects created and the number of imported projects.

Related to:

Screenshots (strongly suggested)

Query plan

Query:

SELECT COUNT("projects"."id") FROM "projects" WHERE "projects"."created_at" BETWEEN '2021-04-19 11:56:32.457238' AND '2021-05-17 11:56:32.457454' AND "projects"."id" >= 1 AND "projects"."id" < 100001

Plan:

 Aggregate  (cost=19972.44..19972.45 rows=1 width=8) (actual time=4016.838..4016.840 rows=1 loops=1)
   Buffers: shared read=21915
   I/O Timings: read=3918.693 write=0.000
   ->  Index Only Scan using index_projects_on_created_at_and_id on public.projects  (cost=0.56..19968.16 rows=1712 width=4) (actual time=4016.832..4016.833 rows=0 loops=1)
         Index Cond: ((projects.created_at >= '2021-04-19 11:56:32.457238+00'::timestamp with time zone) AND (projects.created_at <= '2021-05-17 11:56:32.457454+00'::timestamp with time zone) AND (projects.id >= 1) AND (projects.id < 100001))
         Heap Fetches: 0
         Buffers: shared read=21915
         I/O Timings: read=3918.693 write=0.000

Recommendations:

❗️ Query processes too much data to return a relatively small number of rows. – Reduce data cardinality as early as possible during the execution, using one or several of the following techniques: new indexes, partitioning, query rewriting, denormalization. See the visualization of the plan to understand which plan nodes are the main bottlenecks. <http://momjian.us/main/writings/pgsql/hw_performance/|Show details>

Statistics:

Time: 4.021 s
  - planning: 3.745 ms
  - execution: 4.017 s
    - I/O read: 3.919 s
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 0 from the buffer pool
  - reads: 21915 (~171.20 MiB) from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Kassio Borges

Merge request reports