Merge request metrics not created for some merge requests

As part of the audit it was raised that merge request metrics wasn't raised for some merge requests.

Here's a sample of them:

• Update GlIcon size in environments.vue (!58208 - merged)
• Clean-Up-Vale-Admin-Rule-(create_har_files) (!70863 - merged)
• Clean up docs warnings detected by the Vale Sub... (!71028 - merged)

They look like MRs merged from a forked repo, without running a pipeline.

select * from merge_request_metrics where merge_request_id in (....) yields no rows in gprd.

changed the description

Hey @ahegyi could you help to identify the right team, (labels) and the root cause with the current issue?

cc @achueshev @wayne @lmai1

@a_akgun,

This can be groupoptimize or groupsource code.

The table has a bit of history and as far as I know, there is no assumption built into the system that requires a merge request metric record to be present.

The metric record may be missing if the project was imported a long time ago when the import script wasn't ensuring the creation of the metric record. Besides, we had a cascade delete FK (already fixed) which deleted the metric record when the pipeline was deleted: !84919 (merged).

The forked repo is a strange case, I tested three MR's you linked in the description and the metrics records were present:

gitlabhq_dblab=# select id from merge_request_metrics where merge_request_id in (select id from merge_requests where iid in (59505, 48589, 62333)  and target_project_id = 278964);
-[ RECORD 1 ]--
id | 1578648957
-[ RECORD 2 ]--
id | 1578648943
-[ RECORD 3 ]--
id | 1578648970

@ahegyi thanks for looking.

I've fixed the list with the ones with missing merge_request_metrics.

gitlabhq_dblab=# select id, iid, target_project_id from merge_requests where id in (94261789, 117822570, 118047729);
    id     |  iid  | target_project_id 
-----------+-------+-------------------
  94261789 | 58208 |            278964
 117822570 | 70863 |            278964
 118047729 | 71028 |            278964

@a_akgun,

I suspect that the missing metrics are related to the old FK (fixed here: !84919 (merged)).

1. Project is forked.
2. MR created.
3. Pipeline executed.
4. Sometime later, the project is deleted.
5. Associated pipelines are removed.
6. On delete cascade FK on pipeline drops the associated metrics rows in the GL project.

Notice that all 3 MRs are created before the fix was merged. None of the MRs have pipelines.

Additionally, I don't see a missing metric since the fix is on PRD:

select id, created_at 
from merge_requests 
where 
target_project_id = 278964 and 
created_at > '2022-04-20' and 
not exists (select 1 from merge_request_metrics where merge_request_id = merge_requests.id)
limit 1;

Thanks @ahegyi that was our guess with @achueshev that forked projects with deleted source projects with no pipelines had no "merged_at" field.

Now that we have verified that the issue has been fixed production I wonder if we need a backfill migration fixing empty metric rows. Such a fix will add a few dozen merge requests to our pre-2021 april merge request lists/counts @lmai1 @meks

@a_akgun, from consistency PoV, it would make sense to backfill the data. Some columns can be easily derived from other DB records.

assigned to @a_akgun

@a_akgun, Please add a group or category label to identify issue ownership. You can refer to the Features by Group handbook page for guidance. If you are unsure about the correct group, please do not leave the issue without a group label. Please refer to GitLab's shared responsibility functionality guidelines for more information on how to triage this kind of issue.

changed the description

This issue was automatically tagged with the label groupcode review by TanukiStan, a machine learning classification model, with a probability of 0.92.

If this label is incorrect, please tag this issue with the correct group label as well as automation:ml wrong to help TanukiStan learn from its mistakes.

If you are unsure about the correct group, please do not leave the issue without a group label. Please refer to GitLab's shared responsibility functionality guidelines for more information on how to triage this kind of issues.

Authors who do not have permission to update labels can leave the issue to be triaged by group leaders initially assigned by TanukiStan

This message was generated automatically. You're welcome to improve it.

added automation:ml groupcode review labels

added devopscreate sectiondev labels

changed the description

added groupoptimize label and removed groupcode review label

added devopsplan label and removed devopscreate label

removed sectiondev label

added sectiondev label

unassigned @a_akgun

added devopsmonitor label and removed devopsplan label