Skip to content
Snippets Groups Projects

Fix duplicate finished build sync events

What does this MR do and why?

This MRs removes 3 feature flags that were used to debug the duplicates being added to the ci_finished_builds_aggregated_queueing_delay_percentiles_mv materialized view, and fixes the problem with sending duplicate build IDs to ClickHouse. We do this by loading the AR relationship before calling pluck(:build_id), so that the call to last doesn't generate an extra query.

EE: true

Part of #428146 (closed)

This change was tested in the production console, where it was confirmed that @processed_record_ids contained duplicate builds IDs at the end of execute.

image

The builds in @processed_build_ids looked like the following, if split by batch:

@processed_build_ids by batch
[[
  # ... 498 unique IDs,
  5_634_258_445,
  5_634_259_020],
[
  5_634_258_445,
  5_634_259_020,
  # ... 494 unique IDs,
  5_634_474_423,
  5_634_474_548,
  5_634_474_559,
  5_634_474_885],
 [5_634_474_423,
  5_634_474_548,
  5_634_474_559,
  5_634_474_885,
  # ... 496 unique IDs
 ],
 [ # ... 496 unique IDs,
  5_634_583_521,
  5_634_583_562,
  5_634_583_577,
  5_634_583_579,
  5_634_583_582,
  5_634_583_584],
 [5_634_583_521,
  5_634_583_562,
  5_634_583_577,
  5_634_583_579,
  5_634_583_582,
  5_634_583_584,
  # ... 492 unique IDs,
  5_634_614_703,
  5_634_614_785],
 [5_634_614_703,
  5_634_614_785,
  # ... 495 unique IDs,
  5_634_636_196,
  5_634_636_225,
  5_634_636_274],
 [5_634_636_196,
  5_634_636_225,
  5_634_636_274,
  # ... 496 unique IDs,
  5_634_651_184],
 [5_634_651_184,
  # ... 492 unique IDs,
  5_634_667_919,
  5_634_667_925,
  5_634_667_929,
  5_634_667_934,
  5_634_667_943,
  5_634_668_045,
  5_634_668_092],
 [5_634_667_919,
  5_634_667_925,
  5_634_667_929,
  5_634_667_934,
  5_634_667_943,
  5_634_668_045,
  5_634_668_092,
  # ... 489 unique IDs,
  5_634_680_068,
  5_634_680_069,
  5_634_680_072,
  5_634_680_076],
 [5_634_680_068,
  5_634_680_069,
  5_634_680_072,
  5_634_680_076,
  # ... 491 unique IDs,
  5_634_688_597,
  5_634_688_600,
  5_634_688_603,
  5_634_688_607,
  5_634_688_608],
 [5_634_688_597,
  5_634_688_600,
  5_634_688_603,
  5_634_688_607,
  5_634_688_608,
  # ... 493 unique IDs,
  5_634_693_846,
  5_634_693_852],
 [5_634_693_846,
  5_634_693_852,
  # ... 349 unique IDs
 ],
 []]

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

n/a

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

n/a

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Pedro Pombeiro

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
Please register or sign in to reply
Loading