Backport of 'Disable Sidekiq retries for ClickHouse pipeline/build sync workers'

What does this MR do and why?

Describe in detail what merge request is being backported and why

Backport of !219149 (merged) to 18.6.

Disables Sidekiq retries for ClickHouse pipeline/build sync workers to prevent duplicate data insertion.

Problem

When Net::ReadTimeout occurs during ClickHouse INSERT operations:

  1. ClickHouse receives and processes the INSERT successfully
  2. The HTTP response times out before reaching the client
  3. Worker throws exception before marking sync events as processed
  4. Events remain processed: false in PostgreSQL
  5. On retry, the same events are sent to ClickHouse again → duplicates

Solution

Setting retry: false prevents this amplification. The cron jobs run every 3 minutes, so any failed work will be picked up by the next scheduled run.

Also changes data_consistency to :sticky to ensure we read from the primary database and don't miss recently created sync events.

  • Part of #586319

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

  • This MR is backporting a bug fix, documentation update, or spec fix, previously merged in the default branch.
  • The MR that fixed the bug on the default branch has been deployed to GitLab.com (not applicable for documentation or spec changes).
  • The MR title is descriptive (e.g. "Backport of 'title of default branch MR'"). This is important, since the title will be copied to the patch blog post.
  • Required labels have been applied to this merge request
  • This MR has been approved by a maintainer (only one approval is required).
  • Ensure the e2e:test-on-omnibus-ee job has succeeded, or if it has failed, investigate the failures. If you determine the failures are unrelated, you may proceed. If you need assistance investigating, reach out to a Software Engineer in Test in #s_developer_experience.

Note to the merge request author and maintainer

If you have questions about the patch release process, please:

Edited by Pedro Pombeiro

Merge request reports

Loading