Skip to content

Attemp to fix reached_end_of_table flag

What does this MR do and why?

This change attempts to fix the reached_end_of_table flag which is set by the EvnetsSyncWorker when processed all records in the table. The assumption is that the events table gets so many inserts, the worker will never reach the end of the table:

  1. Select a batch (maybe we already reached the end of the table)
  2. Format data
  3. Send data to ClickHouse (take a few secs)
  4. Next batch would normally return as no data, but in the meantime there were inserts, so the batching will continue

The fix records the last events.id and processes the rows until we reach that id.

Related to #431547 (closed)

Edited by Adam Hegyi

Merge request reports