Implement dynamic month-end event tracking instead of static 1-hour window

Problem

Currently we process previous month data until a static 1-hour window after month change. We need a more organic/dynamic way to identify when delayed events have arrived before starting consumption of new month data.

Current State

  • Static 1-hour window after month boundary
  • Arbitrary cutoff may miss delayed events or wait unnecessarily
  • No tracking of delayed event patterns

Proposed Solution

  • Analyze delayed event patterns at month-end
  • Implement dynamic window based on historical data
  • Track event arrival times to determine optimal cutoff

Acceptance Criteria

  • Analyze historical delayed event data
  • Identify patterns in event arrival times
  • Design dynamic window algorithm
  • Implement dynamic month-end handling
  • Add monitoring for delayed events
  • Document month-end event handling process