Implement dynamic month-end event tracking instead of static 1-hour window
Problem
Currently we process previous month data until a static 1-hour window after month change. We need a more organic/dynamic way to identify when delayed events have arrived before starting consumption of new month data.
Current State
- Static 1-hour window after month boundary
- Arbitrary cutoff may miss delayed events or wait unnecessarily
- No tracking of delayed event patterns
Proposed Solution
- Analyze delayed event patterns at month-end
- Implement dynamic window based on historical data
- Track event arrival times to determine optimal cutoff
Acceptance Criteria
- Analyze historical delayed event data
- Identify patterns in event arrival times
- Design dynamic window algorithm
- Implement dynamic month-end handling
- Add monitoring for delayed events
- Document month-end event handling process