Optimize overage submission batching based on upload size instead of query size
Problem
Currently we query overaged subscriptions in batches of 200, but based on eligibility criteria, actual batch sizes sent to Zuora are often <200. This results in variable, inefficient batch sizes.
Current State
- Query batches: 200 subscriptions
- Actual upload batches: variable, often <200 (due to eligibility filtering)
- While Zuora processes individual records correctly, batching is inefficient
Proposed Solution
- Batch based on upload size rather than query size
- Ensures consistent, predictable batch sizes for Zuora uploads
- Consider separating OverageRecords and OverageSubmission concerns
Acceptance Criteria
- Analyze current batching efficiency
- Design new batching strategy based on upload size
- Implement size-based batching
- Measure improvement in batch consistency
- Consider OverageRecords/OverageSubmission separation
- Document new batching approach