Optimize overage submission batching based on upload size instead of query size

Problem

Currently we query overaged subscriptions in batches of 200, but based on eligibility criteria, actual batch sizes sent to Zuora are often <200. This results in variable, inefficient batch sizes.

Current State

  • Query batches: 200 subscriptions
  • Actual upload batches: variable, often <200 (due to eligibility filtering)
  • While Zuora processes individual records correctly, batching is inefficient

Proposed Solution

  • Batch based on upload size rather than query size
  • Ensures consistent, predictable batch sizes for Zuora uploads
  • Consider separating OverageRecords and OverageSubmission concerns

Acceptance Criteria

  • Analyze current batching efficiency
  • Design new batching strategy based on upload size
  • Implement size-based batching
  • Measure improvement in batch consistency
  • Consider OverageRecords/OverageSubmission separation
  • Document new batching approach