dispatcher: add configurable batch processing via `BATCH_SIZE` setting
Summary
Process up to BATCH_SIZE (default: 16) requests per tick instead of one.
Depends on: !9 (merged)
Problem
The dispatcher processes only one request per tick (1s loop). It fetches ALL new requests from the API, picks one at random, processes it, then sleeps 1s and repeats. With 852 backlogged requests, clearing the queue takes ~14 minutes.
Changes
- Added
BATCH_SIZE: 16tosettings.yaml - Renamed
get_request()→get_requests(): returns up toBATCH_SIZErandomly shuffled requests instead of a single random pick - Extracted per-request processing into
_process_request()for clarity - Main loop iterates over the batch within a single tick
- Configurable via
TF_DISPATCHER_BATCH_SIZEenv var orsettings.yaml
Impact
With BATCH_SIZE=16, a backlog of 852 requests clears in ~53 ticks (~1 min) instead of ~852 ticks (~14 min). Setting BATCH_SIZE=1 restores the original behavior.
Assisted-by: Claude Code
Edited by Miroslav Vadkerti