dispatcher: add configurable batch processing via `BATCH_SIZE` setting

Summary

Process up to BATCH_SIZE (default: 16) requests per tick instead of one.

Depends on: !9 (merged)

Problem

The dispatcher processes only one request per tick (1s loop). It fetches ALL new requests from the API, picks one at random, processes it, then sleeps 1s and repeats. With 852 backlogged requests, clearing the queue takes ~14 minutes.

Changes

  • Added BATCH_SIZE: 16 to settings.yaml
  • Renamed get_request()get_requests(): returns up to BATCH_SIZE randomly shuffled requests instead of a single random pick
  • Extracted per-request processing into _process_request() for clarity
  • Main loop iterates over the batch within a single tick
  • Configurable via TF_DISPATCHER_BATCH_SIZE env var or settings.yaml

Impact

With BATCH_SIZE=16, a backlog of 852 requests clears in ~53 ticks (~1 min) instead of ~852 ticks (~14 min). Setting BATCH_SIZE=1 restores the original behavior.

Assisted-by: Claude Code

Edited by Miroslav Vadkerti

Merge request reports

Loading