v3.13.0 — fix sd.cpp image-gen flakiness via resource_manager pre-warm
Minor release. Eliminates the "/v1/images/generations returns HTTP
200 with empty data" failure mode on local-sdcpp-{cuda,cpu}-*
routes.
Root cause: sd.cpp's TryLockModel 503s during model load (5-20 s)
-> LiteLLM router num_retries=3 retries internally + walks
fallback chain (4-5 entries per model) -> one external call
generates 15-30 backend hits, all 503 -> LiteLLM image-gen
fallback exhaustion returns HTTP 200 + empty data (upstream bug).
Fix: resource_manager.py async_pre_call_hook now issues an
explicit blocking POST /sdcpp/v1/load?model=<key> for cuda-img /
cpu-img requests, immediately after the existing competing-group
unload. By the time LiteLLM dispatches the actual image-gen call,
sdcpp is fully warm and attempt 1 succeeds. Pre-load failures are
logged rather than raised.
Other model groups (chat / embed / talkies / vllm / llamacpp /
predictalot) are UNCHANGED — the pre-warm step only fires for the
sd.cpp groups.
Verified: 5/5 sequential cold/warm/swap calls across sd-turbo /
sdxl-turbo / sdxl-lightning return valid b64 PNG data. No new
config knobs. No new env vars. No new endpoints.
Doc sync: docs/resource-management.md + README.md both updated to
describe the new pre-warm step.
Known residual: 5+ concurrent calls against different models can
still flake (LiteLLM router internal parallel retry bypasses the
proxy hook). Beyond realistic ~1-2 req/min ad-hoc workload.