v0.10.0 — CUDA audio, resource manager, new ollama models, audio routing fixes
New:
- speaches-cuda: CUDA-accelerated Whisper STT (CUDA=1)
- qwen3-cuda-tts: CUDA TTS with voice cloning, voices alloy/echo/fable (CUDA=1)
- resource_manager callback: unloads competing CUDA/CPU groups before each request
- CUDA groups: cuda-llm (ollama-cuda), cuda-tts (qwen3-cuda-tts), cuda-stt (speaches-cuda)
- CPU groups: cpu-tts (kokoro), cpu-stt (whisper/parakeet)
- speaches: DELETE /api/ps/{model_id} — evicts from memory, keeps on disk
- ollama: keep_alive=0 — frees VRAM/RAM before loading competing model
- prevents OOM when switching between LLM, TTS, STT workloads
- ollama-cuda-gemma3-4b (vision), ollama-cuda-dolphin3, ollama-cuda-dolphin-phi
- ollama-cpu-dolphin-phi
- 9 new litellm tests (CPU TTS, CPU STT, TTS→STT roundtrip, resource manager
unload verification, CUDA TTS, CUDA STT, CUDA resource manager, CUDA dolphin3/dolphin-phi)
Fixed:
- speaches 404: hosted_vllm/ → openai/ model prefix + /v1 in api_base
- Makefile down: now stops all profiles including CUDA/speaches
- test_security.sh: container name fix (aigate-claudebox-1)
- test_browser/integration/mcp: updated for SAB v1.0.0 run_script API
Changed:
- GPU_NVIDIA → CUDA (env var, profile, service names, model names) — CUDA is more
precise: these services require CUDA, not just any NVIDIA GPU feature
- stealthy-auto-browse v1.0.0: all MCP browser tools → single run_script tool;
add NUM_REPLICAS env var
- ollama CPU and CUDA share model storage (DATA_DIR_OLLAMA), no duplicate downloads
- OLLAMA_NUM_GPU=0 on CPU ollama to prevent accidental GPU use
- ollama-cuda-pull deduplicated (gemma3:4b, dolphin-phi already pulled by CPU pull)
- recommend-limits.sh: proportional scaling based on enabled flags; CUDA group
counted as one model slot (resource manager aware)
- removed GPU_VRAM and DATA_DIR_OLLAMA_GPU env vars