Tags

Tags give the ability to mark specific points in history as being important

  • v1.0.0

    85638c6d · replace global CUDA=1 with per-service flags, fix test flakes · 
    v1.0.0 — per-service CUDA flags replace global CUDA=1

Breaking: CUDA=1 no longer exists. Replace with:
- OLLAMA_CUDA=1 (GPU inference)
- SDCPP_CUDA=1 (GPU image generation)
- SPEACHES_CUDA=1 (GPU STT)
- QWEN_TTS_CUDA=1 (GPU TTS)

Each CUDA service is now independently toggleable.

  • v0.13.3

    edcffbe6 · merge ollama pullers into single service, clean up stale refs · 
    v0.13.3 — merge ollama pullers into single service

- One ollama-pull service with PULL_CPU/PULL_CUDA flags replaces
  separate ollama-pull + ollama-cuda-pull
- CUDA puller now pulls all models (CPU + CUDA)
- Removed stale ollama-cuda-pull refs from recommend-limits.sh
- Service count 28 → 27

  • v0.13.2

    f0ec01ec · complete fallback chains for all 99 models, add docs sections, fix stale refs · 
    v0.13.2 — fallback chains for all 99 models, resource management docs

- Complete fallback coverage: every model has a chain, local preferred
  over paid for image/TTS/STT, bogus cpu-flux-schnell refs removed
- README: fallback chains, resource management, troubleshooting, logs
- Fixed stale defaults, missing endpoints, and MCP server bug

  • v0.13.1

    3f6adb4f · hardcode sdcpp wrapper listen address, update docs · 
    v0.13.1 — hardcode sdcpp listen address, documentation updates

- Hardcode wrapper listen address (0.0.0.0:7234) in Go binary, remove
  from docker-compose env config
- Comprehensive documentation updates for sd.cpp integration

  • v0.13.0

    49a056e1 · feat: add stable-diffusion.cpp image generation with CUDA resource management · 
    v0.13.0 — stable-diffusion.cpp image generation + CUDA resource semaphore

Major additions:
- sd.cpp Go wrapper with CPU/CUDA backends, model hot-swap, idle timeout
- CUDA/CPU semaphore: one local job at a time, prevents GPU OOM
- Non-blocking wrapper: 503 on contention instead of request queuing
- MCP auto-discovers sdcpp models, generate_image works end-to-end
- E2E test: LLM calls tool → MCP generates image → LLM responds with link
- 5 image models: sd-turbo, sdxl-turbo, sdxl-lightning, flux-schnell,
  juggernaut-xi (CUDA); sd-turbo, sdxl-turbo (CPU)

  • v0.12.1

    c2e8aa84 · fix model prefix: local-cpu/cuda → local-ollama-cpu/cuda · 
    v0.12.1 — fix model prefix to include provider name

Corrects v0.12.0 rename: local-cpu-* → local-ollama-cpu-*,
local-cuda-* → local-ollama-cuda-*. Follows the established
local-<provider>-<hardware>-<model> convention.

  • v0.12.0

    199ab820 · rename ollama model prefixes to local-cpu/local-cuda, rewrite README intro · 
    v0.12.0 — rename ollama models to local-cpu/local-cuda prefix

Breaking change: all Ollama model aliases renamed from ollama-cpu-* and
ollama-cuda-* to local-cpu-* and local-cuda-* respectively. Any client
using the old names must update.

- 19 models renamed across provider configs, fallback chains, resource
  manager, tests, and all documentation
- README intro rewritten to reflect full platform scope
- Makefile restart now uses background mode (run-bg)

  • v0.11.1

    35e0f536 · docs: add LibreChat, MCP tools, fix stale counts · 
    v0.11.1 — docs: add LibreChat, MCP tools, fix stale counts

Docs-only patch. No code changes.

- Full documentation for LibreChat web UI and MCP media tools server
- Fixed model count (94→92), tool count (18→20), test count (91→97)
- Removed phantom HuggingFace image models from provider docs
- All docs now consistent with v0.11.0 codebase

  • v0.11.0

    aeddb434 · add MCP media tools server, LibreChat web UI, pin all images · 
    v0.11.0 — MCP media tools, LibreChat web UI, image pinning

New features:
- MCP server for image generation + TTS (generate_image, generate_tts)
  with dynamic model discovery, structured JSON returns, HybridS3 uploads
- LibreChat web UI at /librechat/ with LiteLLM backend and MCP tools
- All provider YAMLs annotated with model_info.mode for media models
- Resource calculator (recommend-limits.sh) covers all services

Infrastructure:
- All container images pinned to exact versions (no :latest)
- LibreChat MCP auth uses apiKey.source=admin (bypasses OAuth probe)
- New test suite for MCP media tools

  • v0.10.1

    2a8e2c04 · fix docs: correct counts, add optional labels, fix broken examples · 
    v0.10.1 — docs: correct counts, optional labels, fix broken browser examples

Docs-only patch. No code changes.

Fixed:
- model count 82 → 94, provider count 12 → 13, MCP tool count 34 → 18
- architecture diagram: removed duplicate provider block, added .env flags
- all non-core services labeled optional with their controlling flag
- SAB MCP section rewritten for v1.0.0 (1 run_script tool, not 17)
- all browser REST API examples now include auth header (were broken)
- vision model list expanded (8 missing multimodal models)
- admin rate limit: 5 → 30 r/m, API rate limit comment: 120 → 500 r/m
- cloudflared docs: COMPOSE_PROFILES → CLOUDFLARED=1
- SAB auth: "leave empty to disable" → "defaults to lulz-4-security"
- testing.md: stale counts and wrong /ui claim fixed

  • v0.10.0

    1106a27a · add CUDA audio services, resource manager, new ollama models, audio routing fixes · 
    v0.10.0 — CUDA audio, resource manager, new ollama models, audio routing fixes

New:
- speaches-cuda: CUDA-accelerated Whisper STT (CUDA=1)
- qwen3-cuda-tts: CUDA TTS with voice cloning, voices alloy/echo/fable (CUDA=1)
- resource_manager callback: unloads competing CUDA/CPU groups before each request
  - CUDA groups: cuda-llm (ollama-cuda), cuda-tts (qwen3-cuda-tts), cuda-stt (speaches-cuda)
  - CPU groups: cpu-tts (kokoro), cpu-stt (whisper/parakeet)
  - speaches: DELETE /api/ps/{model_id} — evicts from memory, keeps on disk
  - ollama: keep_alive=0 — frees VRAM/RAM before loading competing model
  - prevents OOM when switching between LLM, TTS, STT workloads
- ollama-cuda-gemma3-4b (vision), ollama-cuda-dolphin3, ollama-cuda-dolphin-phi
- ollama-cpu-dolphin-phi
- 9 new litellm tests (CPU TTS, CPU STT, TTS→STT roundtrip, resource manager
  unload verification, CUDA TTS, CUDA STT, CUDA resource manager, CUDA dolphin3/dolphin-phi)

Fixed:
- speaches 404: hosted_vllm/ → openai/ model prefix + /v1 in api_base
- Makefile down: now stops all profiles including CUDA/speaches
- test_security.sh: container name fix (aigate-claudebox-1)
- test_browser/integration/mcp: updated for SAB v1.0.0 run_script API

Changed:
- GPU_NVIDIA → CUDA (env var, profile, service names, model names) — CUDA is more
  precise: these services require CUDA, not just any NVIDIA GPU feature
- stealthy-auto-browse v1.0.0: all MCP browser tools → single run_script tool;
  add NUM_REPLICAS env var
- ollama CPU and CUDA share model storage (DATA_DIR_OLLAMA), no duplicate downloads
- OLLAMA_NUM_GPU=0 on CPU ollama to prevent accidental GPU use
- ollama-cuda-pull deduplicated (gemma3:4b, dolphin-phi already pulled by CPU pull)
- recommend-limits.sh: proportional scaling based on enabled flags; CUDA group
  counted as one model slot (resource manager aware)
- removed GPU_VRAM and DATA_DIR_OLLAMA_GPU env vars

  • v0.9.0

    77db6c0a · add GPU_NVIDIA support, configurable data dirs, rename ollama models · 
    v0.9.0 — GPU support, configurable data dirs, rename ollama models

New:
- GPU_NVIDIA=1: separate ollama-gpu instance with nvidia runtime
- 5 GPU models sized for 3060 12GB with per-model num_gpu control
- flash attention + q8_0 KV cache for VRAM efficiency
- DATA_DIR / DATA_DIR_<SERVICE> env vars for relocating data directories
- GPU integration tests (registration, chat, uncensored, vision)

Changed:
- all model names: local-ollama-* → ollama-cpu-*, GPU uses ollama-gpu-*
- moondream replaced with gemma3:4b (working vision model)
- all .data/ volume mounts parameterized via env vars
- tests, README, fallbacks updated for new naming

  • v0.8.0

    7decb4f8 · replace moondream with gemma3:4b, add ollama test suite · 
    v0.8.0 — replace moondream with gemma3:4b, add ollama tests

New:
- ollama test suite (4 tests: model registration, chat, embedding, vision)
- tests/.fixtures/cow.jpg for vision verification
- gemma3:4b as vision+chat model (replaces broken moondream)

Changed:
- ollama provider: moondream → gemma3:4b (general chat + vision)
- test_litellm.sh: conditionally checks ollama models when OLLAMA=1

Removed:
- moondream from pull list (broken, unmaintained GGUF)
- llama-swap config remnants from .env

  • v0.7.3

    8d6d50ea · bump proxq · 
    bump proxq

  • v0.7.2

    785a84a7 · bump proxq to v0.5.1, expose all proxq config in env · 
    v0.7.2: proxq v0.5.1, full proxq config via env

- bump proxq v0.4.1 → v0.5.1 (per-upstream retries, OpenAI client pkg)
- expose all proxq config as env vars in docker-compose:
  PROXQ_CONCURRENCY, PROXQ_TASK_RETENTION, PROXQ_UPSTREAM_TIMEOUT,
  PROXQ_MAX_RETRIES, PROXQ_RETRY_DELAY, PROXQ_MAX_BODY_SIZE,
  PROXQ_DIRECT_PROXY_THRESHOLD, PROXQ_CACHE_MODE, PROXQ_CACHE_TTL,
  PROXQ_CACHE_MAX_ENTRIES
- .env.example: document all proxq vars with defaults
- .env.example: uncomment Mistral/Cohere API key placeholders
- README: add proxq configuration table with all vars and defaults

  • v0.7.1

    d9e85a5e · configurable rate limits and timeouts, bump proxq to v0.5.1, fix free tier labels · 
    v0.7.1: configurable rate limits + timeouts, proxq v0.5.1

- bump proxq v0.4.1 → v0.5.1 (adds Go OpenAI client package, job header tracking)
- all nginx rate limits configurable via env vars:
  RATELIMIT_ADMIN, RATELIMIT_API, RATELIMIT_STORAGE,
  RATELIMIT_CLAUDEBOX, RATELIMIT_BROWSER, RATELIMIT_PROXQ
  plus corresponding _BURST vars for each zone
- all nginx proxy timeouts configurable via env vars:
  TIMEOUT_ADMIN (60s), TIMEOUT_API (600s), TIMEOUT_CLAUDEBOX (600s),
  TIMEOUT_BROWSER (600s), TIMEOUT_PROXQ (30s)
- NGINX_MAX_BODY_SIZE configurable (default 50m)
- README: fix Mistral/Cohere free tier labels in flags table,
  update nginx service description with configurable limits,
  note Redis DB 1 usage by proxq
- .env.example: add rate limit, timeout, and proxq config vars

  • v0.7.0

    e0576e79 · add proxq async job queue, fix flaky tests · 
    v0.7.0: proxq async job queue proxy, test fixes

- add proxq (psyb0t/proxq:v0.4.1) as always-on service in front of LiteLLM
  async HTTP job queue: submit request → get job ID → poll for result
  nginx routes /q/ to proxq, strips prefix, forwards to litellm:4000
  uses existing redis on DB 1 (litellm uses DB 0)
  whitelist mode: only OpenAI API paths are queued
    /v1/chat/completions, /v1/completions, /v1/embeddings,
    /v1/images, /v1/audio, /v1/responses, /chat/completions,
    /completions, /embeddings
  everything else (health, models, keys, UI) passes through directly
  10m upstream timeout for slow LLM calls, 1h job retention
- proxq config via docker compose configs block (YAML)
- nginx: add /q/ location block with proxq rate limit zone (120r/m)
- nginx depends_on proxq (service_healthy)
- add tests/test_proxq.sh (4 tests):
  health passthrough, model list passthrough,
  full async job lifecycle (submit/poll/content/cancel),
  nonexistent job 404
- fix test_health_cloudflared_tunnel: support named tunnels (not just quick tunnels)
- fix test_nginx_admin_rate_limit: fire 40 requests (burst=20 needs >21 to trigger)
- fix test_nginx_claudebox_status: handle 503 during warmup
- fix security tests: accept any non-200 as rejected (401 or 503)
  browser auth, claudebox auth, claudebox file auth,
  claudebox run auth, browser js injection, cross-token
- README: add proxq to architecture diagram, services table,
  async usage examples, update test count to 73

  • v0.6.6

    4b4b551a · f · 
    fix db

  • v0.6.5

    8436ff29 · dynamic litellm config, all providers opt-in via flags, fix postgres persistence · 
    v0.6.5: dynamic config build, all providers/services opt-in via flags

- split litellm/config.yaml into per-provider fragments under litellm/config/
- build-config.py assembles config at startup based on .env (stdlib only, no deps)
  runs automatically via python:3.12-alpine container on make run/run-bg
- all providers/services now opt-in via flags (GROQ=1, CEREBRAS=1, CLAUDEBOX=1, etc.)
  API keys are stored in .env but nothing activates unless the flag is set
- MCP servers (browser, hybrids3, claudebox, claudebox-zai) also controlled by flags
- add Docker profiles for hybrids3, browser, ollama, speaches — all opt-in via flags
- claudebox/claudebox-zai profiles now driven by CLAUDEBOX/CLAUDEBOX_ZAI=1 flags
- hybrids3 removed from nginx depends_on (502s cleanly when disabled)
- litellm/config.yaml is generated at runtime — removed from git tracking
- rename ZAI_AUTH_TOKEN → CLAUDEBOX_ZAI_AUTH_TOKEN
- fix postgres data loss: mount .data/postgres directly to PGDATA (/var/lib/postgresql/data)
  previously mounted to parent dir — postgres fell back to ephemeral anonymous volume

  • v0.6.4

    c70f8c9c · bump claudebox and stealthy-auto-browse versions · 
    v0.6.4: bump service image versions

- claudebox: v1.3.0-minimal → v1.4.0-minimal
- stealthy-auto-browse: v0.21.0 → v0.22.5