Tags

Tags give the ability to mark specific points in history as being important

  • v3.8.0

    a43f6df6 · v3.8.0: bump audiolla v1.0.3 → v1.0.5 + talkies v0.5.0 → v0.9.0 + 9 new LiteLLM model entries · 
    v3.8.0 — bump audiolla v1.0.3 → v1.0.5 + talkies v0.5.0 → v0.9.0

Minor release. Image-pin bump on all four services (audiolla CPU +
CUDA, talkies CPU + CUDA) plus nine new LiteLLM provider model
entries that match the engines the upstream talkies image now ships.

AUDIOLLA v1.0.3 → v1.0.5 (patch-level, wire-compatible):
  - v1.0.4 — deepfilter no longer 400s before first load, structured
    JSON logging rewrite.
  - v1.0.5 — UVR _STEM_RE regex for newer audio-separator filename
    format + phantom-output filter (fixes the "model produced no
    output files" bug on UVR restore endpoints). 470/479 pytest
    cases green on CUDA.

TALKIES v0.5.0 → v0.9.0 (four wire-compatible minor versions):
  - v0.6.0 — New TTS slug kokoro-82m-nvidia (ONNXRuntime path, no
    PyTorch on inference hot path).
  - v0.7.0 — Qwen3-TTS PCM streaming via HTTP/1.1 chunked
    transfer-encoding (TTFA ~3-8 s → ~200-700 ms).
  - v0.8.0 — Full Qwen3-TTS mode coverage. Four new slugs:
    qwen3-tts-1.7b (Base), qwen3-tts-0.6b-custom (9 preset speakers),
    qwen3-tts-1.7b-custom (preset + emotion via `instructions`),
    qwen3-tts-1.7b-design (NL voice description via `instructions`).
  - v0.9.0 — New multilingual ASR nemotron-3.5-asr-0.6b via
    parakeet.cpp (40+ locales, OpenMDW-1.1, per-word timestamps).
    Plus a latent CUDA dealloc race fix in the sibling-eviction path.

LiteLLM provider configs grow nine new model entries:
  - local-talkies-nemotron-3.5-asr-0.6b
  - local-talkies-cuda-nemotron-3.5-asr-0.6b
  - local-talkies-kokoro-82m-nvidia
  - local-talkies-cuda-kokoro-82m-nvidia
  - local-talkies-cuda-qwen3-tts-1.7b
  - local-talkies-cuda-qwen3-tts-0.6b-custom
  - local-talkies-cuda-qwen3-tts-1.7b-custom
  - local-talkies-cuda-qwen3-tts-1.7b-design

All ten verified end-to-end against the live aigate stack via the
LiteLLM router (not direct container ports).

See CHANGELOG.md for the full details + the first-boot CLOSE_WAIT
snag on kokoro-82m-nvidia's HF snapshot.

  • v3.7.1

    23104655 · v3.7.1: bump audiolla v1.0.1 → v1.0.3 (HF lazy-download + token mirror, upstream) · 
    v3.7.1 — bump audiolla v1.0.1 → v1.0.3

Patch release that picks up two upstream fixes for the HF auth/offline
defaults that affected every text-to-audio engine in v1.0.1:

- v1.0.2 flipped HF_HUB_OFFLINE default to 0 so the engines can
  lazy-download weights on first request.
- v1.0.3 made HUGGINGFACE_TOKEN and HF_TOKEN interchangeable so gated
  repos work regardless of which env name the operator picked.

The compose env block reverts to just `HUGGINGFACE_TOKEN: ${HF_TOKEN:-}`
— the two HF_* override blocks added during v3.7.0 smoke-testing are
deleted, since they're now redundant with upstream defaults.

All three generation engines verified end-to-end on the live aigate
stack: audioldm2, musicgen-small, stable-audio-open.

See CHANGELOG.md for the full details.

  • v3.7.0

    94876b7f · v3.7.0: audiolla v0.23.1 → v1.0.1 (breaking REST contract migration) · 
    v3.7.0 — audiolla v0.23.1 → v1.0.1 (breaking REST contract migration)

Bumps the audiolla service to v1.0.1 (CPU + CUDA). Upstream's REST
contract changed completely: JSON bodies everywhere, raw bytes only at
PUT /v1/files/{path}, output_path xor output_url required for any
audio-producing endpoint, *_base64 response fields gone.

5 new text-to-audio generation engines added on the CUDA variant
(stable-audio-open, musicgen-small/medium, riffusion, audioldm2).
MusicGen weights are CC-BY-NC — gated behind a new
AUDIOLLA_ENABLE_NONCOMMERCIAL env var; the other three are commercial-
safe or self-hosted-OK without opt-in.

Aigate's smoke tests + docs + MCP fragment descriptions all rewritten
for the new shape. 14/14 tests pass on both CPU and CUDA after the
upgrade.

See CHANGELOG.md for the full details.

  • v3.6.0

    849bad97 · v3.6.0: pibox-zai v0.8.0 + LiteLLM client-header forwarding + Makefile profile autodetect · 
    v3.6.0 — pibox-zai v0.8.0 + LiteLLM client-header forwarding +
Makefile autodetect fix

Bumps pibox-zai to v0.8.0 (riding on aicodebox v0.7.0). Brings the
full set of nine `x-aicodebox-*` per-request agent knobs onto the
OpenAI-compatible endpoint — workspace, continue, append-system-prompt,
json-schema, resume, no-tools, tools-allowlist, extra-args,
timeout-seconds — reachable from any OpenAI SDK via `extra_headers`.

Enables `forward_client_headers_to_llm_api: true` in LiteLLM so those
headers actually traverse the proxy to pibox-zai. Authorization stays
on the LiteLLM side (upstream gets its own configured key).

Fixes Makefile's autodetect ifeq blocks to include the v3.4.0 vllm /
vllm-cuda / audiolla / audiolla-cuda profiles so `make run-bg`
(and `make restart`) actually starts them when their .env flag is set.

See CHANGELOG.md for the full details.

  • v3.5.0

    948728ac · v3.5.0: predictalot CUDA + predictalot MCP wiring backport + SEP-986 + healthcheck binaries · 
    v3.5.0 — predictalot CUDA restored + predictalot MCP fixed + SEP-986 +
healthcheck binaries

Restores predictalot CUDA on its own route + alias so it runs alongside
the CPU sibling instead of replacing it. Pattern mirrors audiolla:

  /predictalot/        → CPU container (PREDICTALOT=1)
  /predictalot-cuda/   → GPU container (PREDICTALOT_CUDA=1)

Both share ${DATA_DIR_PREDICTALOT}/models, both register as independent
MCP servers in the /mcp/ aggregator (`predictalot-*` and
`predictalot_cuda-*` tool prefixes).

Backports the v3.4.0 audiolla MCP wiring fixes — trailing-slash URL and
static Host: 127.0.0.1:8080 header — to predictalot.yaml. Without them
LiteLLM's MCP aggregator was silently dropping every predictalot tool.

Also fixes the v3.4.0 SEP-986 regression on audiolla-cuda's yaml key
(LiteLLM v1.80.18+ rejects dashes in MCP server names — the proxy
refused to start whenever AUDIOLLA_CUDA=1) and switches the predictalot
/ vllm / vllm-cuda healthchecks off bare `python` / `wget` so containers
report healthy and parent depends_on chains accept them.

See CHANGELOG.md for the full per-file breakdown.

  • v3.4.0

    2ff8d431 · v3.4.0: in-repo vLLM wrapper (CPU + CUDA) + audiolla (CPU + CUDA,... · 
    v3.4.0 — vLLM wrapper (CPU + CUDA) + audiolla (CPU + CUDA, side-by-side) + /mcp/ aggregator fixes

Two new local services:

  VLLM=1 / VLLM_CUDA=1
                          — supervised `vllm serve` for text LLM +
                             embeddings (Nomic Embed v2 + Qwen3-0.6B by
                             default; flat HF-repo layout shared by CPU
                             and CUDA wrappers).
  AUDIOLLA=1 / AUDIOLLA_CUDA=1
                          — self-hosted audio-production REST + MCP
                             (psyb0t/docker-audiolla v0.23.1: Demucs, UVR,
                             librosa, basic-pitch, pyannote, CLAP,
                             fluidsynth, sox + ffmpeg DSP, workflow
                             presets + pipelines). CPU and CUDA run
                             side-by-side on /audiolla/ and
                             /audiolla-cuda/; both share the same weight
                             cache.

Also fixes /mcp/ aggregator for token-protected servers: the LiteLLM-side
`os.environ/X_AUTH_TOKEN` lookup resolves against the LiteLLM container's
env, so the AIGATE_TOKEN fallback chain has to be re-exported there.
FastMCP's DNS-rebinding protection (Host allowlist) is handled via a
static Host: 127.0.0.1:8000 header for both audiolla fragments. Tool-list
timeouts bumped to 120s for the larger MCP surfaces.

See CHANGELOG.md for the full details.

  • v3.3.0

    313ade1f · v3.3.0: bump hybrids3 v0.1.0 → v0.2.0 (presigned PUT URLs) · 
    v3.3.0 — hybrids3 v0.2.0 (presigned PUT URLs)

Adds upload presigning. ?method=PUT (or boto3 put_object) returns a
signed PUT URL; default remains GET. Method is bound into the SigV4
canonical request — GET-signed URLs cannot PUT, vice versa. Public
buckets still require a signed URL for PUT.

See CHANGELOG.md for full details.

  • v3.2.0

    0fc3e425 · v3.2.0: drop predictalot-cuda service · 
    v3.2.0 — drop predictalot-cuda service

CPU-only predictalot. Both variants shared the same network alias
so running them simultaneously was broken by design. CPU is fast
enough; CUDA variant removed entirely.

  • v3.1.0

    5d6c73cd · v3.1.0: AIGATE_TOKEN master auth + talkies network fix + dead-var cleanup · 
    v3.1.0 — AIGATE_TOKEN master auth + talkies network fix + dead-var cleanup

Single AIGATE_TOKEN now defaults every aigate-owned service's bearer
(LiteLLM, claudebox, pibox-zai, predictalot, mcp, telethon,
stealthy-auto-browse, hybrids3). Per-service tokens still override.
Backwards compatible — existing .env files keep working unchanged.

Also: talkies + talkies-cuda now on `aigate-public` so their HF
prefetch at boot can resolve huggingface.co (was failing on the
internal-only network). Dead TALKIES_PREFETCH env passthrough (never
read by upstream psyb0t/talkies) stripped.

See CHANGELOG.md for full details.

  • v3.0.0

    63a4ae90 · v3.0.0: drop in-repo talkies/asr-canary/vllm/qwen3-cuda-tts, pin external psyb0t/talkies · 
    v3.0.0 — talkies migration

Removes in-repo talkies/, asr-canary/, vllm/, qwen3-cuda-tts/ source
trees. Switches transcription + TTS to external psyb0t/talkies image
(pinned to v0.5.0 / v0.5.0-cuda). One container exposes both
/v1/audio/transcriptions and /v1/audio/speech (Kokoro + Qwen3-TTS).

Breaking: env-var renames (SPEACHES / ASR_CANARY / VLLM_CUDA /
QWEN_TTS_CUDA → TALKIES / TALKIES_CUDA), alias renames (local-speaches-*,
local-asr-canary-*, local-vllm-cuda-*, local-qwen3-cuda-tts → local-talkies-*
family), data directory consolidated to .data/talkies/,
distil-whisper-large-v3 dropped (use whisper-large-v3-turbo).

See CHANGELOG.md for the full migration matrix.

  • v2.9.0

    82605a2c · v2.9.0: asr-canary verbose_json + srt/vtt with segment/word timestamps · 
    v2.9.0 — asr-canary verbose_json with timestamps

Return OpenAI Whisper-shape verbose_json (segments + words) for
canary-180m-flash and canary-1b-flash. Plus srt + vtt subtitle
output built from segments. SALM-based canary-qwen-2.5b stays
text-only (no timestamp head).

Whisper-only segment fields (avg_logprob, no_speech_prob,
compression_ratio, tokens) are null-filled so existing whisper
clients work unchanged.

Live e2e tests (verbose_json + srt) added — asr-canary 8/8 green.

  • v2.8.0

    909e29aa · v2.8.0: add vllm-cuda audio-LLM microservice (Qwen3-ASR + Voxtral) · 
    v2.8.0 — vllm-cuda audio-LLM microservice

Add vllm-cuda: Python FastAPI supervisor wrapping a single `vllm serve`
subprocess. Ships Qwen3-ASR-1.7B + Voxtral-Mini-3B with transcribe and
chat-with-audio endpoints (4 LiteLLM aliases). Only one model resident
at a time; LiteLLM resource_manager cuda-vllm group handles eviction.

Wrapper rewrites multipart response_format=verbose_json -> json before
forwarding to vllm (LiteLLM forces verbose_json; vllm doesn't accept
it). 9/9 live e2e tests pass.

Granite-Speech-3.3-2B was tried and dropped — its rel_pos_emb tensor is
architecturally ~9.5 GiB, incompatible with 12GB cards in a shared-GPU
deployment.

  • v2.7.0

    67b9aefa · v2.7.0: add asr-canary STT service, split cuda-stt group, fix parakeet · 
    v2.7.0 — asr-canary STT service + cuda-stt group split + parakeet fix

- New aigate/asr-canary/ FastAPI wrapper (OpenAI-compat transcribe +
  speaches-compat /api/ps). CPU: canary-180m-flash. CUDA:
  canary-180m-flash, canary-1b-flash, canary-qwen-2.5b (SALM).
- Pull-sidecar pattern on aigate-public; main services internal-only.
  Shared DATA_DIR_ASR_CANARY across CPU + CUDA.
- Resource manager: cuda-stt / cpu-stt split into speaches + canary
  groups for cross-service eviction.
- LiteLLM whisper_transformation monkey-patch keeps parakeet endpoints
  from receiving verbose_json.
- crisper-whisper alias dropped (upstream HF card missing library_name).
- Docs, Makefile, .env.example, tests updated.

  • v2.6.0

    ffea2480 · add three new local transcription aliases on Speaches CPU + CUDA, make idle unload explicit · 
    v2.6.0 — three new Speaches transcription aliases + explicit idle unload

New first-class LiteLLM aliases on both Speaches CPU and Speaches CUDA:

- local-speaches[-cuda]-whisper-large-v3-turbo
      deepdml/faster-whisper-large-v3-turbo-ct2 — ~8x faster than
      large-v3 at near-identical WER.

- local-speaches[-cuda]-crisper-whisper
      nyrahealth/faster_CrisperWhisper — verbatim transcription that
      preserves disfluencies, fillers, repetitions.

- local-speaches[-cuda]-parakeet-tdt-0.6b-v3
      istupakov/parakeet-tdt-0.6b-v3-onnx — multilingual (25 EU
      languages, upgrade from English-only v2).

Existing whisper-distil and parakeet-v2 aliases untouched.

Idle auto-unload made explicit: STT_MODEL_TTL / TTS_MODEL_TTL set on
both speaches services (default 600s, configurable via
SPEACHES_*_MODEL_TTL env vars). Acts as a safety net alongside the
LiteLLM resource manager's competing-job-arrival unload.

  • v2.5.0

    a376acc6 · add OpenAI gpt-4o-transcribe + gpt-4o-mini-transcribe LiteLLM aliases · 
    v2.5.0 — OpenAI gpt-4o-transcribe + gpt-4o-mini-transcribe aliases

Exposes OpenAI's March-2025 GPT-4o-based transcription models as
first-class LiteLLM aliases alongside the existing openai-whisper
(whisper-1). Lower WER than whisper, streaming partial chunks, same
/v1/audio/transcriptions endpoint.

Not yet wired into fallbacks.json. Output format caveat: the new
models only support json / text, not srt / vtt / verbose_json — keep
whisper / Groq / local-speaches for word-level timestamps.

  • v2.4.0

    6885856f · bump predictalot CPU + CUDA -> v0.2.1 (type-routed API + auth on /models) · 
    v2.4.0 — predictalot v0.2.1 (type-routed API + auth on /models)

Bumps docker-predictalot CPU + CUDA to v0.2.1. Flat /v1/forecast and
/v1/models endpoints are gone; each forecast type now has its own URL
family at /v1/<type>/{forecast,forecast/ensemble,models}. Breaking for
any direct-HTTP or MCP caller of predictalot — see CHANGELOG.md for
migration. MCP tool surface goes from 7 to 26 (per-(type,model) +
per-type ensemble + per-type listing).

Adds true multivariate forecasting, covariate-conditioned forecasting,
raw-sample-path forecasting, per-type ensembles, and an open /healthz
liveness probe. v0.2.1 closes a v0.2.0 information-leak by requiring
the bearer on all /v1/<type>/models endpoints.

Tests, docs, and the LiteLLM MCP description rewritten for the new
type-routed surface.

  • v2.3.1

    a1bb5ca5 · revert ollama-cuda NUM_PARALLEL default 50 -> 1 (ollama pins embed models to 1 anyway) · 
    v2.3.1 — revert ollama-cuda NUM_PARALLEL bump from v2.3.0

Patch on top of v2.3.0. OLLAMA_NUM_PARALLEL default goes back to 1
because ollama's scheduler pins embedding models to 1 slot anyway
(server/sched.go:414-417), making the 50 default misleading for the
use case its docs advertised.

For parallel embedding throughput in ollama, batch the inputs via
input:[...] on /v1/embeddings — that's the correct mechanism.

pibox stays at v0.7.0.

  • v2.3.0

    5bf3073a · bump ollama-cuda NUM_PARALLEL default 1 -> 50; pibox v0.3.1 -> v0.7.0 · 
    v2.3.0 — ollama-cuda batched concurrency + pibox v0.7.0

ollama-cuda
- Default OLLAMA_NUM_PARALLEL bumped 1 -> 50. One loaded model can now
  serve up to 50 concurrent requests on the same weights (continuous
  batching). Server-wide setting — caller is responsible for not
  blowing up VRAM with a big chat model on the same instance.
- Tunables documented in .env.example: OLLAMA_CUDA_NUM_PARALLEL,
  OLLAMA_CUDA_MAX_LOADED, OLLAMA_CUDA_KV_CACHE.

pibox
- psyb0t/pibox bumped v0.3.1 -> v0.7.0 (drop-in for aigate; only the
  OpenAI-compatible surface is used). Fixes the bind-mount .init-done
  bug that pinned pi to api.anthropic.com on recreate, plus picks up
  proper SSE streaming and the new jsonSchema/verbose envelopes for
  callers that hit pi's direct surface.

  • v2.2.1

    00148f06 · docs: cross-cutting README sweep + usage/testing coverage for v2.1/v2.2 services · 
    v2.2.1 — docs sweep for v2.1/v2.2 services

Docs-only patch on top of v2.2.0. No code, no config schema, no behaviour
changes — only documentation accuracy.

README
- Intro, resource management, usage curl examples, Logs and Debugging
  table, troubleshooting, and the "Full usage guide" blurb all now thread
  predictalot / mailbox / telethon mentions where the surrounding section
  was listing capabilities.

docs/usage.md
- Four new sections: Web search (SearXNG MCP), Time-series forecasting
  (predictalot), Email gateway (mailbox), Telegram client (telethon).

docs/testing.md
- Test-suite list extended with the new predictalot, mailbox, telethon
  suites + the opt-in mailbox e2e gate.

litellm/config/mcp/mcp.yaml
- mcp_tools aggregator description now lists search_web alongside
  generate_image and generate_tts.

  • v2.2.0

    91ee523d · add predictalot (time-series forecasting) + mailbox (IMAP+SMTP gateway) · 
    v2.2.0 — add mailbox (IMAP+SMTP gateway)

New service psyb0t/docker-mailbox v0.3.0 at /mailbox/. Stateless
gateway across N email accounts from a single YAML config — unified
inbox, per-account list/search/CRUD, SMTP send. No DB; every read
hits upstream IMAP live.

Direct route only (not in LiteLLM providers). MCP enabled by default
with 9 flat tools via the /mcp/ aggregator — per-account tools take
`mailbox` as a parameter, catalog stays flat regardless of inbox count.

Bearer-token auth (MAILBOX_AUTH_TOKEN). Config file holds plaintext
creds — MAILBOX_CONFIG must point at a gitignored host path; template
at mailbox/config.example.yaml; MAILBOX_CONFIG added to Makefile's
_FILE_VARS so the stack refuses to start without it.

tests/test_mailbox.sh: 4 always-on tests + 1 opt-in e2e test
(send→recv→read→delete) gated on MAILBOX_TEST_MAILBOX_NAME +
MAILBOX_TEST_ADDRESS. All gated on MAILBOX=1.

Co-shipped with v2.1.0 (predictalot) on the same commit — shared
edits to docker-compose, .env.example, Makefile, README, docs.

Also fixes dangling .data/pibox-zai/ .gitkeep allowlist from v2.0.0.