llm-pool

Project information

Single-header C++17: worker pool with a priority queue and requests-per-minute and tokens-per-minute limits. Part of llm-cpp.

Created on

September 30, 2026

Loading