chore(deps): update dependency litellm to v1.40.15 (!962) · Merge requests · GitLab.org / ModelOps / AI Assisted (formerly Applied ML) / Code Suggestions / AI Gateway

renovate requested to merge renovate/litellm-1.x-lockfile into main Jun 15, 2024

This MR contains the following updates:

Package	Type	Update	Change
litellm	dependencies	patch	`1.40.0` -> `1.40.15`

⚠️ Warning

Some dependencies could not be looked up. Check the warning logs for more information.

Release Notes

BerriAI/litellm (litellm)

`v1.40.15`

Compare Source

What's Changed

[Fix] Security Fix bump docusaurus version by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4213
build(deps): bump azure-identity from 1.15.0 to 1.16.1 by @dependabot in https://github.com/BerriAI/litellm/pull/4130
build(deps): bump braces from 3.0.2 to 3.0.3 in /ui/litellm-dashboard by @dependabot in https://github.com/BerriAI/litellm/pull/4131
fix(build): .dockerignore not picked up by @bcvanmeurs in https://github.com/BerriAI/litellm/pull/3116
[Refactor-Proxy] Refactor user_api_key_auth to be it's own file by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4215
[Reliability Fix] Anthropic / Bedrock HTTPX - Cache Async Httpx client by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4211
refactor(utils.py): Cut down utils.py to <10k lines. by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4216
ui - show exceptions by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4222
fix - non sso ui sign up flow by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4220
[Refactor-Proxy] Make proxy_server.py < 10K lines (move management, key, endpoints to their own files) by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4217

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.14...v1.40.15

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.15

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	120.0	132.58387302297749	6.398687111538595	0.0	1915	0	97.12711200000967	1186.0091809999744
Aggregated	Passed ✅	120.0	132.58387302297749	6.398687111538595	0.0	1915	0	97.12711200000967	1186.0091809999744

`v1.40.14`

Compare Source

What's Changed

ui - fix team based usage crashing by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4198
[Fix + Refactor] - Router Alerting for llm exceptions + use separate util for sending alert by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4197
[Bug fix] Don't cache team, user, customer budget after calling /update, /delete by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4203
feat(router.py): support content policy fallbacks by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4207
fix(slack_alerting.py): allow new 'alerting_metadata' arg by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4205
build(pyproject.toml): require pydantic v2 by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4151
[Feat] send email alerts when budget exceeded by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4209
[Fix] redact_message_input_output_from_logging deepcopy bug by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4210

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.13...v1.40.14

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.14

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	120.0	141.18410333195084	6.441903839147897	0.0	1928	0	105.22602600002529	510.8018800000025
Aggregated	Passed ✅	120.0	141.18410333195084	6.441903839147897	0.0	1928	0	105.22602600002529	510.8018800000025

`v1.40.13`

Compare Source

What's Changed

fix(parallel_request_limiter.py): rate limit keys across instances by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4150
Langfuse integration support for parent_observation_id parameter by @hburrichter in https://github.com/BerriAI/litellm/pull/3559

New Contributors

@hburrichter made their first contribution in https://github.com/BerriAI/litellm/pull/3559

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.12...v1.40.13

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.13

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	74	85.12421177852299	6.470441187117138	0.0	1937	0	63.80303100002038	1377.5951729999178
Aggregated	Passed ✅	74	85.12421177852299	6.470441187117138	0.0	1937	0	63.80303100002038	1377.5951729999178

`v1.40.12`

Compare Source

What's Changed

add llama 3 family from deepinfra by @themrzmaster in https://github.com/BerriAI/litellm/pull/4191
feat(proxy/utils.py): allow budget duration in months (1mo) by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4188
fix(utils.py): check if model info is for model with correct provider by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4186
Retry on connection disconnect by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4178

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.11...v1.40.12

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.12

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	94	113.78855698077908	6.432303146239259	0.0	1925	0	80.02467099998967	1025.8250419999513
Aggregated	Passed ✅	94	113.78855698077908	6.432303146239259	0.0	1925	0	80.02467099998967	1025.8250419999513

`v1.40.11`

Compare Source

What's Changed

[Fix] Redacting messages from OTEL + Refactor utils.py to use litellm_core_utils by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4176
Stop throwing constant S3 spam on cache misses by @Manouchehri in https://github.com/BerriAI/litellm/pull/4177
[Feat] - Prometheus add remaining_team_budget gauge by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4179
fix - clean up swagger spend endpoints 🧹🧹🧹🧹 by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4180
[Fix] Fix bug when updating team budgets on UI + display budget =0.0 correctly on UI by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4183
Reset budget option for teams by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4185
feat(init.py): allow setting drop_params as an env by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4187
[Doc] Setting Team budgets by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4189

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.10...v1.40.11

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.11

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	120.0	140.50671503682315	6.351765918831661	0.0	1901	0	96.28972799998792	1490.2560670000184
Aggregated	Passed ✅	120.0	140.50671503682315	6.351765918831661	0.0	1901	0	96.28972799998792	1490.2560670000184

`v1.40.10`

Compare Source

What's Changed

[Feat] add VertexAI vertex_ai/text-embedding-004 , vertex_ai/text-multilingual-embedding-002 by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4145
Add IAM cred caching for OIDC flow by @Manouchehri in https://github.com/BerriAI/litellm/pull/3712
feat(util.py/azure.py): Add OIDC support when running LiteLLM on Azure + Azure Upstream caching by @Manouchehri in https://github.com/BerriAI/litellm/pull/3861
[Feat] Support task_type, auto_truncate params by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4152
[Feat] support dimensions for vertex embeddings by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4149
docs - run proxy on custom root path by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4154
[Fix] user was inserted in Proxy Server embedding requests + added param mapping for mistral by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4156
[Fix] Add ClarifAI support for LiteLLM Proxy by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4158
[Admin UI] Fix error Internal Users see when using SSO by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4164
[Fix] - Error selecting model provider from UI by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4166
[UI] add Azure AI studio models on UI by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4167
feat(vertex_httpx.py): Support Vertex AI system messages, JSON Schema, etc. by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4160
Fix errors in the Vertex AI documentation by @yamitzky in https://github.com/BerriAI/litellm/pull/4171
feat(prometheus): add api_team_alias to exported labels by @bcvanmeurs in https://github.com/BerriAI/litellm/pull/4169

New Contributors

@yamitzky made their first contribution in https://github.com/BerriAI/litellm/pull/4171

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.9...v1.40.10

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.10

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	140.0	172.37660025809805	6.297822628765798	0.0	1883	0	114.60945100003528	3651.5153230000124
Aggregated	Passed ✅	140.0	172.37660025809805	6.297822628765798	0.0	1883	0	114.60945100003528	3651.5153230000124

`v1.40.9`

Compare Source

What's Changed

fix opentelemetry-semantic-conventions-ai does not exist on LiteLLM Docker by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4129
[Feat] OTEL - allow propagating traceparent in headers by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4133
Added mypy to the Poetry dev group by @jamesbraza in https://github.com/BerriAI/litellm/pull/4136
Azure AI support all models by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4134
feat(utils.py): bump tiktoken dependency to 0.7.0 (gpt-4o token counting support) by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4119
fix(proxy_server.py): use consistent 400-status code error code for exceeded budget errors by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4139
Allowing inference of LLM provider in get_supported_openai_params by @jamesbraza in https://github.com/BerriAI/litellm/pull/4137
[FEAT] log management endpoint logs to otel by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4138

New Contributors

@jamesbraza made their first contribution in https://github.com/BerriAI/litellm/pull/4136

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.8...v1.40.9

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.9

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	95	118.26463258740928	6.42020613574963	0.0	1922	0	78.571060999991	1634.9082140000064
Aggregated	Passed ✅	95	118.26463258740928	6.42020613574963	0.0	1922	0	78.571060999991	1634.9082140000064

`v1.40.8`

Compare Source

What's Changed

[FEAT]- OTEL log litellm request / response by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4076
[Feat] Enterprise - Attribute Management changes to Users in Audit Logs by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4083
[FEAT]- OTEL Log raw LLM request/response on OTEL by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4078
fix(cost_calculator.py): fixes tgai unmapped model pricing by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4085
fix(utils.py): improved predibase exception mapping by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4080
[Fix] Litellm sdk - allow ChatCompletionMessageToolCall, and Function to be used as dict by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4086
Update together ai pricing by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4087
[Feature]: Proxy: Support API-Key header in addition to Authorization header by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4088
docs - cache controls on litellm python SDK by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4099
docs: add llmcord.py to side bar nav by @jakobdylanc in https://github.com/BerriAI/litellm/pull/4101
docs: fix llmcord.py side bar link by @jakobdylanc in https://github.com/BerriAI/litellm/pull/4104
[FEAT] - viewing spend report per customer / team by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4105
feat - log Proxy Server auth errors on OTEL by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4103
[Feat] Client Side Fallbacks by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4107
Fix typos: Enterpise -> Enterprise by @msabramo in https://github.com/BerriAI/litellm/pull/4110
assistants.md: Remove extra trailing backslash by @msabramo in https://github.com/BerriAI/litellm/pull/4112
assistants.md: Add "Get a Thread" example by @msabramo in https://github.com/BerriAI/litellm/pull/4114
ui - Fix Test Key dropdown by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4108
fix(bedrock_httpx.py): fix tool calling for anthropic bedrock calls w/ streaming by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4106
fix(proxy_server.py): allow passing in a list of team members by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4084
fix - show model group in Azure ContentPolicy exceptions by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4116

Client Side Fallbacks: https://docs.litellm.ai/docs/proxy/reliability#test---client-side-fallbacks

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.7...v1.40.8

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.8

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.8

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	140.0	169.11120714803027	6.281005310183787	0.0	1878	0	114.50119100004486	1457.4686270000257
Aggregated	Passed ✅	140.0	169.11120714803027	6.281005310183787	0.0	1878	0	114.50119100004486	1457.4686270000257

`v1.40.7`

Compare Source

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.6...v1.40.7

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.7

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	97	126.50565680197539	6.4278560269757214	0.003340881510902142	1924	1	82.64289499999222	1316.4627209999935
Aggregated	Passed ✅	97	126.50565680197539	6.4278560269757214	0.003340881510902142	1924	1	82.64289499999222	1316.4627209999935

`v1.40.6`

Compare Source

🚨 Note: LiteLLM Proxy Added `opentelemetry` as a dependency on this release. We recommend waiting for a stable release before upgrading your production instances

✅ LiteLLM Python SDK Users: You should be unaffected by this change (`opentelemetry` was only added for the proxy server)

🔥 LiteLLM 1.40.6 - Proxy 100+ LLMs AT Scale with our production grade OpenTelemetry logger. Trace LLM API Calls, DB Requests, Cache Cache Requests 👉 Start here: https://docs.litellm.ai/docs/proxy/logging#logging-proxy-inputoutput-in-opentelemetry-format

🐞 [Fix]- Allow redacting messages from slack alerting https://docs.litellm.ai/docs/proxy/alerting#advanced---redacting-messages-from-alerts

🔨 [Refactor] - Refactor proxy_server.py to use common function for add_litellm_data_to_request

✨ [Feat] OpenTelemetry - Log Exceptions from Proxy Server

✨ [FEAT] OpenTelemetry - Log Redis Cache Read / Writes

✨ [FEAT] OpenTelemetry - LOG DB Exceptions

✨ [Feat] OpenTelemetry - Instrument DB Reads

🐞 [Fix] UI - Allow custom logout url and show proxy base url on API Ref Page

What's Changed

feat(bedrock_httpx.py): add support for bedrock converse api by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4033
feature - Types for mypy - issue #360 by @mikeslattery in https://github.com/BerriAI/litellm/pull/3925
[Fix]- Allow redacting messages from slack alerting by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4047
Fix to support all file types supported by Gemini by @nick-rackauckas in https://github.com/BerriAI/litellm/pull/4055
[Feat] OTEL - Instrument DB Reads by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4058
[Refactor] - Refactor proxy_server.py to use common function for add_litellm_data_to_request by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4065
[Feat] OTEL - Log Exceptions from Proxy Server by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4067
Raw request debug logs - security fix by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4068
[FEAT] OTEL - Log Redis Cache Read / Writes by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4070
[FEAT] OTEL - LOG DB Exceptions by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4071
[Fix] UI - Allow custom logout url and show proxy base url on API Ref Page by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4072

New Contributors

@mikeslattery made their first contribution in https://github.com/BerriAI/litellm/pull/3925
@nick-rackauckas made their first contribution in https://github.com/BerriAI/litellm/pull/4055

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.5...v1.40.6

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.6

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	130.0	151.53218399526997	6.362696017911015	0.0	1903	0	109.01354200001379	1319.1295889999992
Aggregated	Passed ✅	130.0	151.53218399526997	6.362696017911015	0.0	1903	0	109.01354200001379	1319.1295889999992

`v1.40.5`

Compare Source

What's Changed

Table format fix and Typo by @SujanShilakar in https://github.com/BerriAI/litellm/pull/4037
feat: add langfuse metadata via proxy request headers by @ndrsfel in https://github.com/BerriAI/litellm/pull/3990
Add Ollama as a provider in proxy ui by @sha-ahammed in https://github.com/BerriAI/litellm/pull/4020
modified docs proxy->logging->langfuse by @syGOAT in https://github.com/BerriAI/litellm/pull/4035
fix tool usage null content using vertexai by @themrzmaster in https://github.com/BerriAI/litellm/pull/4039
Fixed openai token counter bug by @Raymond1415926 in https://github.com/BerriAI/litellm/pull/4036
feat(router.py): enable settting 'order' for a deployment in model list by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4046
docs: add llmcord.py to projects by @jakobdylanc in https://github.com/BerriAI/litellm/pull/4060
Fix log message in Custom Callbacks doc by @iwamot in https://github.com/BerriAI/litellm/pull/4061
refactor: replace 'traceback.print_exc()' with logging library by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4049
feat(aws_secret_manager.py): Support AWS KMS for Master Key encrption by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4054
[Feat] Enterprise - Enforce Params in request to LiteLLM Proxy by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4043
feat - OTEL set custom service names and custom tracer names by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4048

New Contributors

@ndrsfel made their first contribution in https://github.com/BerriAI/litellm/pull/3990
@sha-ahammed made their first contribution in https://github.com/BerriAI/litellm/pull/4020
@syGOAT made their first contribution in https://github.com/BerriAI/litellm/pull/4035
@Raymond1415926 made their first contribution in https://github.com/BerriAI/litellm/pull/4036
@jakobdylanc made their first contribution in https://github.com/BerriAI/litellm/pull/4060
@iwamot made their first contribution in https://github.com/BerriAI/litellm/pull/4061

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.4...v1.40.5

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.5

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	98	123.75303621190369	6.512790176735744	0.0	1949	0	80.83186400000386	1991.117886999973
Aggregated	Passed ✅	98	123.75303621190369	6.512790176735744	0.0	1949	0	80.83186400000386	1991.117886999973

`v1.40.4`

Compare Source

What's Changed

feat: clarify slack alerting message by @nibalizer in https://github.com/BerriAI/litellm/pull/4023
[Admin UI] Analytics - fix div by 0 error on /model/metrics by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4021
Use DEBUG level for curl command logging by @grav in https://github.com/BerriAI/litellm/pull/2980
feat(create_user_button.tsx): allow admin to invite user to proxy via user-email/pwd invite-links by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4028
[FIX] Proxy redirect to PROXY_BASE_URL/ui after logging in by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4027
[Feat] Audit Logs for Key, User, ProxyModel CRUD operations by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4030

New Contributors

@nibalizer made their first contribution in https://github.com/BerriAI/litellm/pull/4023

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.3...v1.40.4

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.4

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.4

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	74	89.43947919222931	6.450062450815326	0.0	1930	0	64.37952199996744	1143.0389689999743
Aggregated	Passed ✅	74	89.43947919222931	6.450062450815326	0.0	1930	0	64.37952199996744	1143.0389689999743

`v1.40.3`

Compare Source

What's Changed

[FIX] Proxy - only log cache credentials in debug mode by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4024

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.2...v1.40.3

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.3

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	130.0	168.35103872813087	6.385058663866248	0.0	1909	0	109.50845100001061	8353.559378
Aggregated	Passed ✅	130.0	168.35103872813087	6.385058663866248	0.0	1909	0	109.50845100001061	8353.559378

`v1.40.2`

Compare Source

What's Changed

Add simple OpenTelemetry tracer by @yujonglee in https://github.com/BerriAI/litellm/pull/3974
[FEAT] Add native OTEL logging to LiteLLM by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4010
[Docs] Use OTEL logging on LiteLLM Proxy by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4011
fix(bedrock): raise nested error response by @pharindoko in https://github.com/BerriAI/litellm/pull/3989
[Feat] Admin UI - Add, Edit all LiteLLM callbacks on UI by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4014
feat(assistants/main.py): add assistants api streaming support by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4012
feat(utils.py): Support stream_options param across all providers by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4015
fix(utils.py): fix cost calculation for openai-compatible streaming object by @krrishdholakia in https://github.com/BerriAI/litellm/pull/4009
[Fix] Admin UI Internal Users by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/4016

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.1...v1.40.2

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.2

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	72	86.0339053382131	6.392727588765549	0.0	1913	0	61.2748209999836	896.4834699999642
Aggregated	Passed ✅	72	86.0339053382131	6.392727588765549	0.0	1913	0	61.2748209999836	896.4834699999642

`v1.40.1`

Compare Source

What's Changed

[Feat] return num_retries and max_retries in exceptions by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3962
[FEAT]- set custom AllowedFailsPolicy on litellm.Router by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3963
feat(proxy_server.py): log litellm api version to langfuse by @krrishdholakia in https://github.com/BerriAI/litellm/pull/3969
feat - add batches api to docs by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3973
[Fix] Traceloop / OTEL logging fixes + easier docs by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3970
add dall-e 3 required comment by @rhtenhove in https://github.com/BerriAI/litellm/pull/3984
[Feat] Log Raw Request from LiteLLM on Langfuse - when "log_raw_request": true by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3983
[Feat] Admin UI - Multi-Select Tags, Viewing spend by tags by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3994
[Feat] UI - Filter model latency by API Key Alias by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3995
feat(assistants/main.py): Azure Assistants API support by @krrishdholakia in https://github.com/BerriAI/litellm/pull/3996
[Admin UI] Filter Model Latency by Customer, API Key by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3997
fix(router.py): use litellm.request_timeout as default for router clients by @krrishdholakia in https://github.com/BerriAI/litellm/pull/3992
[Doc] - Spend tracking with litellm by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3991

New Contributors

@rhtenhove made their first contribution in https://github.com/BerriAI/litellm/pull/3984

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.40.0...v1.40.1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	120.0	139.78250550967104	6.395300383667639	0.0	1913	0	95.28932899991105	1526.2213239999483
Aggregated	Passed ✅	120.0	139.78250550967104	6.395300383667639	0.0	1913	0	95.28932899991105	1526.2213239999483

Configuration

📅 Schedule: Branch creation - "every weekend" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this MR and you won't be reminded about this update again.

If you want to rebase/retry this MR, check this box

This MR has been generated by Renovate Bot.

Edited Jun 17, 2024 by renovate

chore(deps): update dependency litellm to v1.40.15

Release Notes

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

What's Changed

New Contributors

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

What's Changed

New Contributors

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

What's Changed

New Contributors

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

What's Changed

Client Side Fallbacks: https://docs.litellm.ai/docs/proxy/reliability#test---client-side-fallbacks

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

🚨 Note: LiteLLM Proxy Added opentelemetry as a dependency on this release. We recommend waiting for a stable release before upgrading your production instances

✅ LiteLLM Python SDK Users: You should be unaffected by this change (opentelemetry was only added for the proxy server)

What's Changed

New Contributors

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

What's Changed

New Contributors

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

What's Changed

New Contributors

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

What's Changed

New Contributors

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Configuration

Merge request reports

🚨 Note: LiteLLM Proxy Added `opentelemetry` as a dependency on this release. We recommend waiting for a stable release before upgrading your production instances

✅ LiteLLM Python SDK Users: You should be unaffected by this change (`opentelemetry` was only added for the proxy server)