gavinj.net

Thoughts on technology, infrastructure, and AI

Viewing posts tagged litellm Clear filter

1 post found

One Endpoint, Every Model: LiteLLM in Front of vLLM

LiteLLM Proxy is the missing internal glue between our developers and our models: one OpenAI-compatible endpoint that fronts the DGX Spark cluster and commercial providers, with per-developer API keys, selective model access, budgets, guardrails and MCP tool gating. Here's how it works, what the Enterprise licence adds, and what else is out there.