gavinj.net

Thoughts on technology, infrastructure, and AI

One Endpoint, Every Model: LiteLLM in Front of vLLM

LiteLLM Proxy is the missing internal glue between our developers and our models: one OpenAI-compatible endpoint that fronts the DGX Spark cluster and commercial providers, with per-developer API keys, selective model access, budgets, guardrails and MCP tool gating. Here's how it works, what the Enterprise licence adds, and what else is out there.

From Ollama to vLLM

From 24GB Blackwell workstation cards to one 128GB DGX Spark, then two Sparks and a QSFP cable: running DeepSeek-V4-Flash-Vision-Exp locally, understanding vLLM, and planning the next step with a 200Gb/s switch.