From Ollama to vLLM
From 24GB Blackwell workstation cards to one 128GB DGX Spark, then two Sparks and a QSFP cable: running DeepSeek-V4-Flash-Vision-Exp locally, understanding vLLM, and planning the next step with a 200Gb/s switch.
Thoughts on technology, infrastructure, and AI
Viewing posts tagged ai Clear filter
16 posts found
From 24GB Blackwell workstation cards to one 128GB DGX Spark, then two Sparks and a QSFP cable: running DeepSeek-V4-Flash-Vision-Exp locally, understanding vLLM, and planning the next step with a 200Gb/s switch.
Every night of our outback trip I sent a handful of photos and a few sentences to my AI agent over Starlink, and woke up to a published, fact-checked diary entry. Here is how that workflow actually went, including the real prompts, the corrections, and the parts that went wrong.
Meta's new Muse agent is built around a curated long-term memory, and every major AI vendor is racing to solve the same problem. Meanwhile my own assistants still get sudden dementia every few days. A look at how agent memory actually works, from Hermes internals to ChatGPT, Claude, Gemini, Apple and xAI's Grok Bot.
A DGX Spark with 128GB of unified memory let me explore much larger local models. Then I discovered Heretic variants, and a hardware experiment became a rather uncomfortable lesson in how AI refusals work, how easily they can be changed, and what that means for the rest of us.
Pangolin 1.22 shipped an AI Gateway this week. The provider list covers OpenAI, Anthropic and Gemini, but there's no Kimi or DeepSeek button. Here's how to register both as custom providers, and why the missing buttons turn out not to matter.