gavinj.net

Thoughts on technology, infrastructure, and AI

Viewing posts tagged vllm Clear filter

1 post found

From Ollama to vLLM

October 21, 2026 • ai, llm, local-ai

From 24GB Blackwell workstation cards to one 128GB DGX Spark, then two Sparks and a QSFP cable: running DeepSeek-V4-Flash-Vision-Exp locally, understanding vLLM, and planning the next step with a 200Gb/s switch.

gav.ilicio.us

eMudhra Intermediate CA Compromise
Eight intermediate CA private keys were recoverable from their certificates, potentially enabling website impersonation for unpatched clients. Goddammit—Patch Tuesday was yesterday!
Permit Pending
SimCity, except every apartment block, bus lane and factory must survive zoning hearings, angry neighbours, lawsuits and the next election. I laughed; urban planners may cry.
How Abliterated Models Can Get You Pwned
ProjectDiscovery poisoned a 7B model for under $50; it behaved perfectly until a hidden trigger made a coding agent steal project credentials. A sobering local-AI supply-chain warning—but oh, please don't stop the fun!
finereli/whatsapp-mcp: A read-only MCP server that lets your AI agent search and read your own WhatsApp messages. Runs on Cloudflare Workers.
this might be handy ...
If you don't run your own LLM proxy, how the fuck do you even know what's going on?
I'll have to have another crack at using pangolin for this exact purpose (I just couldn't get it working last time I tried!).
Meta unveils Muse Spark Open, the new top U.S. open weights model — and it runs on two paired Mac desktops
Meta's Muse Spark Open is a U.S.-trained open-weights model that the company says can run on two paired Mac desktops. The announcement positions it as the leading open-weights model from the U.S., with benchmarks that place it alongside top closed and open alternatives.
Prerequisites for Microsoft Defender for Endpoint on Linux
Defender for Endpoint now supports Linux on the desktop, in public preview. Every distribution already covered on servers carries over, with support tied to the distribution and kernel version rather than the desktop environment on top. The full matrix runs from RHEL and Ubuntu LTS through Debian, Fedora, SUSE and the RHEL clones, on both x64 and ARM64.
Iosevka
A beautifully compact, endlessly configurable typeface built for code and terminals—with dedicated Term and Fixed variants. My terminal may be getting a makeover.
Crafting Apps: open-source creative tools
A vibe-coded Adobe suite written in Rust, cross-platform with GPU acceleration. I am going to have to give these a try.
GPT-6 and Intelligent UI for everyone
OpenAI's announcement for GPT-6, rolling out across ChatGPT alongside what it calls Intelligent UI: responses that arrive with clickable buttons, forms and charts instead of plain text. The notable shift is the model composing an interface on the fly rather than just answering in prose.