Machine learning engineer. I build AI tools, CLI applications, and civic tech, and write about the parts that were harder than they should have been.
Five years of image search in one place. CLIP made text a query for pictures, compression got a proof and moved inside the graph, cross-modal queries turned out to break the assumption every index was built on, and storing a vector fell from memory prices to object-storage prices. A sourced synthesis, with no original benchmarks.
A measured MLX vs llama.cpp comparison on a 512GB Mac Studio M3 Ultra at a 234,158-token prompt. mlx-lm 0.31.3 decodes 1.49x faster than llama.cpp b10200 and ties on prefill, while llama.cpp uses 29% less system-wide memory and outranks every MLX serving stack measured on decode. The llama.cpp arm is a single unrepeated run, and this post is mostly about why that matters.
Sixteen model and engine configurations at a real 262,000-token prompt on a 512GB Mac Studio M3 Ultra. Nemotron-3.5-Lightning-30B-A3B wins at 60.45 tok/s decode with 8/8 needle recall, and it wins by degrading slowly: its decode falls 1.83x from 2k to 262k where a dense 14B falls 10.68x. Architecture beats parameter count, a 1M-native model fails at 1M, and all five recall failures I published turned out to be my own harness.
My LLM benchmarks kept collapsing 4x mid-session. macmon caught a firmware power limiter clamping the GPU to 338 MHz and holding it while the die cooled, fans never past 70%. Pinning them at max with fanpro: 2.57x sustained decode, and a 100k-context job in 259 s instead of 568+, byte-identical.
Last post I called local 1M context a mirage. Then Kimi-Linear-48B passed: 8/8 needle recall at a real 1,048,692-token prompt on a 512GB M3 Ultra, full ingestion proven. Native 1M is finally real. It just takes a five-hour prefill.
I ran a 754B model on a 512GB Mac Studio. It fits, it crawls, and its 1M context flag is a warning message and a silent cap. Where local models fail as coding tools, and the one job they are great at.
Easy-Switch moves the keyboard and abandons the mouse. One HID++ packet fixes that, once you get past the two permission gates macOS hides between you and your own devices.
Two switches I leave on: the advisor tool and /effort ultracode. An honest, log-backed account of where they help, where they are overkill, and where the model wisely skips them.
An opinionated Cloudflare-first template. Next.js 16, D1, Better Auth, Stripe. No monorepo. No edge-runtime religion. No status page.
Head-to-head on a Mac Studio M3. 26 prompts, 6 categories, and one surprising finding about thinking mode overhead.
Explore 1,680 candidates across 275 constituencies. Filter by coalition, division, party, and more.