Blog
-
Redacting secrets from LLM requests without breaking the prompt cache
My gateway's secret redaction used a fresh placeholder per request, which silently turned every turn of every agent conversation into a prompt-cache miss. How provider caching matches prefixes, and the deterministic placeholders, total ordering and streaming hold-back that keep a rewriting proxy cache-friendly.
-
What I learned running a fleet of AI coding agents
About 10,800 pull requests across my hobby projects with AI coding agents in Kubernetes pods writing nearly every change. The failures that cost the most were steps that exited green without doing anything, and the guardrails that mattered were single-pass loops, server-side pauses and a definition of done that ends at the deployed result.
-
Single-user speed vs total throughput: reading inference-chip claims
Most chip headlines in the millions of tokens per second are aggregate throughput across many users, often modeled. Sorting late-2026 inference-chip claims by what they measure: the best substantiated single-stream result is about 17,000 tokens/s on a quantized 8B model.
-
Hardware transcoding Jellyfin on an Intel NUC
Every hardware transcode on my Intel NUC failed with ffmpeg exit 234 because the iGPU driver only supports CQP rate control and Jellyfin always asks for VBR. The ffmpeg wrapper that fixed it, why Dolby Vision Profile 5 turns pink and green, and why an embedded subtitle took eight minutes to appear.
-
Looking at one token of LLM inference
An interactive visualizer that breaks one decode step into its operation types and static, ephemeral and mutable data, then measures how serial and how parallel each token really is.
-
What CI costs on Hetzner versus GitHub Actions
Self-hosted CI runners and remote Rust builders on Hetzner came out several times cheaper per runner-hour than GitHub Actions at list price. Most of the real waste was failed builds, stale work, cold caches and an autoscaler that would not give capacity back.
-
Indexing Lance files without the Dataset API
How I index immutable Lance part files with per-part sidecars instead of the Lance dataset layer, what that bought on a 7.67 million row test (up to 19,973× fewer rows decoded), and the byte and fragmentation problems it ran into in production.
-
CI that tests only what changed
Selecting only the downstream units of a change in a Rust monorepo of nearly a thousand crates, and skipping tests that already passed at the same content fingerprint. Then a week of cost data showed 75% of runner time going to runs that failed anyway.
-
Agents write tests that can't fail
Coding agents are good at writing tests that stay green whatever the code does: assert!(true) after a constructor, credential-less skips that return Ok, mocks asserting on themselves. Here are the shapes, why agents produce them, and the cheap lexical CI scan and test rules that catch most of them.
-
Proving dataflow operators coordination-free in Lean 4
Using the CALM theorem to decide where a query planner needs barriers, proving the leaf aggregation and connected-components kernels in Lean 4 against Aeneas-extracted Rust, and an exact accounting of what is proven, what is only tested, and what is still a design.
-
Cheap LLM serving on consumer Blackwell: NVFP4, FP8 KV cache and MTP speculative decoding
What raised throughput and cut cost when serving many agents from a 35B MoE on cheap Blackwell cards: FP8 KV cache gave 21% more agent throughput, NVFP4 weights moved the model and its LoRA onto a 48 GB card about 5x cheaper, and MTP speculative decoding accepted most of its guesses without raising fleet throughput. Includes the KV-cache math for working out how many agents fit on a card.
-
Fine-tuning a 35B mixture-of-experts model on rented GPUs
LoRA fine-tuning Qwen3.6-35B-A3B on vast.ai and AWS spot: why a B200 ran no faster than an H200, a 16x-slower grouped-GEMM fallback on RTX PRO 6000 cards, and a library import that silently stopped the routed experts from training. Plus the offer filters and reaper I built after rented boxes failed in ways the listings didn't show.
-
Ubuntu KVM QEMU GPU passthrough to Windows VM
It's always been annoying to rely on dual-boot Ubuntu / Windows requiring me to switch between OS's on my home computer, but it seems in the past that's been the only solution to getting any apps to run correctly with proper performance.