Articles for people who
ship the stack
Rewrites on React, Node.js, TypeScript and AI. A translated body is shown when it exists. If it does not, you see the English text and a note that says so.
Tagged: kv-cache
Context Engineering for Long-Running Agents: Vector Database, KV Cache, Claude Code
Context engineering for agents that run 200 steps: why long contexts decay, KV cache economics, vector database memory, and how Claude Code and others differ.
13 min readQUOKA Explained: Choosing Which KV History Attention Should Read
How QUOKA speeds up long-prompt prefill: KV cache basics, chunked prefill, picking outlier queries, max scoring, and cheap pre-aggregation for grouped-query attention.
9 min readThe KV Cache Explained: Why Long Context Is a Memory Bill, Not Smarts
A plain-language guide to the KV cache: what it stores, how to compute its size, why long context eats memory, and four ways to cut the cost on local models.
8 min readMoving KV Cache Between Qwen3 Models Instead of Re-Prefilling
NVIDIA research maps Qwen3-14B attention state into Qwen3-32B with a ridge-regression bridge. See what the latency numbers mean, where it fails, and why agents care.
8 min readWhy Round-Robin Fails LLM Fleets: How llm-d Routes by KV Cache and Queue Depth
Round-robin load balancing wastes KV cache hits and ignores request size. How llm-d scores vLLM replicas on Kubernetes and splits prefill from decode.
10 min readKV cache is the unseen database eating GPU memory in LLM decode
The KV cache quietly eats GPU memory during LLM inference: prefill vs decode, KV math, GQA and MQA, PagedAttention, quantization and eviction policies.
9 min readLLM Inference End to End: Prefill, Decode, KV Cache, and Sampling
A whiteboard-ready tour of tokenizer through streaming—why prefill is compute-bound, decode is bandwidth-bound, and how GQA, paging, and speculation change the economics.
40 min read
About these articles
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.