Articles for people who
ship the stack
Original rewrites on React, Node.js, TypeScript and AI — practical notes from the same engineering practice behind our operator software. Article bodies are in English.
Tagged: inference
Continuous batching and chunked prefill in vLLM and SGLang
Static LLM batches waste GPU on padding. Iteration-level scheduling and chunked prefill keep decode streams smooth while new prompts still make progress.
1144 wordsRead articleLLM inference optimization: prefill, decode, and enterprise LLMOps
Prefill vs decode bottlenecks, continuous batching, FlashAttention, quantization, PagedAttention, speculative decoding, chunked prefill, and disaggregated serving.
1809 wordsRead articleWhy a 17 GB Model Download Doesn't Mean 17 GB of Memory to Run It
Learn how active parameters, total parameters, KV cache growth and reasoning effort decide the real memory and compute cost of running a language model locally.
2722 wordsRead articleWhen Quantization Passes Perplexity and Silently Breaks Model Safety
Low-bit KV cache quantization can wipe out a model's refusals while perplexity barely moves; here is why the standard metrics miss it and what to add to your gate.
1625 wordsRead article
About these articles
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.