Articles for people who
ship the stack
Original rewrites on React, Node.js, TypeScript and AI — practical notes from the same engineering practice behind our operator software. Article bodies are in English.
Tagged: vllm
Continuous batching and chunked prefill in vLLM and SGLang
Static LLM batches waste GPU on padding. Iteration-level scheduling and chunked prefill keep decode streams smooth while new prompts still make progress.
1144 wordsRead articleLLM inference optimization: prefill, decode, and enterprise LLMOps
Prefill vs decode bottlenecks, continuous batching, FlashAttention, quantization, PagedAttention, speculative decoding, chunked prefill, and disaggregated serving.
1809 wordsRead articleLLMOps for Small Language Models: Serving Frameworks and Production Playbooks
Why SLMs win on cost and privacy, how vLLM, SGLang, TGI, llama.cpp, Ollama, WebLLM, ONNX, and TensorRT-LLM compare, and how to quantize, evaluate, and route them in production.
4127 wordsRead articleServing Qwen3.8-27B on One RTX 3090 With a Patched vLLM and DFlash2
How a pinned vLLM 0.28.0 fork with requantized embeddings and DFlash2 speculation serves a 27B hybrid model on a 24 GB card, and where long context slows it down.
3193 wordsRead article
About these articles
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.