Articles for people who
ship the stack
Original rewrites on React, Node.js, TypeScript and AI — practical notes from the same engineering practice behind our operator software. Article bodies are in English.
Tagged: gpu
Continuous batching and chunked prefill in vLLM and SGLang
Static LLM batches waste GPU on padding. Iteration-level scheduling and chunked prefill keep decode streams smooth while new prompts still make progress.
1144 wordsRead articleLLM inference optimization: prefill, decode, and enterprise LLMOps
Prefill vs decode bottlenecks, continuous batching, FlashAttention, quantization, PagedAttention, speculative decoding, chunked prefill, and disaggregated serving.
1809 wordsRead articleServing Qwen3.8-27B on One RTX 3090 With a Patched vLLM and DFlash2
How a pinned vLLM 0.28.0 fork with requantized embeddings and DFlash2 speculation serves a 27B hybrid model on a 24 GB card, and where long context slows it down.
3193 wordsRead articleServing Qwen3.8-Flash-Next on RTX 3090s with llama.cpp Tensor Offloading
Why an 88GB, 180B-parameter sparse model fits consumer GPUs, and how llama.cpp flags like -ot, -ncmoe and mmap split it across VRAM, RAM and NVMe.
2691 wordsRead article
About these articles
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.