Articles for people who
ship the stack
Original rewrites on React, Node.js, TypeScript and AI — practical notes from the same engineering practice behind our operator software. Article bodies are in English.
Tagged: attention
QUOKA Explained: Choosing Which KV History Attention Should Read
How QUOKA speeds up long-prompt prefill: KV cache basics, chunked prefill, picking outlier queries, max scoring, and cheap pre-aggregation for grouped-query attention.
2028 wordsRead articleWhat GPT Really Stands For and Why AI Engineering Is Still Engineering
Unpack Generative Pre-trained Transformer, attention, fit and predict, and see why good AI work still depends on ordinary engineering judgment and better questions.
845 wordsRead articleThe KV Cache Explained: Why Long Context Is a Memory Bill, Not Smarts
A plain-language guide to the KV cache: what it stores, how to compute its size, why long context eats memory, and four ways to cut the cost on local models.
1921 wordsRead articleFrom Rules to Transformers: How NLP Learned to Handle Context
A short history of natural language processing: rule-based systems, N-grams, Markov models, word vectors, RNNs, attention, Transformers and the large language models built on them.
1674 wordsRead articleWhy Transformers Use Sine and Cosine for Positional Encoding
A step-by-step explanation of sinusoidal positional encoding, from why attention ignores word order to clocks at different speeds, a worked 12-dimension example and rotation.
1919 wordsRead articleAttention Math Step by Step: From the Input Matrix X to Softmax and V
Trace one attention head by hand: where Q, K and V come from, why scores get divided by root d_k, how softmax weights them, plus a NumPy example and KV cache notes.
1565 wordsRead articleBuild a Transformer From Scratch in PyTorch: Attention, Encoder and Decoder
Step-by-step PyTorch code for the core Transformer pieces: positional encoding, scaled dot-product and multi-head attention, feed-forward layers, encoder stack and decoder basics.
1819 wordsRead articleHow an LLM Works: Tokens, Attention and Next-Token Prediction
A plain tour of LLM architecture from tokens and embeddings through attention, causal masking and generation, plus why training and inference differ.
1801 wordsRead article
About these articles
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.