写给真正
交付技术栈的人
关于 React、Node.js、TypeScript 与 AI 的原创文章，来自与我们操作层软件同一套工程实践。正文为英文。
标签：attention
QUOKA Explained: Choosing Which KV History Attention Should Read
How QUOKA speeds up long-prompt prefill: KV cache basics, chunked prefill, picking outlier queries, max scoring, and cheap pre-aggregation for grouped-query attention.
2028 词阅读What GPT Really Stands For and Why AI Engineering Is Still Engineering
Unpack Generative Pre-trained Transformer, attention, fit and predict, and see why good AI work still depends on ordinary engineering judgment and better questions.
845 词阅读The KV Cache Explained: Why Long Context Is a Memory Bill, Not Smarts
A plain-language guide to the KV cache: what it stores, how to compute its size, why long context eats memory, and four ways to cut the cost on local models.
1921 词阅读From Rules to Transformers: How NLP Learned to Handle Context
A short history of natural language processing: rule-based systems, N-grams, Markov models, word vectors, RNNs, attention, Transformers and the large language models built on them.
1674 词阅读Why Transformers Use Sine and Cosine for Positional Encoding
A step-by-step explanation of sinusoidal positional encoding, from why attention ignores word order to clocks at different speeds, a worked 12-dimension example and rotation.
1919 词阅读Attention Math Step by Step: From the Input Matrix X to Softmax and V
Trace one attention head by hand: where Q, K and V come from, why scores get divided by root d_k, how softmax weights them, plus a NumPy example and KV cache notes.
1565 词阅读Build a Transformer From Scratch in PyTorch: Attention, Encoder and Decoder
Step-by-step PyTorch code for the core Transformer pieces: positional encoding, scaled dot-product and multi-head attention, feed-forward layers, encoder stack and decoder basics.
1819 词阅读How an LLM Works: Tokens, Attention and Next-Token Prediction
A plain tour of LLM architecture from tokens and embeddings through attention, causal masking and generation, plus why training and inference differ.
1801 词阅读