Articles for people who
ship the stack
Rewrites on React, Node.js, TypeScript and AI. A translated body is shown when it exists. If it does not, you see the English text and a note that says so.
Tagged: deep-learning
Superposition in Neural Networks: Toy Models, SAEs and Feature Merging
How networks store more features than dimensions, what interference costs, and why sparse autoencoders can fuse features that the data can't separate.
19 min readHow Transformers Work, From RNN Bottlenecks to Attention, Agents and Context Windows
A step-by-step tour of the transformer: why RNNs stalled, how tokens and embeddings work, attention, QKV, heads, positions, block structure, scale and agents.
40 min readHow AI Reads and Rebuilds a Photo: The Science Behind 1980s Portraits
How vision models turn pixels into embeddings and how generative AI uses them to restyle a photo as 1980s film, with limits on accuracy, identity and detection.
9 min readHow a Transformer Reads "The Cat Sat on the Mat", Step by Step
Follow one sentence through a transformer layer: tokens, embeddings, positions, self-attention, residual connections, normalization and the feed-forward network.
6 min readWhy Transformers Use Sine and Cosine for Positional Encoding
A step-by-step explanation of sinusoidal positional encoding, from why attention ignores word order to clocks at different speeds, a worked 12-dimension example and rotation.
8 min readResidual Connections Explained: Why Deep Transformers Can Train at All
Why stacking layers breaks training, how x + F(x) skip connections fix gradient flow, how Pre-LayerNorm blocks use them, and a short PyTorch implementation.
4 min readBuild a Transformer From Scratch in PyTorch: Attention, Encoder and Decoder
Step-by-step PyTorch code for the core Transformer pieces: positional encoding, scaled dot-product and multi-head attention, feed-forward layers, encoder stack and decoder basics.
8 min read
About these articles
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.