Articles for people who
ship the stack
Original rewrites on React, Node.js, TypeScript and AI — practical notes from the same engineering practice behind our operator software. Article bodies are in English.
Tagged: pytorch
Residual Connections Explained: Why Deep Transformers Can Train at All
Why stacking layers breaks training, how x + F(x) skip connections fix gradient flow, how Pre-LayerNorm blocks use them, and a short PyTorch implementation.
1031 wordsRead articleBuild a Transformer From Scratch in PyTorch: Attention, Encoder and Decoder
Step-by-step PyTorch code for the core Transformer pieces: positional encoding, scaled dot-product and multi-head attention, feed-forward layers, encoder stack and decoder basics.
1819 wordsRead articleHow Embedding Layers Work: From Text to Token IDs to Vectors in an LLM
A beginner-friendly look at embedding layers: tokenization, lookup tables, learned vectors, the 205 million parameter example, and a short PyTorch demo with nn.Embedding.
1201 wordsRead articleBuilding a Byte Pair Encoding Tokenizer From Scratch for a Small GPT
Implement a character-level BPE tokenizer in Python, train it on WikiText-2, save and fingerprint it, and retrain a small GPT on shorter, denser token sequences.
6273 wordsRead articleDebugging a Small GPT in PyTorch: Tests That Isolate Each Failure
A stage-by-stage workflow for debugging a character-level GPT in PyTorch, from token IDs and target shifting to gradients, NaN losses and checkpoints.
6828 wordsRead article
About these articles
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.