Engineering notes
Articles for people who
ship the stack
Original rewrites on React, Node.js, TypeScript and AI — practical notes from the same engineering practice behind our operator software. Article bodies are in English.
Tagged: bpe
Tokenization Is a System Design Decision, Not a Preprocessing Step
How byte-level BPE builds the vocabulary your LLM sees, why token counts differ by language, which failures it causes, and what to measure before blaming the model.
2081 wordsRead articleWhy Indian Languages Cost More Tokens in LLMs and How Tokenizers Get Fixed
The same sentence takes 5 tokens in English and 34 in Tamil. See how BPE training data causes it, what rebalancing and vocabulary size change, and how vocabulary expansion works.
1932 wordsRead article
FAQ
About these articles
Yes. Each piece is an original English rewrite prepared for this site. Structure and code examples follow the source material; prose is rewritten and checked for overlap before export.
Technical posts ship in English first so code and API names stay exact. The site chrome (navigation, SEO shells) is available in all nine languages.
New drafts land in the publish queue as they pass editorial checks. Running the site import picks up only new or changed slugs — nothing is duplicated.
Treat samples as illustrative. Validate against your stack and licences before production use. For delivery work, contact us with the brief.
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.