Engineering notes
Articles for people who
ship the stack
Original rewrites on React, Node.js, TypeScript and AI — practical notes from the same engineering practice behind our operator software. Article bodies are in English.
Tagged: tiktoken
Tokenization Is a System Design Decision, Not a Preprocessing Step
How byte-level BPE builds the vocabulary your LLM sees, why token counts differ by language, which failures it causes, and what to measure before blaming the model.
2081 wordsRead article
FAQ
About these articles
Yes. Each piece is an original English rewrite prepared for this site. Structure and code examples follow the source material; prose is rewritten and checked for overlap before export.
Technical posts ship in English first so code and API names stay exact. The site chrome (navigation, SEO shells) is available in all nine languages.
New drafts land in the publish queue as they pass editorial checks. Running the site import picks up only new or changed slugs — nothing is duplicated.
Treat samples as illustrative. Validate against your stack and licences before production use. For delivery work, contact us with the brief.
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.