A Tiered Map of AI Engineering Concepts and When They Matter

A list of nouns treats all twenty items as equivalent, and they clearly are not. Six of them determine whether your system functions in the first place. Seven more become relevant once you start building something for production. The last seven are things you should be able to recognize in conversation but can safely postpone learning deeply for a year — and, inconveniently, those are usually the ones people spend their free time studying.

What follows covers the same ground, but attaches three things to each concept: what it actually gives you, the point at which it starts to matter, and a way to check whether you genuinely understand it instead of just recognizing the term.

The checks are the part worth paying attention to. Most people find out they've been nodding along to a topic for months only when they try to explain it out loud for the first time.

Tier 1: The six that decide whether your system works

1. Embeddings and vector search

What it gives you: Nearly everything involving retrieval depends on this concept, and getting it wrong causes failures that never throw an error.

The check: Two embedding models each produce 1024-dimensional vectors. Explain why a classifier trained on the output of one will still generate confident garbage when given vectors from the other.

If matching dimensions seemed like a sign of compatibility, that's exactly the misunderstanding. An embedding model defines its own geometry. Two different models will place an identical sentence in completely different locations within spaces that merely happen to share the same shape. Nothing in your stack will warn you about this.

2. Retrieval quality, which is not the same as RAG

What it gives you: Almost all of the answer quality in a retrieval-based system — and almost none of that quality comes from the language model itself.

The check: Name the three points where a RAG pipeline can fail before the language model is ever invoked.

The answer is chunking, embedding, and ranking. Split a document at the wrong boundary and you lose the exact sentence that answered the question. Use an embedding model trained on the wrong domain and your jargon lands in the wrong neighborhood of the vector space. Skip a reranker and your retriever returns results that are topically adjacent but factually irrelevant. Teams that think they have a hallucination problem usually have a retrieval problem instead, and they often spend a month tweaking prompts before checking that.

3. Evaluation

What it gives you: The ability to tell whether a change actually improved anything, which is what separates engineering from guessing.

The check: Describe the golden set you evaluate against — how many examples, where they came from, what they're scored on, and who reviews the scores.

If you can't answer with actual numbers, what you have isn't evaluation, it's vibes plus a demo that happened to work on Tuesday. A reasonable starting point is somewhere between two hundred and five hundred real prompt-and-response pairs sampled from actual production traffic, scored across a handful of named dimensions, with a human spot-checking a portion of the results. It's also worth knowing that if you use a model as your judge, that judge needs its own evaluation — a recursive problem that's uncomfortable but unavoidable.

4. Structured output and tool use

What it gives you: The connection between a system that generates text and a system that actually does things.

The check: The model returns JSON that fails validation against your schema. Describe exactly what your system does next.

Saying "we retry" is where most people stop, but that's where the real questions start. How many retries, with what backoff strategy, and does the retry attempt include the validation error so the model has a chance to fix its own mistake? What happens after the final failure — does the user see an error message, or a degraded but usable answer? A tool call is essentially a function invocation across a boundary that can hallucinate, so every standard practice around validating untrusted input applies here just as strongly.

5. Cost and latency control

What it gives you: The difference between a feature that actually ships and a demo that gets killed by finance.

The check: State your current cost per request, then name five ways to cut it in half, ranked by how much impact each would have.

Five approaches are worth having ready. Send easy requests to a cheaper, smaller model instead of your default one. Cache responses for requests that are semantically close to ones you've already answered, not just identical ones. Trim or compress whatever you're stuffing into the prompt. Group anything that doesn't need a live response into batches and run it offline. And shorten what the model outputs, since the tokens it generates usually cost more than the tokens you send in. Stripe reportedly paid more than 7 billion dollars for OpenRouter this month — a company whose core product essentially automates the first two of those levers — which suggests the industry has stopped treating cost control as a minor detail.

6. Context management

What it gives you: Predictable behavior when the input exceeds the context window, something that happens constantly in production and almost never in demos.

The check: When the context window fills up, what gets dropped, and who made that decision?

A solid answer names an actual policy: drop the oldest turns first, drop the lowest-scoring retrieved chunks first, summarize the middle section, or reject the request outright. A concerning answer is that the framework takes care of it, because that usually means something important is being silently discarded and nobody has actually checked what.

Tier 2: The seven you learn while building something real

These become relevant once a system is live and real users are interacting with it. There is no harm in studying them earlier, but studying them before Tier 1 puts the effort in the wrong order.

7. Prompts as versioned artifacts

The skill of wordsmithing a prompt gets far more attention than it deserves, while the engineering discipline around prompts gets far less. What you actually need is a registry, pinned versions, side-by-side comparison, and the ability to roll back, since a prompt is effectively code that goes to production without ever passing through a compiler.

8. Chunking strategy

Splitting text into fixed-size windows, splitting along semantic boundaries, and adding overlap between chunks are all different trade-offs between recall and precision. The right choice depends on how the underlying documents are structured, not on whatever default a tutorial happens to recommend.

9. Reranking

The usual pattern is to retrieve a wide, inexpensive shortlist first, then run a more expensive scoring pass over that shortlist. Skipping that second pass is probably the single most common reason a retrieval pipeline produces results that feel almost, but not quite, right.

10. Guardrails and prompt injection

This calls for layered defenses: cheap filters at the entry point, more expensive checks closer to the model itself. Any text that comes from a user or from a document your system fetched should be treated as potentially hostile input, never as trustworthy instructions.

11. Observability for nondeterministic systems

What you want here is traces, not plain logs. The question that actually matters is which chunks were retrieved and which prompt version was live when a specific bad answer came out three days ago, and only trace-level detail can answer that.

12. Choosing between prompting, retrieval, and fine-tuning

This decision carries far more weight than mastering any single technique. Retrieval supplies knowledge, fine-tuning shapes behavior and output format, and prompting covers everything else you can manage without either. A common mistake is reaching for fine-tuning to solve what is really a retrieval gap.

13. Agent loops and tool selection

A single agent equipped with a well-chosen toolset and a bounded execution loop handles a surprisingly large fraction of the problems people try to solve by reaching for multi-agent architectures instead.

Tier 3: The seven worth recognising and deferring

For this tier, it's enough to know the vocabulary well enough to follow a conversation about it. Going deeper can wait until a concrete problem forces the issue, and for many practitioners that moment never comes.

14. Multi-agent orchestration

This is the most overhyped entry on the list. A growing body of writing has examined why these systems break down once deployed, and the recurring conclusion is that coordination overhead and compounding error rates tend to make them perform worse than a single, well-scoped agent for most use cases. Know what the term refers to, and reach for the pattern only as a last resort.

15. Quantization and serving optimisation

This matters a great deal if you are hosting your own model weights, and barely at all if you are simply calling an API.

16. Transformer internals

Attention mechanisms, positional encodings, and the rest of the architecture show up far more often in interviews than in day-to-day work. Worth learning once, but understanding it deeply changes surprisingly little about how you build systems.

17. Distillation

This becomes useful once you already have a working but expensive system and need to cut costs, which is a problem you earn rather than one you start with.

18. Semantic caching

A powerful technique with real sharp edges: a cache hit on a near-match query returns a wrong answer delivered with total confidence.

19. Knowledge graphs and GraphRAG

These offer genuine benefits when the underlying data is truly relational, and they demand considerable added complexity to unlock those benefits.

20. Preference tuning and the RLHF family

This is mostly relevant to teams actually training models, rather than teams building applications on top of ones that already exist.

The uncomfortable part

Look back across the three tiers and you'll notice something backwards.

Multi-agent orchestration, transformer internals, and prompt wording are the topics that absorb the largest share of people's learning effort, and all three land in Tier 2 or Tier 3. Meanwhile evaluation, retrieval quality, and cost control are what actually determine whether a real system works, yet they get a fraction of the attention, mainly because none of them make for a compelling demo.

There's a clear reason for that gap, and it's worth stating plainly rather than as a criticism. Tier 3 topics are easy to consume. You can read about multi-agent orchestration during a commute and walk away feeling like you learned something. Evaluation, by contrast, forces you to build a golden dataset, argue with a teammate about what "good" even means, and sometimes accept that your system was never as strong as the demo made it look. One of those activities is comfortable. The other is the one that actually helps.

How to actually learn these rather than collect them

The trap with any list like this one is treating it as a syllabus to read through. Reading gets you as far as recognition, and recognition collapses the moment someone asks a follow-up question.

Two habits work far better than reading alone.

First, build one small system end to end, then deliberately break it. Point a retrieval pipeline at documents you genuinely care about, and sabotage each stage on purpose. Chunk the text badly and watch answer quality fall apart. Remove the reranker and observe what changes. Feed the system a prompt injection attempt and see what it leaks. A weekend spent this way teaches more than a month of reading, because what you're left with is memories of specific failures, not abstract definitions.

Second, test that vocabulary against realistic questions. The checks described throughout this piece are modeled on the kinds of questions that actually come up in practice, and working through real system-design style prompts is the fastest way to expose the gaps in your understanding, largely because real questions come with follow-ups, and follow-ups are exactly where surface-level recognition stops being enough.

Where I might be wrong

This ranking is an opinionated judgment shaped by the specific systems examined here, not the output of a formal survey, so it's more honest to call it a defensible ordering than the definitive one.

There are two placements worth arguing about openly. Multi-agent orchestration sits in Tier 3 partly because current evidence on how these systems behave in production is unflattering, and a genuinely solid framework showing up in the near future could easily move it up. Transformer internals rank low here because working with AI models is increasingly an integration discipline rather than a modeling one, and anyone working closer to the models themselves should bump this topic several spots higher.

The placement worth defending most firmly is evaluation at number three, and there's a real case for putting it at number one instead. Every other item on this list turns into guesswork without it, since you cannot improve what you have no way of measuring, and very few teams building on top of models today can actually say whether last week's change made things better or worse.

If you would rearrange this ranking, the most interesting disagreements likely live in the boundary between Tier 1 and Tier 2. The more useful conversation is naming which item you'd move up and what that change actually bought you, rather than producing yet another list of twenty terms.