Fifteen LLM Concepts Traced Through a Single Support-Bot Request

A customer opens a shop's chat widget and asks whether headphones bought ten days earlier can still be returned. A few seconds later the assistant reads the question, consults the store's return policy and explains the options. That exchange looks trivial, but it passes through a chain of distinct components: text is converted into numbers, a trained model computes over them, the application may search internal documents, and it may call a tool to start the return.

The vocabulary engineers use for those steps (tokens, embeddings, inference, RAG and the rest) is easy to repeat and easy to misuse. This guide walks through fifteen core terms in the order they matter for that single request, grouped into what happens before the customer ever arrives, what happens while the answer is produced, and what it takes to let the system act and to trust it. For each concept you get what it is, why it matters in engineering terms, and the misconception it most often invites. Treat the list as a foundation, not the whole field; for a broader map of what to learn next, see our tiered map of AI engineering concepts.

Before the question arrives: the model and how it learned

Everything in this section happened long before the customer opened the chat. It determines what the assistant can do at all.

1. Model: a learned input-to-output function

A model is a mathematical system that maps an input to an output using patterns extracted from data. One model classifies photos of animals; another forecasts delivery times. A language model operates on text and produces it by predicting plausible next tokens, one step at a time.

A rough analogy is a student who has worked through thousands of practice problems. Faced with a new one, the student applies what practice taught rather than looking up an identical question. The analogy is imperfect, since models neither learn nor understand the way people do, but it captures the core idea of generalizing from examples.

In the shop scenario, the language model interprets the question and drafts the reply. The finished product, though, also includes a user interface, data sources and possibly tools.

• Engineering relevance: models are specialized. One that writes fluent text is not automatically good at demand forecasting or fraud scoring.
• Common mistake: equating "the model" with "the application". The model is one component among several.

2. Parameters: the numbers that encode behavior

A neural network contains a very large number of adjustable values called parameters. Weights are the most discussed kind; biases are another. Together they determine how information is transformed as it flows through the network.

Picture a mixing desk covered in knobs: turning them changes the sound, and changing parameters changes the outputs. Training is the process that turns the knobs.

What a language model has learned is spread across huge numeric arrays. There is generally no single location that holds "Paris is the capital of France". Some training text can be memorized verbatim, but the model is not a searchable archive of the documents it saw.

• Engineering relevance: parameter count drives storage, memory and compute requirements, and it influences capability. A larger model is still not guaranteed to be better for your specific task.
• Common mistake: imagining parameters as a database of stored answers. They are learned settings.

3. Training: practice with feedback

Training adjusts a model's parameters using data and an objective. For language models, a standard objective is next-token prediction.

Take the training sentence "The capital of France is Paris." The model is shown the beginning, produces a prediction for what comes next, and the training system measures the gap between that prediction and the real next token. The parameters are then nudged to shrink the error. Repeated over an enormous number of examples, this looks far more like drilling with corrections than like copying a book into storage.

The shop's model was trained before any customer showed up. That gave it general language competence, but nothing guarantees it knows this particular store's current return rules.

• Engineering relevance: the choice of data and objective shapes both what the model can do and the kinds of errors it makes.
• Common mistake: assuming a model that has absorbed language patterns has also verified facts. It has not.

4. Transformer: the architecture most language models share

A transformer is a neural-network design that combines attention with other processing layers to turn input representations into useful outputs. Think of an assembly line: each station refines the work passed to it rather than starting from the raw text again.

Inside a transformer-based language model, token representations pass through a stack of repeated layers. Attention blends information across positions, other components transform each position's representation, and positional information lets the model tell "dog bites man" from "man bites dog". The final representation is turned into a score for every candidate next token, the scores are converted into probabilities, and one token is chosen.

• Engineering relevance: knowing the architecture ties the other concepts together. Tokens feed it, embeddings represent them, attention relates them, and the context window bounds how much it can see.
• Common mistake: confusing an architecture with a trained model. A transformer is a blueprint; a trained model is a specific instance with learned parameters.

While the answer is produced: from text to tokens to output

Now the customer hits send. This section covers what happens during that request.

5. Inference: running the trained model

Inference means executing a trained model on new input to get an output. If training is practice, inference is sitting the exam.

When the headphones question arrives, the model computes over the supplied input using the parameters it already has. In normal operation it does not retrain on each message; its parameters stay fixed while it answers. For text generation, inference is a loop: compute probabilities for the next token, pick one, append it and repeat until the reply is complete. How that "pick one" step works is its own topic, covered in our guide to temperature, top-k and top-p.

• Engineering relevance: inference is where latency and per-request compute cost live, so it shapes user experience and your bill.
• Common mistake: assuming anything said in a conversation is learned permanently. The model can use information in the current context without writing it into its parameters.

6. Token: the unit of text the model actually sees

A token is a piece of text as defined by a tokenizer. Depending on the tokenizer, it might be a whole word, a fragment of a word, a punctuation mark or some other chunk. As an illustration only, "headphones" could be kept whole by one tokenizer and split into something like "head" and "phones" by another. Never assume one word equals one token.

Each token is mapped to an integer ID, and the model turns those IDs into numeric representations before processing them.

• Engineering relevance: token counts determine whether input fits, how long outputs can be, and frequently what an API call costs. The same meaning can cost different numbers of tokens in different languages or formats; JSON and code, for instance, often tokenize differently from plain prose.
• Common mistake: mixing up tokens and token IDs. Tokens are text pieces; IDs are their numeric labels.

7. Embedding: meaning as a position in space

An embedding is a numeric representation of some item: a token, a sentence, a document or an image. Usually it is a vector, an ordered list of numbers.

Imagine arranging a library on a map so that books on similar subjects sit close together. An embedding gives each item coordinates on such a map, which makes similarity measurable. A well-chosen search embedding model might place "How does this item get sent back?" close to "What is your return policy?" even though the two share almost no words.

It helps to keep two uses apart. Inside a language model, token embeddings are part of how input is represented. In a search system, document embeddings are used to find relevant material. The ideas are related, but the representation and even the model producing it are often different.

• Engineering relevance: embeddings are what make meaning-based search and retrieval work.
• Common mistake: treating closeness as correctness. The nearest document can still be the wrong policy, for example an outdated version.

8. Prompt: the instructions and material you supply

A prompt is everything handed to the model to steer its response: instructions, the user's question, examples and supporting text.

Compare two requests. "Explain returns" is vague. "Using the policy below, decide whether headphones purchased ten days ago can be returned; if a required detail is missing, ask for it, and do not make up rules" defines the task, the evidence and the fallback behavior. It is the difference between asking a friend to "get something from the shop" and handing them a list with a budget.

In the assistant, the final prompt is usually assembled by the application: developer instructions, the customer's message and policy text fetched from a knowledge base.

• Engineering relevance: precise prompts reduce ambiguity and make expected behavior testable.
• Common mistake: relying on prompt instructions as security. Instructions influence behavior; they do not enforce permissions. Anything that must never happen needs a control outside the model.

9. Context window: limited working space

The context window is how much material the model can consider in one go, measured in tokens. Think of a desk: you can spread notes across it while working, but the surface is finite.

What occupies that space includes system instructions, conversation history, retrieved documents and tool results; depending on the system, the generated output counts against the limit too. When a conversation grows too long, the application has to intervene, for example by dropping older turns, summarizing them, or retrieving selected pieces again when needed.

Context and parameters are different things. Parameters were fixed during training; context is information provided for the current task only.

• Engineering relevance: you have to budget the space deliberately. Stuffing in more text does not mean supplying more useful text, and irrelevant material can distract the model.
• Common mistake: treating a large window as unlimited memory, or assuming every detail in it will be used correctly.

10. Attention: relating parts of the input

Attention is how a model mixes information drawn from different places in its input. Take a message saying the headphones were delivered yesterday but were ordered ten days earlier. For a returns question, whether the clock starts at delivery or at purchase may be decisive, and attention is what lets the representation of the question reflect the relationship between delivery, ordering and the two points in time.

The everyday analogy is rereading the relevant sentences of a passage before answering a question about it. Technically, attention computes numeric relevance scores between positions and uses them to form weighted mixtures of information. Different heads and layers can pick up different kinds of relationships.

• Engineering relevance: attention explains why wording, order and surrounding text change the answer, which matters when you design prompts and chunk documents.
• Common mistake: reading attention weights as a full explanation of the model's reasoning. They are one mathematical signal, not a transparent account of why an answer was produced.

Grounding and adapting the model

The model's training alone cannot know this shop's current policy. Two techniques address that gap in very different ways.

11. RAG: look it up, then answer

Retrieval-Augmented Generation means finding relevant information at request time and placing it in the model's context so the answer can draw on it. It resembles an open-book exam: the book is available, but the student still has to understand the question, locate the right page and apply it properly.

For the shop assistant, the flow typically looks like this:

• The customer's question arrives.
• The application searches the policy documents for relevant passages.
• The best-matching passages are inserted into the prompt alongside the question.
• The model writes an answer grounded in those passages.

Retrieval can rely on keyword search, embeddings or a hybrid of both. Crucially, it leaves the model's parameters untouched. The existing article RAG explained: how AI systems retrieve fresh knowledge on demand goes deeper into the retrieval side.

• Engineering relevance: RAG is how applications use private, specialized or frequently changing documents without retraining.
• Common mistake: assuming RAG makes answers correct. It reduces unsupported answers, but the system can still fetch the wrong passage or misinterpret the right one.

12. Fine-tuning: additional practice on a narrower task

Fine-tuning continues training a model that is already trained, usually on a smaller, focused dataset aimed at a particular objective. It is like a fluent English speaker practicing concise customer-support replies specifically.

A business might fine-tune on pairs of customer questions and ideal responses to teach a consistent tone, output format or specialized task behavior. Unlike putting a policy in the prompt, fine-tuning modifies learned parameters, or adds trained components that alter behavior. It can also inject domain knowledge, but it is a poor mechanism for facts that change often, because every change would require another training run.

• Engineering relevance: fine-tuning pays off when a repeated behavior cannot be achieved reliably through prompting.
• Common mistake: seeing RAG and fine-tuning as rivals. RAG supplies reference material at answer time; fine-tuning shapes behavior through training. Many systems use both.

When the system acts, and how you know it works

The final group covers what goes wrong, what changes when the assistant can do things, and how to measure the whole.

13. Hallucination: fluent, confident and unsupported

A hallucination is output that is false, invented or not backed by the relevant evidence. Suppose the shop allows returns within seven days, and the assistant confidently replies that all headphones can be returned within thirty. The sentence is plausible and well formed; it is also wrong.

The root cause is that language models generate likely continuations rather than checking claims against reality. A high probability for the next token says nothing about whether the complete statement is true. Hallucinations show up as made-up policies, citations, people, figures and even fabricated tool results.

• Engineering relevance: a polished wrong answer can cause real harm, so systems need trustworthy evidence, output validation and a sanctioned way to admit uncertainty.
• Common mistake: interpreting confident phrasing as reliability.

14. Agent: choosing and executing steps toward a goal

An AI agent is a system that uses a model to decide on and carry out steps toward an objective, usually by invoking tools. A plain chatbot can explain the return process; an agent can look up the order, check eligibility and draft the return request.

A typical loop for the returns case:

• Read the customer's goal.
• Decide which information is missing, such as the order record.
• Call an order-lookup tool provided by the application.
• Evaluate the result against the policy.
• Either prepare the return, ask a clarifying question, or hand off to a human.

The model never reaches into a database by itself. The application exposes the tool, checks permissions and executes the operation; the model only proposes the call. Autonomy varies widely, from a single tool call to long multi-step sequences, and not every fixed automation deserves the label "agent". For more on the loop itself, see understanding AI agents: goals, tools, memory and the agent loop.

• Engineering relevance: actions need boundaries, error handling and approval checkpoints. Drafting a refund is not the same as authorizing one.
• Common mistake: equating the ability to act with permission to act unsupervised.

15. Evaluation: evidence that the application works

Evaluation is the systematic measurement of how well a model or a complete AI application performs its intended job. Think of a bicycle: it has to pass brake, steering and wheel checks under varied conditions before you ride it in traffic, no matter how good it looks.

For the returns assistant, a useful test set would include at least:

• A return that is clearly eligible.
• A return requested after the allowed window.
• A question that omits the purchase date.
• Two policy documents that contradict each other.
• An order lookup that fails.
• An attempt to obtain a refund the customer is not entitled to.

Measure answer correctness, whether answers are backed by evidence, whether uncertainty is expressed appropriately, whether tools are used safely, plus latency and cost. Include cases the team did not use while building the system; testing only on familiar examples overstates quality.

• Engineering relevance: a demo shows that one example worked. Evaluation tells you how often the system works and where it breaks.
• Common mistake: judging the model's prose instead of testing the full application end to end.

Key takeaways

• The model is only one part of an AI application; data sources, prompts, tools, permissions and tests determine most real-world behavior.
• Parameters are fixed at inference time. Anything the model should know about today, such as a current policy, must arrive through context, typically via retrieval.
• Tokens and the context window are budgets. Decide deliberately what earns a place in them.
• Similarity, fluency and confidence are not correctness; validation and evidence are.
• Tools let a system act, but authority belongs in application code and approval steps, not in prompt instructions.
• Evaluate the whole pipeline against realistic and adversarial cases before trusting it with customers.