Six AI Concepts That Tell You What to Check Before Trusting an Answer

You hand an AI assistant a report and get back a polished summary that includes a figure you cannot find anywhere in the document. When you ask about it, the assistant apologizes and offers a different figure. Was information missing, did retrieval fail, or was the number simply invented? Six core concepts, understood in terms of the decisions they affect, let you replace "why is it lying?" with a precise diagnostic question, and they also help you control cost and evaluate vendor claims.

Why skepticism is a reasonable default

Distrust of AI output is common even among people who use it every day. In the 2025 Stack Overflow Developer Survey, 46% of respondents to the accuracy question said they distrusted AI output, against 33% who trusted it, from 33,244 responses (survey results). Read that carefully: it records developers' opinions, not a measurement of model accuracy, and it is not a representative sample of the population. It does capture a real tension, though. People adopt these tools and still do not believe everything they say.

First, apply the same scrutiny to headlines

Content about AI often promises that learning a handful of terms puts you "ahead of 90% of people". That sounds like a research result, but without a study behind it, it is marketing dressed up as a statistic. Ask the obvious questions: ahead of whom, measured how, with which participants, and where are the results published? No assessment, no sample, no data means the percentage is unsupported. That does not make every explanation attached to it wrong, and it says nothing about anyone's intentions; it simply means the number carries no weight.

Knowing definitions is a starting point. Applying them, recognizing the exceptions and checking real output are separate skills, and none of them is measured by a flattering percentage.

The same discipline applies to promises that a single prompt can stand in for a whole team, or that some tool multiplies everyone's speed tenfold. Request the specific task, the baseline, how it was measured and what the limitations were. A single impressive demo cannot establish a general result. A strong headline is fine; the trouble starts when the claim is more precise than the evidence supporting it. Hold this guide to the same standard.

1. Tokens: the real size of a task

A token is the unit of text a language model actually processes. Depending on the tokenizer, a token might be a whole word, a fragment of a word, a punctuation mark or some other piece of text, and the tokenizer maps each piece to a numeric ID.

There is no fixed rule such as "one word equals one token". Hugging Face's tokenizer overview describes several approaches, including byte-pair encoding, WordPiece and methods related to SentencePiece, and the same sentence can be split quite differently by different tokenizers. Languages other than English, code and unusual formatting often use more tokens per word.

Why this matters becomes clear with a question like:

Which three customer complaints came up most often?

The question itself is tiny. But if answering it means reading thousands of support messages, the question is a rounding error in the total input. As a back-of-the-envelope illustration rather than a measurement: 400 messages at around 150 tokens apiece already add up to 60,000 tokens before any instructions or other context.

So the useful question is not how long your prompt is, but how much material the task forces the system to process. Since pricing, latency and context limits are usually expressed in tokens, that is also what drives cost. Before a large document task, strip out repeated email signatures, irrelevant appendices and duplicate records that add no evidence, while keeping everything the answer genuinely depends on.

2. Context window: what the model can see in one request

The context window caps how much material a model can work with in a single request. System instructions, conversation history, retrieved passages and any other input all consume that budget; how output tokens are counted against it depends on the model and API. Google's long-context documentation shows how large windows make it possible to work across big collections of text and other media.

Accepting a document, however, is not the same as reliably using every relevant detail in it. The 2023 study "Lost in the Middle" tested multi-document question answering and key-value retrieval and found that, for the models examined, accuracy often dropped when the relevant information sat in the middle of a long input instead of near the start or end. Treat that as historical evidence about those specific experiments, not as a score for today's models, but the lesson about checking still applies.

Chat products also rarely behave the way their interface suggests. When a conversation outgrows the window, an application does not have to just drop the oldest messages; it may summarize, select or retrieve earlier material instead. What you see in the chat history is not a reliable picture of what reaches the model on each call.

In practice:

• For long-running work, maintain a short, explicit brief with the current requirements and decisions, and restate it when it matters.
• When a conclusion hinges on one passage, ask the assistant to quote or locate that passage before it draws the conclusion.

A big window gives the system room to work; it does not prove the system used the right evidence.

3. Temperature: control over variety, not truth

At each step of generation, the model scores every possible next token. Temperature reshapes the probability distribution used to sample from those scores: low values concentrate probability on the most likely candidates, while high values spread it across more options. Hugging Face documents temperature together with related generation controls such as top-p sampling and greedy decoding.

The tempting shortcut is "low temperature means factual". It does not. If the model's most likely answer is wrong, making it less varied cannot supply the missing fact; it only makes the same mistake more consistently. Raising temperature, equally, does not guarantee better ideas, only more varied ones.

Two contrasting tasks show the difference. Generating five names for a fictional café benefits from variety. Extracting invoice numbers benefits from consistent formatting, but the numbers must still match the actual invoices, and temperature has nothing to say about that.

Treat temperature as one setting to experiment with, and evaluate against what the task actually needs. For extraction, count wrong and missing fields; for brainstorming, ask whether the ideas are both usable and genuinely different from each other. Predictability and correctness need separate checks.

4. Hallucination: output that outruns its evidence

Here, a hallucination means generated content that is fabricated, factually wrong or not supported by the material it claims to describe. A confident tone makes it harder to catch, but confidence is not part of the definition; a hedged sentence can be just as unsupported.

A made-up research paper is the obvious case. A subtler and more common one is a real paper cited for a result it never reported, which survives a quick glance precisely because the citation exists.

The TruthfulQA benchmark introduced 817 questions across 38 categories built around common misconceptions. In its original evaluation, the best model tested was truthful on 58% of questions, compared with 94% for humans. These are historical results from 2021 and 2022 research, not a measure of current chatbots and not a universal hallucination rate. What the work does illustrate is that models can faithfully reproduce false beliefs that appear in human-written text.

When a summary contains a surprising number, ask for its provenance explicitly:

Point to the source passage, give its date and the population it measured. If the passage does not support the figure, flag the figure as unsupported.

Then check the reference with your own eyes. Any citation a model produces is only an assertion until you have confirmed both that the page exists and that it actually supports the specific sentence.

5. RAG: fetching evidence before answering

Retrieval-augmented generation pairs a search step with a generation step. The system looks up relevant material in an external collection, passes it to the model, and asks the model to answer from that material.

The influential 2020 RAG paper combined a pretrained generator with a retriever over a Wikipedia index and, when published, set the best known scores on three open-domain QA benchmarks. Those results describe one research system, not a quality guarantee for every product that carries the RAG label.

Consider an employee asking how many days they have to file an expense claim. A good system retrieves the current policy and answers from it. If it retrieves last year's policy instead, well-written prose will not fix the error; the answer is fluent and wrong. That is why RAG failures are usually easier to diagnose by stage than by staring at the final answer, an approach covered in evaluating RAG by failure stage.

Two misconceptions are worth clearing up:

• RAG does not depend on having a specialized vector store. The lookup step might be classic keyword matching, embedding similarity or a hybrid of both, and Microsoft's RAG overview discusses these options along with the importance of preparing content so it can be searched well.
• Uploading a PDF does not prove that retrieval is happening. Some systems place the document's content straight into the model's context window, as the Google long-context documentation describes. Retrieval and direct long-context processing are distinct design choices, and a product may combine them.

To evaluate any document assistant, ask two separate questions: did it find the right passage, and does its answer faithfully reflect that passage?

6. Agents: systems that choose their next step

"Agent" is used loosely, so a concrete distinction helps. Anthropic's guide to building effective agents describes workflows as systems that follow predefined code paths, whereas agents let the model dynamically direct its own process and tool use.

A fixed workflow might extract fields from an invoice, validate them and save a record, always in that order. An agent faced with an incomplete invoice might decide on its own to open an attachment, look up the related order and send a request for the missing details.

The key question therefore becomes: which decisions and actions is the system allowed to take? A drafted reply and an issued refund carry very different consequences. An agent needs clearly scoped permissions, observable results for each action and a way to stop when it cannot determine a sensible next step.

More steps also mean more chances to fail. As a deliberately simplified illustration, if a task requires ten steps and each succeeds independently with 95% probability, the chance that all ten succeed is 0.95 to the tenth power, roughly 60%. In practice agent steps depend on one another and retries alter the math, so this is an intuition about compounding risk rather than a benchmark. The practical consequence is to measure whether the whole task was completed correctly, side effects included, not whether individual steps looked plausible. For a fuller treatment of the loop itself, see understanding AI agents: goals, tools, memory and the agent loop.

A six-question checklist for real tasks

Each concept maps to a question you can ask of any real task:

• Tokens: how much material does this task actually require the system to process, and what can be removed without losing evidence?
• Context window: which passage did the answer depend on, and did the system really use it?
• Temperature: is this task about variety or consistency, and have correctness and predictability been checked separately?
• Hallucination: where exactly is the source for each surprising claim, and does it say what the answer says it does?
• RAG: was the right, current document retrieved, and was it represented accurately?
• Agents: what is the system permitted to do, and was the entire task, including its side effects, completed correctly?

Wrapping up

Try these questions on a single real task, such as a document summary, a code change or a customer-support reply, with the source material open beside you. Distinguish the evidence it drew on, the conclusions it reached on its own and the actions it really performed. Doing this consistently reveals more about a tool's reliability than any demo or headline statistic, and it turns vague distrust into specific, fixable problems: a bloated input, a missed passage, a wrong document, an unsupported figure or an agent with too much latitude.