Ten ways to cut hallucinations without fine-tuning
Ten system-level fixes that cut LLM hallucinations in an e-commerce agent: better context, retrieval, graphs, tools, validation and evals, with no fine-tuning.
This article is published in English.
Sources: 10 Ways to Reduce AI Hallucinations Without Fine-Tuning Your Model
When an assistant tells a customer "your refund has been processed" and no refund exists, the model usually isn't the culprit. The system around it left a gap, and the model filled that gap with a guess. Order status was missing, retrieval returned junk, the context was cluttered, tool calls went unchecked, earlier successes were forgotten.
You can cut hallucinations a lot without touching the model's weights. The running example here is an e-commerce assistant that checks orders, issues refunds and updates deliveries, and we'll harden it one layer at a time.
The naive version
A customer writes: "Where is my order #48291?" The first-draft app just forwards the question:
const response = await client.responses.create({
model: "gpt-5",
input: `
Where is order #48291?
`
});
console.log(response.output_text);
The model might come back with "Your order is out for delivery and will arrive today." Lovely, except it has no idea where order 48291 is. It produced a believable sentence from patterns in its training data. That's a hallucination.
What you want looks more like this:
Customer
↓
AI Agent
↓
Order System
↓
Actual Order State
↓
AI Agent
↓
Response
The model should look up reality, not make it up.
1. Hand over context you already have
Rule one: don't make the model guess what your backend knows. Say the backend holds this record:
{
"orderId": "48291",
"status": "SHIPPED",
"carrier": "FedEx",
"trackingNumber": "784512963",
"estimatedDelivery": "2026-08-25"
}
Pass it in with the question:
const context = {
order: {
id: "48291",
status: "SHIPPED",
carrier: "FedEx",
trackingNumber: "784512963",
estimatedDelivery: "2026-08-25"
}
};
const response = await client.responses.create({
model: "gpt-5",
input: `
Answer the customer using only the supplied order information.
Context:
${JSON.stringify(context, null, 2)}
Customer:
Where is my order #48291?
`
});
Now the model isn't asked to know the order state. Its job shrinks to explaining a known fact to the customer, which is much safer. Compare the two flows:
Bad:
Customer → LLM → Answer
Better:
Customer
↓
Retrieve state
↓
Build context
↓
LLM
↓
Answer
This is the core of context engineering.
2. Retrieve first, generate second
Another customer asks about the return policy for electronics bought in a holiday sale. That answer doesn't live in the model. It sits in your policy database, product docs, promotion rules and regional terms. So fetch it before generating. A basic RAG flow:
Customer Question
↓
Retrieve
↓
Rank
↓
Build Context
↓
LLM
↓
Answer
And a bare-bones implementation:
const results = await vectorStore.search({
query: customerQuestion,
topK: 10
});
const relevant = results
.filter(item => item.score > 0.8)
.slice(0, 5);
const context = relevant
.map(item => item.content)
.join("\n\n");
const answer = await generateAnswer(
customerQuestion,
context
);
Quality beats quantity here. Feeding the model 30 documents doesn't make it smarter. If three matter, the other 27 are clutter.
3. Cut the noise
Picture the assistant receiving this pile:
20 retrieved documents
+
15 previous messages
+
10 previous tool responses
+
customer profile
+
product catalog
+
order history
+
promotion metadata
Now the customer only asks, "Can I return these headphones?" The model doesn't need the last fifteen conversations, unrelated orders, other product lines, expired promotions or old shipping rules. A better pipeline filters before prompting:
Available Information
↓
Relevance Filtering
↓
Metadata Filtering
↓
Ranking
↓
Deduplication
↓
Context Compression
↓
LLM
In code, something like:
const context = results
.filter(x => x.score >= 0.82)
.filter(x => x.metadata.category === "returns")
.filter(x => x.metadata.region === customer.region)
.sort((a, b) => b.score - a.score)
.slice(0, 5)
.map(x => x.content);
You're not trying to maximize tokens. You're trying to raise the signal-to-noise ratio. A good rule: give the model everything it needs and nothing it doesn't.
4. Use a knowledge graph for structured facts
RAG suits documents. Many e-commerce questions aren't about documents at all. They're about relationships: which warehouse fulfills this order, which payment paid for it, which products belong to a promotion, which orders are hit if a warehouse goes offline. Those are graph problems. A simplified graph:
Customer
│
└── PLACED → Order
│
├── CONTAINS → Product
│
├── PAID_BY → Payment
│
├── FULFILLED_BY → Warehouse
│
└── SHIPPED_BY → Carrier
When a user asks which warehouse is handling their order, the system can walk this path:
Order #48291
↓
FULFILLED_BY
↓
Warehouse #17
with a graph query like:
MATCH (o:Order {id: "48291"})
-[:FULFILLED_BY]->
(w:Warehouse)
RETURN w.id, w.name, w.location;
The result becomes trusted context:
{
"orderId": "48291",
"warehouse": {
"id": "WH-17",
"name": "Delhi Fulfillment Center",
"location": "Delhi"
}
}
Set the two approaches side by side. Without structure:
Without structured knowledge:
User → LLM
↓
Guess
And with a graph in the loop:
With Knowledge Graph:
User
↓
Entity Identification
↓
Graph Traversal
↓
Verified Relationship
↓
Context
↓
LLM
It pays off most when the assistant must reason across several systems.
5. Ask for evidence
Don't just ask for the answer. Ask for the answer plus what supports it. A structured response might look like:
{
"answer": "Your order is being fulfilled by Warehouse WH-17.",
"confidence": 0.98,
"evidence": [
{
"type": "order_record",
"source": "orders_db",
"orderId": "48291"
},
{
"type": "warehouse_relationship",
"source": "knowledge_graph",
"warehouseId": "WH-17"
}
]
}
You can enforce the habit through instructions:
For every factual claim:
1. Identify supporting evidence.
2. Use only available evidence.
3. Never invent a source.
4. If evidence is unavailable, say so.
5. Clearly distinguish facts from inference.
Then the application can act on confidence:
if (result.confidence < 0.7) {
return escalateToHuman(result);
}
That changes a bare flow:
LLM → Answer
into one with checks along the way:
LLM
↓
Answer
↓
Evidence
↓
Confidence
↓
Decision
6. Give the agent tools instead of letting it guess
Sometimes the best prevention is not letting the LLM answer at all. If a customer asks whether a refund went through, expose a lookup:
const tools = [{
name: "get_refund_status",
description: "Retrieve the current refund status for an order",
parameters: {
type: "object",
properties: {
orderId: {
type: "string"
}
},
required: ["orderId"]
}
}];
Execution then runs like this:
Customer
↓
LLM
↓
get_refund_status()
↓
Payment System
↓
Actual Refund State
↓
LLM
↓
Customer
Instead of a vague "your refund should arrive in 5 to 7 days," the agent can say the refund was started yesterday and is still pending, because it actually looked. If the real world can answer, ask the real world.
7. Validate tool inputs and outputs
Tool calling alone doesn't make an agent reliable. The model can still produce a malformed request. A customer says to refund $100 from order 48291, and the model emits:
{
"orderId": "48291",
"amount": "one hundred",
"currency": "dollars"
}
but your payment API wants:
{
"orderId": "48291",
"amount": 100,
"currency": "USD"
}
Add schema validation, for instance with Zod:
import { z } from "zod";
const RefundRequest = z.object({
orderId: z.string(),
amount: z.number().positive(),
currency: z.enum(["USD", "EUR", "GBP", "INR"])
});
const refundRequest = RefundRequest.parse(
modelOutput
);
Don't stop at types. Check business rules too:
if (refundRequest.amount > order.total) {
throw new Error(
"Refund amount exceeds order total"
);
}
The pipeline then becomes:
LLM
↓
Schema Validation
↓
Business Validation
↓
Permission Check
↓
Execute
↓
Response Validation
↓
Accept / Retry / Escalate
This matters most for agents that take real actions. The LLM should never have the last word over data that gets executed.
8. Keep facts apart from reasoning
A subtle but effective trick is storing what the system knows separately from what the model concluded:
{
"facts": [
"Order 48291 is shipped",
"Order 48291 is fulfilled by Warehouse WH-17",
"Warehouse WH-17 is currently operating"
],
"reasoning": [
"The order is likely to remain on schedule"
],
"conclusion": "The order is currently expected to arrive on time."
}
They carry different certainty. The first statements came from systems, while the last is an inference. When a final answer turns out wrong, you can ask one clean question:
Was the fact wrong?
OR
Was the reasoning wrong?
That's far easier to debug. You can even surface the split internally:
FACT
→ Order shipped
FACT
→ Estimated delivery: Aug 25
INFERENCE
→ Delivery is currently expected on schedule
A little separation gives you a lot of traceability.
9. Learn from runs, not from training
Let's make the agent improve without fine-tuning. A customer asks to cancel order 48291. The agent checks the order:
{
"orderId": "48291",
"status": "PROCESSING",
"cancelable": true
}
Then calls:
cancel_order(48291)
and gets back:
{
"success": true,
"cancellationId": "CAN-83921"
}
That outcome is worth keeping. Store the successful run:
{
"task": "Cancel order",
"orderState": "PROCESSING",
"action": "cancel_order",
"result": "SUCCESS",
"cancellationId": "CAN-83921"
}
When a similar request shows up later, look for the matching past success first:
New Request
↓
Find Similar Successful Execution
↓
Retrieve Relevant Pattern
↓
Check Current Order State
↓
Generate Action
↓
Validate
↓
Execute
Failures teach too. An attempt that went wrong leaves a trace:
Attempt 1
↓
cancel_order()
↓
Rejected: Order already shipped
↓
Agent retrieves shipping information
↓
Explains cancellation is unavailable
Those traces become knowledge, and the feedback loop looks like this:
Execute
↓
Observe
↓
Evaluate
↓
Store Experience
↓
Improve Future Context
Call it runtime learning. No retraining involved.
10. Evaluate every change
Finally, measure whether anything helped. Say you altered the retrieval approach. "The answers feel better" isn't evidence. Build an evaluation set:
[
{
"question": "Where is order 48291?",
"expected": "SHIPPED"
},
{
"question": "Which warehouse fulfills order 48291?",
"expected": "WH-17"
},
{
"question": "Can order 48291 be cancelled?",
"expected": false
}
]
Then track metrics such as these:
Answer Accuracy
Groundedness
Retrieval Precision
Tool Selection Accuracy
Tool Success Rate
Task Success Rate
Recovery Rate
Hallucination Rate
Latency
Cost
A before-and-after table makes the gain concrete:
Before After
Answer Accuracy 72% 95%
Groundedness 69% 97%
Tool Success 81% 98%
Hallucination Rate 17% 3%
Now you're not claiming the new prompt works better, you're showing it. This is where AI engineering starts to resemble ordinary software engineering:
Change
↓
Test
↓
Measure
↓
Compare
↓
Improve
All ten layers together
Stack the ten layers around the LLM and the architecture comes out like this:
┌──────────────────┐
│ Knowledge Graph │
└────────┬─────────┘
│
┌────────▼─────────┐
│ RAG / Search │
└────────┬─────────┘
│
Customer → Intent → Context Engine → LLM
↑ │
│ ▼
Memory Tool Selection
↑ │
│ ▼
│ Validation
│ │
│ ▼
│ Real Systems
│ │
│ ▼
└─────── Feedback
│
▼
Evaluation
Notice what didn't happen: the model wasn't fine-tuned. The system around it improved.
Questions to ask before blaming the model
The first instinct on a hallucination is often a better model. Sometimes that's right. First, though, run through a checklist.
- Was the right context supplied?
- Was the right information retrieved?
- Was irrelevant material removed?
- Could structured knowledge answer it deterministically?
- Could a tool read the actual system state?
- Was the model's output validated?
- Were facts kept apart from reasoning?
- Did the system learn from earlier runs?
- Was evidence provided?
- Are the results measured?
Those checks usually lead to something cheaper, safer and easier to control than fine-tuning.
The model is one component
A useful picture:
┌───────────────────┐
│ Knowledge + RAG │
└─────────┬─────────┘
↓
Customer → Context → LLM → Tools → Real World
↑ ↓ ↓
Memory Reasoning Validation
↑ ↓
└──── Feedback
↓
Evaluation
The LLM produces possibilities. Everything around it supplies facts, context, constraints, tools, validation, memory and feedback, and that's what makes the whole thing dependable.
Hallucinations are often a system design problem more than a model problem. Better context, better knowledge, better tools, validated output and learning from what really happened will get you further than a new training run. The practical question moves from which model to use toward what the model should know, what it may do, and how you'll tell whether it got things right.