A Local AI Agent with Ollama: Narrow Tools, Validation and Hard Step Limits

The first version of a local invoice agent looks impressive for roughly five minutes. The job is simple: help a small business owner chase overdue invoices. The agent can look up customers, find unpaid invoices, work out how late they are and draft a reminder. A test prompt asks it to find the overdue invoices for Acme Plumbing and prepare a friendly reminder. It answers at once with an invoice, a balance and a polished email.

The trouble is that part of it is made up. The customer exists, but the invoice number doesn't. The amount sounds plausible and is wrong. The agent has produced the shape of a successful result without doing the work.

That episode teaches the most important lesson about building agents: sounding intelligent is not reliability. An agent is reliable when each significant claim traces back to a controlled action or a trusted source. What follows rebuilds the invoice agent from Ollama and Python, adding explicit tools, validation and a compact agent loop. Inference happens on the local machine, but the lessons apply to nearly any agent framework.

Why Ollama

A local setup avoids sending customer records to an outside model provider during development, and it allows swapping among open models without rebuilding the application. Ollama serves models locally behind a plain chat API. Tool calling is supported too, so a model can request a function rather than bluff an answer it doesn't have. Setup is short:

ollama pull qwen3
pip install ollama

A normal chat request from Python looks like this:

from ollama import chat

response = chat(
    model="qwen3",
    messages=[
        {"role": "user", "content": "Explain what an overdue invoice is."}
    ],
)print(response.message.content)

That yields a chatbot, not yet an agent.

A chatbot answers, an agent acts in steps

The word agent gets used loosely. A workable definition is a model running inside a loop, with access to tools, state and rules that govern what it may do next. For the invoice task the loop goes:

1. The user makes a request.
2. The model decides whether a tool is needed.
3. The application validates the tool request.
4. The application runs the tool.
5. The result is appended to the conversation.
6. The model asks for another tool or gives a final answer.

The model never touches the database. It can only ask the application to call a tool the developer defined. That boundary matters because language models are good at interpreting intent and generating text, while application code is better at permissions, queries, calculations and business rules.

Narrow tools first

A common first mistake is a single powerful tool, say manage_invoices, that can search, update, draft and send. It makes debugging hard and misuse dangerously easy. Small tools with one job each work better:

• find_customer turns a customer name into a real customer ID.
• get_overdue_invoices returns the overdue invoices for that ID.
• draft_reminder produces text and sends nothing.
• send_reminder performs the external action and needs approval.

A simplified local data layer supplies the facts:

CUSTOMERS = {
    "acme plumbing": {
        "customer_id": "cus_1042",
        "name": "Acme Plumbing",
        "email": "billing@example.com",
    }
}

INVOICES = [
    {
        "invoice_id": "INV-2048",
        "customer_id": "cus_1042",
        "amount": 1850.00,
        "days_overdue": 18,
    }
]
def find_customer(name: str) -> dict:
    customer = CUSTOMERS.get(name.strip().lower())
    return customer or {"error": "customer_not_found"}
def get_overdue_invoices(customer_id: str) -> dict:
    matches = [
        invoice
        for invoice in INVOICES
        if invoice["customer_id"] == customer_id
        and invoice["days_overdue"] > 0
    ]
    return {"invoices": matches, "count": len(matches)}

Look at what these functions refrain from doing. They never guess. A missing customer produces an explicit error, and no matching invoices produce an empty list.

Real tools, not imagined access

Ollama accepts Python functions as tools, and the names, parameters and docstrings give the model what it needs to pick the right one.

import json
from ollama import chat

def find_customer(name: str) -> dict:
    """Find a customer by business name and return its verified record."""
    customer = CUSTOMERS.get(name.strip().lower())
    return customer or {"error": "customer_not_found"}
def get_overdue_invoices(customer_id: str) -> dict:
    """Return overdue invoices for a verified customer ID."""
    matches = [
        invoice
        for invoice in INVOICES
        if invoice["customer_id"] == customer_id
        and invoice["days_overdue"] > 0
    ]
    return {"invoices": matches, "count": len(matches)}
TOOLS = {
    "find_customer": find_customer,
    "get_overdue_invoices": get_overdue_invoices,
}

The registry dictionary is more than a convenience, since it is an allowlist. Even if the model requests a function called delete_customer, the application refuses because no such tool exists in TOOLS.

The agent loop

The loop is what turns tool calling into an agent. It starts with a system prompt:

SYSTEM_PROMPT = """
You are an invoice assistant.

Rules:
- Never invent a customer, invoice, email address, balance, or date.
- Use find_customer before requesting invoices.
- Only use customer IDs returned by tools.
- If a tool returns an error or no records, explain that clearly.
- You may draft communication, but you cannot send it.
"""
def run_agent(user_request: str) -> str:
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_request},
    ]    for _ in range(6):
        response = chat(
            model="qwen3",
            messages=messages,
            tools=list(TOOLS.values()),
        )        messages.append(response.message)        if not response.message.tool_calls:
            return response.message.content        for call in response.message.tool_calls:
            name = call.function.name
            arguments = call.function.arguments            if name not in TOOLS:
                result = {"error": "tool_not_allowed"}
            else:
                try:
                    result = TOOLS[name](**arguments)
                except (TypeError, ValueError) as error:
                    result = {
                        "error": "invalid_tool_arguments",
                        "detail": str(error),
                    }            messages.append(
                {
                    "role": "tool",
                    "tool_name": name,
                    "content": json.dumps(result),
                }
            )    return "I stopped because the task exceeded the maximum number of steps."

The cap of six iterations is deliberate, because an agent must never run forever while reconsidering the same problem. The loop also treats tool output as fresh evidence. The model resolves the name "Acme Plumbing" first, gets cus_1042 back, and requests invoices for that verified ID only after that.

A better prompt wasn't the fix

Hallucinations tempt a developer to add one more sentence to the system prompt, such as a plea to be accurate and never invent anything. That instruction helps but isn't a control. The reliable fix came from restructuring:

• The database stays the source of truth.
• The model may request data and can't invent a record that passes as valid.
• Tool names are allowlisted and arguments are validated.
• The loop has a hard step limit.
• Reads and writes are kept apart.
• Sending an email needs human approval.

Prompts influence behavior, and code enforces it.

Structured output at the boundary

Free-form text suits a friendly email but is risky when another part of the application consumes the answer. If the agent must return a review screen with the customer, invoice IDs, total balance and a draft, a Pydantic model can define that contract:

from pydantic import BaseModel, Field

class ReminderReview(BaseModel):
    customer_name: str
    invoice_ids: list[str]
    total_due: float = Field(ge=0)
    draft_subject: str
    draft_body: str
    requires_approval: bool = True

Ollama accepts a JSON schema through its format parameter:

review_response = chat(
    model="qwen3",
    messages=messages,
    format=ReminderReview.model_json_schema(),
)

review = ReminderReview.model_validate_json(
    review_response.message.content
)

That guarantees the response has the expected shape. It doesn't prove the values are true, so invoice_ids and total_due still get compared with actual tool results before the approval button appears. Schema validation asks whether something is shaped correctly. Business validation asks whether it is allowed and true. A production agent needs both.

State is not memory

Long conversations expose another problem: the model forgets which customer was selected or blends an old invoice into a new request. The cause is treating the whole chat transcript as application state. The conversation is useful context, but critical state belongs in an explicit structure:

task_state = {
    "customer_id": "cus_1042",
    "verified_invoice_ids": ["INV-2048"],
    "approved_actions": [],
}

The model may summarize events, yet the application owns verified IDs and approvals. If the conversation gets truncated to fit the context window, those facts survive.

Context length also costs performance, since more context uses more memory and takes longer to process. Rather than sending every message and database row indefinitely, pass the smallest set of information the current decision needs.

Local doesn't mean safe

Running a model through Ollama keeps inference under local control but doesn't secure the rest of the application. An agent can still leak data into logs, call an unsafe internal endpoint, follow malicious instructions hidden in a document or act with excessive permissions. The usual backend security principles apply:

• Give each tool only the permissions it needs.
• Keep secrets out of prompts and source code.
• Treat retrieved text as untrusted data, never as system instructions.
• Record every tool call with its result, latency and the final decision.
• Strip sensitive fields from results before they enter the model's context.
• Demand approval for external or irreversible actions.
• Make important writes idempotent, so retries don't duplicate them.

The model should never be the security boundary.

Testing

Happy-path demos hide most agent failures, so a small evaluation set is worth building. Cases cover a known customer with one late invoice and one with none, a name that matches nobody, two lookalike company names, an attempted send that skips approval, an injected instruction to drop the rules, a badly formed tool argument and a tool that times out. For each, record the expected tools, permitted actions and final outcome, and rerun the suite whenever the model, system prompt or tool descriptions change.

The most useful measurement isn't whether the final reply sounds good. It is whether the agent followed the right sequence of verified steps.

The working version

After the redesign, the original request leaves a trace like this:

User request
  → find_customer(name="Acme Plumbing")
  → verified customer_id: cus_1042
  → get_overdue_invoices(customer_id="cus_1042")
  → verified invoice: INV-2048, $1,850, 18 days overdue
  → generate draft
  → wait for human approval

It is less magical than the first demo and far more useful. A nonexistent customer makes the agent stop and ask for clarification. No overdue invoices gets reported as such. An unknown tool request is rejected by the application, and a loop that runs too long stops safely. None of that is glamorous, and all of it builds trust.

The final lesson

Ollama makes running a capable local model easy, but the model is only one part. The real agent is the system around it: the loop, tool contracts, source-of-truth data, validation layer, state machine, permission boundaries and approval step.

For a first agent, skip the ten tools and full application access. Start with one real problem and two narrow read-only tools, record every decision, add failure cases, and introduce write access only after the read path is boringly reliable. The shift that matters is replacing the question of how to make the model more autonomous with the question of how to make each step observable, verifiable and safe. Autonomy makes a good demo, while controlled autonomy makes a useful product.