A Local Support Agent With SmolLM3: Think Mode, Tools and Guardrails

Can a 3B-parameter model handle a support ticket without a 70B model standing behind it? For the dull 90% of tickets, yes, but only once you stop trusting the model's own tool calls. Here SmolLM3 runs locally as a small support agent and shows where it behaves and where the code around it has to be stricter than the model.

Customer support is a poor place to drop a general chatbot. Typical tickets need a lookup, a label and a brief policy answer, while a minority are tangled up in duplicate charges, lost orders or upset customers. Serving both well from one model means running it in two different ways.

SmolLM3-3B suits this for three reasons that the parameter count hides:

• Two reasoning modes ship with it. Adding /no_think makes it answer straight away, and /think has it write a hidden reasoning trace beforehand.
• Tool calling is built in: tools described as JSON schemas go into the chat template, and the model answers with a <tool_call> block when it wants one run.
• It's small enough for a laptop. Everything here ran on a MacBook with 24GB of unified memory: the full-precision model through Transformers on MPS, and an 8-bit MLX build for the agent loop.

No training takes place, and the weights are never touched. The work consists of separating thinking from acting and making the surrounding code stricter than the model.

The setup

Two scripts use two runtimes but one model family. For the think-mode comparison, sampling follows the model card, with temperature=0.6 and top_p=0.95. The agent runs cooler at temperature=0.3 so the tool JSON stays dull.

On this machine the 8-bit agent round trip took about four seconds. The full 3B model under Transformers produced roughly 10 tokens per second, which is fine for reading logs later and too slow for a customer watching a chat window. Transformers is handy for studying the template. When a real person is on the other end, switch to MLX or llama.cpp.

Thinking is a product decision

SmolLM3 thinks by default. There are two equivalent ways to turn that off. One is a system message:

messages = [
    {"role": "system", "content": "/no_think"},
    {"role": "user", "content": prompt},
]

The other is a flag when rendering the template:

tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,  # the /no_think flag in the system prompt wins if both are set
)

The template does more than flip a boolean. It writes the reasoning mode into the system header, and when /no_think is chosen it additionally inserts an empty think block ahead of time, so generation continues directly with the reply:

<|im_start|>assistant
<think>

</think>

That's why a plain generate() call in no-think mode can still return <think></think> in the text. For a customer-facing reply, strip it:

final = re.sub(r"<think>.*?</think>", "", raw, flags=re.DOTALL).strip()

The same support prompt went to both modes: a customer is charged twice for one order, so what three concrete steps should support take?

Two clocks

Throughput was about the same in both modes, but the amount of work wasn't. Think mode spent most of its 939 tokens inside the think block. The customer never reads that trace but still waits through it.

Two playbooks

/no_think went straight to a stock checklist: confirm the order, reach out to the merchant, refund the charge. It works as a first draft, but it's slightly off for a merchant's own support team, since "contact the merchant" is advice a bank gives, not something an in-house agent tells its own customer.

/think spent about a page arguing with itself over refund versus credit, escalation and whether to check shipping, and then landed on a tighter plan:

1. Apologise and collect the order number, date and payment method.
2. Locate the duplicate transaction and cancel it.
3. Explain to the customer what went wrong and confirm the money is coming back, as a refund or store credit.

The split worth shipping follows from that:

• /no_think for high-volume work: intent classification, "where is my order", canned policy answers and tool calls.
• /think where a wrong step is expensive: refunds, chargebacks, safety issues, and anything that needs a human-style checklist before it responds.

Hugging Face's own reported figures point the same way for tool use. On the BFCL benchmark, SmolLM3 is listed at 92.3 without extended thinking and 88.8 with it. Thinking isn't free, and for this kind of task it wasn't even better. The agent below forces enable_thinking=False, because an order lookup should be a function call and not an essay.

One tool and a two-turn loop

SmolLM3 expects tools as JSON-schema objects passed under xml_tools. The template copies them into a tools block and instructs the model to answer with this shape:

<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call>

The support tool is intentionally narrow:

TOOLS = [
    {
        "name": "lookup_order_status",
        "description": (
            "Look up the current status, estimated delivery date, and carrier "
            "for a specific customer order. Call this when the customer mentions "
            "an order number or asks where their order is."
        ),
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {
                    "type": "string",
                    "description": "The order ID, usually in the format ORD-XXXXXX.",
                }
            },
            "required": ["order_id"],
        },
    }
]

Behind it sits a fake database. In production that would be a SQL query or a call to an order service.

ORDERS = {
    "ORD-4821": {"status": "shipped", "eta": "June 18, 2026", "carrier": "DHL"},
    "ORD-3307": {"status": "processing", "eta": "June 20, 2026", "carrier": None},
    "ORD-1190": {"status": "delivered", "eta": None, "carrier": "FedEx"},
}

The loop is the whole product. The model gets the customer message, produces a first reply, and if that is a tool call the code runs the function, feeds the result back and asks for a second reply:

def respond_with_tools(user_message: str) -> str:
    messages = [{"role": "user", "content": user_message}]
    turn1 = generate_reply(messages)
    name, args = parse_tool_call(turn1)
    if name == "lookup_order_status":
        result = lookup_order_status(**args)
        messages += [
            {"role": "assistant", "content": turn1},
            {"role": "tool", "content": json.dumps(result), "name": name},
        ]
        return generate_reply(messages)
    return turn1

The happy path

A customer asks where order ORD-4821 is after a week. The model doesn't answer. It calls the tool:

<tool_call>
{"name": "lookup_order_status", "arguments": {"order_id": "ORD-4821"}}
</tool_call>

The function returns shipped, DHL, June 18, 2026. The second turn, 4.2 seconds in total, produces a reply saying the order has shipped and should arrive by June 18 via DHL.

For ORD-3307 the status is processing with no carrier and a June 20 estimate. The model said the carrier wasn't available instead of making one up, which is exactly what you want from a small model: read the tool JSON and don't decorate it.

For ORD-9999 the tool call was correct and the function answered not_found. The reply apologised and asked the customer to check the number. A structured not-found from the function was enough on its own, with no special prompting.

Where the naive loop dies

The third query was meant as a negative control: the customer just wants to change an email address. No email tool exists, so a good agent says so and stops. The naive loop did something else:

1. It began correctly, saying it can't change personal information and the customer should contact service.
2. Then it kept going and emitted a tool call carrying a placeholder:

<tool_call>
{"name": "lookup_order_status", "arguments": {"order_id": "your_order_id"}}
</tool_call>

1. The parser executed anything wrapped in <tool_call>. It searched the fake database for your_order_id, received not_found and passed that back to the model.
2. On the second turn the model exposed the tool's schema to the customer and told them to swap your_order_id for a real order number.

Four things failed at once, stacked on one another. The model's part is the least interesting. Small models hedge, and that's normal. If your loop amounts to a regex for <tool_call> followed by calling the function with whatever arguments appear, you'll eventually run lookup_order_status("your_order_id") in production, and it won't happen during a demo.

Guards that don't need fine-tuning

Two small rules fix most of it without changing weights.

First, run a tool call only when the whole turn is nothing but that call. A paragraph followed by a call counts as ordinary text, and the XML should be removed before the customer sees it.

Second, validate arguments before they reach the database. Order IDs here are ORD- plus digits, so your_order_id never gets through.

ORDER_RE = re.compile(r"^ORD-\d{4,}quot;)
PURE_TOOL_RE = re.compile(r"^\s*<tool_call>(.*?)</tool_call>\s*quot;, re.DOTALL)

def parse_executable_tool_call(output: str):
    cleaned = re.sub(r"<think>.*?</think>", "", output, flags=re.DOTALL).strip()
    match = PURE_TOOL_RE.match(cleaned)
    if not match:
        return None
    payload = json.loads(match.group(1).strip())
    order_id = str(payload.get("arguments", {}).get("order_id", "")).strip()
    if not ORDER_RE.match(order_id):
        return None
    return payload

Tighten the tool description as well, saying to call it only when the customer has already supplied a concrete order ID. After that change, the email question on the guarded run made no tool call at all, taking 3.5 seconds against 15.4 for the broken two-turn version. The reply apologised, said it couldn't handle account changes and pointed to customer service. That's still generic, but with no update_email tool there shouldn't be a magic self-serve path anyway. At least it stopped pretending to look up an order.

The happy-path lookups kept working and ORD-9999 was still reported as missing. The model didn't get any smarter. Think of this as a seatbelt rather than a transplant.

What this doesn't prove

This is a demo, and a real support copilot needs far more than one function and a dictionary of three orders. Several claims should be resisted:

• It isn't multi-tool planning. One tool is a lookup, and around two tools is where 3B models start inventing order IDs to satisfy schemas they don't fully understand.
• The database is fake, so authentication, personal data and "may this user see this order" were never the model's problem.
• Sampling was enabled throughout, so rerunning the email query won't return the same words. What did repeat run after run was the failure itself, a tool call filled with a placeholder. The precise phrasing varies, so don't count on it.
• Nothing was compared with a cloud model or run against a real ticket export, so the numbers describe one machine and not a benchmark.

That leaves a narrower claim than "small models can do support": a 3B model can call one well-described tool, summarise the result and get out of the way, provided the tool call itself is treated as something needing a permission check and not something to trust because the model produced it.

Running it

From the project directory with its virtual environment active, run the three scripts in order: the think versus no-think comparison, the plain agent and the guarded agent.

python llm-learning/smollm3-local-support-agent/code/think_vs_nothink.py
python llm-learning/smollm3-local-support-agent/code/support_agent.py
python llm-learning/smollm3-local-support-agent/code/support_agent_guarded.py

The scripts write transcripts you can read afterwards to compare runs.