ToolCallingAgent vs CodeAgent on Local Qwen2.5 14B Models

Local agent setups expose weak spots fast. The machine here was a base Apple M4 Mac with 16GB of unified memory, with no cloud inference and no extra VRAM to fall back on, so even a moderate context window can trigger swapping. Two agent styles from smolagents, ToolCallingAgent and CodeAgent, were each run against two models served locally: qwen2.5-coder:14b and qwen2.5:14b-instruct. The two share a 14B size and a base architecture, and both ran on the same box.

• Qwen2.5:14b-instruct is the general-purpose model in the 14B lineup, tuned on broad instruction and dialogue data so it follows commands well.
• qwen2.5-coder:14b uses the same backbone but received extra pretraining on a large body of code and fine-tuning aimed at it.

Given such a tight hardware budget, neither setup was expected to shine. The more interesting question was where each one breaks, and whether picking a different kind of agent could soften those limits.

The setup

A temperature sweep at 0, 0.1, 0.3 and 0.6 ran across both agent types on a single task: find the five best pizza places in Seattle, check the wait at each and recommend only those under the customer's threshold. The functions that compute wait times and compare them to the threshold are deterministic.

@tool
def get_wait_time(restaurant_name: str) -> int:
    """Current wait in minutes for a restaurant. Deterministic, not a live lookup."""

@tool
def restaurants_in_wait_limit(wait_time: int, wait_threshold: int) -> bool:
    """True if the wait is within the customer's threshold."""

The deterministic design is deliberate. A live lookup would let a model guess a plausible wait and keep going. A forced function call checks what matters here: whether the agent defers to the tool rather than inventing a value, and whether that still holds when the temperature or the agent type changes.

Three things were tracked: latency, whether a run completed and routing accuracy, defined as the share of the whole tool chain executed correctly over every call. Treat the numbers as a probe, since this is no formal benchmark.

How the two agents differ

With ToolCallingAgent, each step produces a single JSON tool call. The model supplies the arguments, the framework executes the tool, and then the next step begins. Five restaurants and two functions add up to ten separate calls, and each one is a fresh chance to botch the structured output.

CodeAgent takes another route and produces a snippet of Python that runs in one step. One loop can cover every restaurant and call both functions inside it. A correct loop is correct for all of them in that run, so the decision happens once rather than ten times.

Results

On Qwen2.5:14b-instruct

Both agents did well. CodeAgent held 1.00 routing accuracy at every temperature, and ToolCallingAgent stayed between 0.95 and 1.00. Correctness showed no real gap.

Speed told a different story. ToolCallingAgent ran about twice as slowly, roughly five minutes per run against roughly two and a half for CodeAgent. The cause is structural. ToolCallingAgent pays for a new model call at every step, resending the conversation context and generating from scratch each time. CodeAgent pays once. On a local machine where each generation is already slow, that overhead can't be brushed aside.

On Qwen2.5-coder:14b

Here the two agents diverged sharply. ToolCallingAgent never reached 50 percent routing accuracy at any temperature and hovered between 0.40 and 0.50. It ran faster than it had on the instruct model, but that wasn't a good sign. It was quick because it gave up. Once the thread was lost, it stopped emitting calls and finished sooner having done less.

CodeAgent won outright with a routing accuracy of 1.00, matching the best instruct numbers. That's expected, since it asks the coder model for Python and never for structured JSON.

What to take from it

CodeAgent's loop advantage is real. On the instruct model, where both agents scored 1.00 on routing, batching the work made CodeAgent twice as fast as ToolCallingAgent.

Qwen2.5:14b-instruct comes across as a dependable generalist, comfortable both with structured JSON calls and with simple code. Qwen2.5-coder:14b is better kept for demanding coding tasks, since it can be unreliable at producing structured JSON and at keeping that up across a chain of calls.

An open question

A follow-up question remains. Two things separate the agents: CodeAgent batches its work into one program, and it writes Python rather than structured JSON. The next experiment could pull those apart by making CodeAgent emit structured tool calls while still executing in a batch. The result would indicate if the advantage comes mainly from looping, or from models finding code easier to produce than a long series of JSON calls.