LLM vs AI Agent: Following an Interview Question Through Six Follow-Ups

Ask a candidate for an AI engineer role what separates an LLM from an AI agent, and a typical first answer is that an LLM produces text while an agent takes actions. The interviewer rarely nods and moves on. The follow-ups that come next show whether the candidate has worked with real systems or just memorized definitions. Here's how that conversation tends to unfold, one question at a time.

What an LLM does by itself

A language model produces output from whatever context it's handed. It can explain an error, summarize logs pasted into the prompt or return structured data. It can't reach into a private database just because someone asked. So when the interviewer asks how an AI support assistant could check a live transaction, the answer is tools plus application code.

How a tool gets used

Picture a customer message: "My payment succeeded, but the order was not created." The backend exposes a few narrow functions:

getPaymentStatus(transactionId)
getOrderStatus(transactionId)
createSupportTicket(details)

The application sends their names and parameter schemas along with the conversation, and the model may reply with something like this:

{
  name: "getPaymentStatus",
  arguments: { transactionId: "txn_123" }
}

The interviewer's next question is the trap: did the model just call the payment service? No. It produced a structured request to use a tool. Nothing has run yet.

Who actually makes the call

The backend does. The true flow has six steps:

1. The application sends the prompt together with the tool definitions.
2. The model returns a tool name and arguments.
3. The backend checks the input and the caller's permissions.
4. It runs the approved function.
5. The result goes back to the model as fresh context.
6. The model asks for another tool or gives its final answer.

In short, the LLM proposes, and the application owns execution, credentials and access control. That sentence is worth having ready, because it's the boundary senior interviewers keep probing.

What turns this into an agent

On a whiteboard the equation is short:

AI Agent = LLM + Instructions + Tools + State + Execution Loop

The agent watches its state, picks a step, gets the result and reassesses, which gives the loop:

Observe → Decide → Request Tool → Execute → Observe Result → Continue or Stop

Then comes the question of who stops it. The model can return a final answer, but the application has to enforce limits. The loop should end on a cap on turns, a timeout, an unrecoverable error or a point where human approval is required.

Running several tools at once

Whether calls can run together depends on data dependencies. Payment status and order status need only the transaction ID, so both can be fetched in parallel:

const [payment, order] = await Promise.all([
  getPaymentStatus(transactionId),
  getOrderStatus(transactionId)
]);

Now suppose the payment response includes a gateway reference that the next call requires. The second operation has to wait for the first:

const payment = await getPaymentStatus(transactionId);
const gateway = await getGatewayDetails(payment.gatewayRef);

A strong answer shows the candidate checking whether calls are independent before parallelizing them.

Letting the agent issue a refund

The right response is that a model's request alone isn't enough. The backend must verify the customer, who owns the transaction, whether it's eligible and what the limits are. The write operation needs an idempotency key and an audit log, and large refunds may call for a human to approve. A system prompt that says "refund only for valid transactions" is guidance for the model. It doesn't count as authorization.

How the interviewer scores it

The opening answer, "an LLM generates responses, an agent performs actions," was too thin for a senior role. It never said who executes actions, who holds credentials or what prevents unsafe operations. After follow-ups the candidate got to a strong answer, but to land at senior level from the start, the first reply should mention the application boundary and the agent loop right away.