Evals for GenAI and Agents: Single-Turn vs Multi-Turn Testing in CI

"It works on my prompt" isn't an engineering standard. A conventional unit test passes or fails deterministically, but generative systems give open-ended, probabilistic answers. A RAG pipeline can be mostly right, and an agent can finish a task in a way that counts as a success on paper and is unsafe in practice. That's why evals, as opposed to gut feeling or a chat channel full of "looks fine to me", now anchor serious ML engineering for Gen AI and agentic systems. What follows covers what to measure, how single-turn and multi-turn evals differ, the trade-offs of each, and Python patterns that fit into a CI pipeline.

Why evals aren't optional

Three traits of LLM applications make them necessary:

1. Non-determinism. Identical prompts can yield different results across runs, model versions or temperature values.
2. Compounding failure in agents. Where a plain chatbot has a single shot at being wrong, an agent strings together anywhere from 10 to 50 model calls, tool calls and decisions. A modest error rate per step snowballs into frequent end-to-end failure.
3. Silent regressions. A prompt tweak, a model upgrade or a vector store re-index can degrade quality without any exception being raised.

An eval turns "it feels better" into a number that can be tracked over time, used to gate deployments and compared between versions.

Two shapes of eval

Nearly every framework, among them DeepEval, Ragas, LangSmith, Braintrust and OpenAI Evals, ends up with one of two patterns.

Single-turn, trace-based

The loop is simple: take each golden, call the application once, score the trace.

Open a test run with trace-level metrics
  for each golden:
    yield golden
    call app with golden.input
    capture trace
    build a test case from the trace
    score test case with metrics
  upload test run (traces + scores)

A golden is a labeled sample: the input, the answer it should produce, and frequently the context or tool calls it should involve. The trace records the full story of one run, including which chunks were retrieved, which tools fired, the reasoning in between, latency and token counts, and not merely the closing text. Scoring that record instead of just the answer pinpoints the failing stage. Retrieval, generation or a tool call midway?

Multi-turn, simulator-based

This is the harder and more realistic pattern for agents and chat assistants.

def simulate(conversational goldens)
  for each golden:
    loop until expected outcome OR max turns:
      simulator model generates a user turn
      ask app for next reply
      capture assistant turn
    build a ConversationalTestCase from the full transcript
  evaluate(test cases, metrics)   # metrics run concurrently
    -> pass/fail + reason per metric

There's no fixed input and output pair here. A simulator model plays a user with a goal, such as canceling a subscription or getting a refund for a particular order, and adapts each message to the assistant's replies, much like a person would. The system is judged on the whole trajectory. Did it stay on task? Did it use tools properly across turns? Did it resolve the goal in the end? Did it invent policy details three turns in?

Agents, multi-step assistants and anything with memory or state need this style. A single-turn eval can't detect an agent contradicting itself on turn 4 or calling one tool five times in a loop.

Choosing metrics

Metrics fall into four groups, and mixing them is wise because none tells the full story.

A handy mental model: single-turn metrics score a snapshot, while agentic metrics score a trajectory.

LLM as a judge

Most reference-free metrics today work through "LLM-as-a-judge". A strong model, often a different and more capable one than the model under test, scores an output against a rubric and explains its score.

from dataclasses import dataclass

JUDGE_PROMPT = """
You are evaluating an AI assistant's response for FAITHFULNESS.
A response is faithful if every factual claim it makes is supported
by the provided context. It is unfaithful if it adds unsupported
claims, contradicts the context, or fabricates details.
Context:
{context}
Response:
{response}
Return JSON: {{"score": <0.0-1.0>, "reason": "<short explanation>"}}
"""

@dataclass
class TestCase:
    input: str
    actual_output: str
    retrieval_context: list[str]

def faithfulness_metric(test_case: TestCase, judge_llm) -> dict:
    prompt = JUDGE_PROMPT.format(
        context="\n".join(test_case.retrieval_context),
        response=test_case.actual_output,
    )
    result = judge_llm.generate(prompt, response_format="json")
    return {"score": result["score"], "reason": result["reason"], "passed": result["score"] >= 0.7}

Judges have pitfalls:

• Self-preference bias. Models rate their own output, or output in their own style, more generously. Use a judge from another model family where you can.
• Prompt sensitivity. Small changes in rubric wording move the scores. Keep judge prompts under version control like code.
• Cost and latency. Judge calls cost money too. Where budgets are tight, sample production traffic and don't judge all of it.
• Position and verbosity bias. Judges lean toward longer, more confident answers whether or not they're right. Calibrate now and then against a small human-labeled set.

Single-turn in practice

A golden set can be as small as a list of dictionaries holding an input and an expected output.

goldens = [
    {
        "input": "What's our refund policy for digital products?",
        "expected_output": "Digital products are refundable within 14 days if unused.",
        "expected_tools": [],
    },
    {
        "input": "Cancel order #48213",
        "expected_output": None,  # open-ended, judged not matched
        "expected_tools": ["lookup_order", "cancel_order"],
    },
]

The runner follows the loop described above: open a test run, go through each golden, call the app, capture the trace, score and upload.

from deepeval import evaluate
from deepeval.metrics import FaithfulnessMetric, AnswerRelevancyMetric
from deepeval.test_case import LLMTestCase

def run_eval_suite(goldens, app_callback, metrics):
    test_cases = []
    for golden in goldens:
        trace = app_callback(golden["input"])   # your traced LLM/RAG app
        test_case = LLMTestCase(
            input=golden["input"],
            actual_output=trace.final_output,
            retrieval_context=trace.retrieved_chunks,
            expected_output=golden.get("expected_output"),
        )
        test_cases.append(test_case)
    results = evaluate(test_cases, metrics)  # runs metrics concurrently
    return results
metrics = [FaithfulnessMetric(threshold=0.7), AnswerRelevancyMetric(threshold=0.7)]
results = run_eval_suite(goldens, my_rag_app, metrics)

An assertion turns the scores into a gate.

def assert_eval_gate(results, min_pass_rate=0.9):
    passed = sum(1 for r in results if r.success)
    pass_rate = passed / len(results)
    assert pass_rate >= min_pass_rate, f"Eval pass rate {pass_rate:.2%}
    below gate {min_pass_rate:.0%}"

Hook it into the PR pipeline the way pytest is hooked in. If a prompt or retrieval change pushes faithfulness under the threshold, the merge should be blocked and nothing should ship quietly.

Multi-turn in practice

Rather than a fixed input, the simulator gets a user profile and a goal, and it steers the conversation.

class ConversationSimulator:
    def __init__(self, simulator_model, max_turns=8):
        self.simulator_model = simulator_model
        self.max_turns = max_turns

    def simulate(self, golden, chatbot_callback):
        transcript = []
        user_turn = self.simulator_model.generate_opening(golden["scenario"])
        for _ in range(self.max_turns):
            transcript.append({"role": "user", "content": user_turn})
            assistant_turn = chatbot_callback(transcript)
            transcript.append({"role": "assistant", "content": assistant_turn})
            if self.simulator_model.is_goal_met(golden["expected_outcome"], transcript):
                break
            user_turn = self.simulator_model.generate_next_turn(transcript, golden)
        return transcript

The resulting conversation is then scored with agentic metrics.

from deepeval.test_case import ConversationalTestCase, Turn
from deepeval.metrics import (
    TaskCompletionMetric,
    ToolCorrectnessMetric,
    ConversationCompletenessMetric,
)

def build_conversational_test_case(transcript, tools_called):
    turns = [Turn(role=t["role"], content=t["content"]) for t in transcript]
    return ConversationalTestCase(turns=turns, tools_called=tools_called)
    metrics = [
    TaskCompletionMetric(threshold=0.8),          # did it accomplish the user's goal?
    ToolCorrectnessMetric(threshold=1.0),          # right tools, right args, right order
    ConversationCompletenessMetric(threshold=0.7), # did it address everything raised?
    ]
    test_case = build_conversational_test_case(transcript, tools_called=agent_trace.tool_calls)
    result = evaluate([test_case], metrics)

Typical agent-level checks include:

• Tool-call correctness: the right tool, with the right arguments, in the right order, which is more than "some tool was called".
• Step efficiency and loop detection: repeated calls to one tool with no progress.
• Role and policy adherence: staying inside the defined persona and guardrails for the entire conversation, not only the opening turn.
• Error recovery: a sensible retry after a failed tool call, instead of a made-up result.
• Context carry-over: remembering what the user said three turns earlier.

Weighing the trade-offs

Because multi-turn results are noisy, interpret them statistically. Run the suite N times and look at distributions and not at one score.

Practices that hold up

1. Begin with a small, clean golden set of 30 to 100 examples. A noisy set of 1,000 is worse than a tidy 50, since bad labels poison the signal.
2. Evaluate retrieval and generation separately in RAG. Faithfulness and contextual recall answer different questions, and merging them hides the real culprit.
3. Judge with a stronger, independent model, and spot-check its scores against human raters now and then. Even 20 to 30 samples a month reveal judge drift.
4. Trace everything. Score from captured traces (retrieved documents, tool calls, intermediate steps) and not just the final text. That's why capturing the trace comes before building the test case.
5. Simulate agents before deploying. Run conversations against realistic personas and adversarial goals, such as an angry user, an ambiguous request or a policy edge case, ahead of shipping and not after an incident.
6. Treat evals as living assets. Rerun the full suite after every prompt change, model upgrade or index rebuild, and put pass-rate thresholds in CI like coverage thresholds.
7. Watch distributions over time. One eval run is a sample, not ground truth. Repeat critical suites and follow the trend.

The takeaway

Evals for generative AI started out as a footnote, a notebook cell someone ran ahead of a demo. Agentic systems have outgrown that. Once software can act several times, call tools and hold state from turn to turn, evaluation is no longer QA. It's the primary lens on how the system actually behaves.

Trace-based single-turn evals supply fast, inexpensive regression checks on output quality. Simulator-driven multi-turn evals supply the one trustworthy reading on whether an agent behaves as intended across an entire interaction, and for most production agents that reading matters most. Build both and automate both. Trust neither blindly, and keep a human checking the judge.