Spring AI in the Enterprise: A Support Agent with Tools, RAG and Review

A support ticket as the test case

An internal AI engineering workshop for software engineers at a financial-technology company started from a scenario nearly every developer meets sooner or later. A support ticket lands. The logs hold only partial evidence. The relevant code sits in an internal repository. Whoever is on call needs a focused first analysis and not a generic answer.

Spring AI suits an exercise like this because it brings AI features into an ecosystem Java and Spring developers already work in daily. It offers abstractions for models, prompts, structured outputs, tool calling and retrieval, and they fit into existing application architecture without much friction.

Calling an LLM from a Java application isn't the interesting part. What matters is doing it while keeping the discipline a fintech setting expects: explicit contracts, tools with least privilege, traceability, and human review wherever a decision carries weight.

So the scope here extends past the workshop. A "Support Agent" project serves as the running example for how to connect AI to existing systems, where a model belongs in the architecture, how it reaches enterprise capabilities and data, and how the surrounding system keeps its behavior useful, predictable and under control.

An integration problem, not a model problem

Fitting AI into enterprise systems is mostly integration work and has little to do with picking a model. Teams usually find this out after the first prototype. Sending a prompt to a model is easy, and the results can look impressive. Once AI enters a production system, however, basic tensions surface.

Enterprise software is built around determinism. It depends on strict contracts, predictable latency and well-defined data flow. A given input should produce a consistent output, and behavior can be traced through code.

AI systems work differently. They're probabilistic by nature, often take natural language in place of structured input, and rely on outside providers. The same request can produce different answers, and part of the logic lives inside the model, beyond the system's control.

The result is a structural mismatch: a non-deterministic component sitting inside a deterministic architecture. Familiar assumptions about testing, monitoring and reliability stop holding. The hard part isn't generating clever responses. It's fitting them into systems that insist on control.

Spring AI responds by treating AI as one more integration concern in the Spring ecosystem and not as a special case. Spring already abstracts databases, messaging and REST calls, and Spring AI adds comparable abstractions for language models, vector stores and AI-driven workflows. The aim isn't to hide complexity entirely but to organize it so it matches existing architectural patterns.

In a typical setup, the AI doesn't live in a separate service. It's embedded as integration code within the application layer.

The project's logic runs as follows. A request arrives and business requests are turned away. Real incidents get a ticket. Spring AI tool calling inspects the logs, and the diagnosis is converted to a Java record. Relevant source code is pulled from a Qdrant-backed vector store, a fix suggestion is generated, and the ticket is updated so a person can review it.

For a developer, AI becomes another dependency, and the application keeps ownership of the workflow. Models get configured like any other piece of infrastructure, and interactions go through well-defined APIs instead of ad-hoc HTTP calls.

In the project, a SupportAgentController receives the inbound request as a SupportTicketRequest. It hands the AI reasoning to SupportAnalysisService, saves state through SupportTicketService, fetches source code via GithubRagService, and stores the generated resolution on the ticket.

A notable side effect is that business logic stays independent of the model provider. Changing providers or adding fallbacks leaves the core code alone. Over the long run this is less about flexibility than about holding on to architectural control. If the model is unreachable, the endpoint below should still create a ticket, mark the AI analysis as unavailable and send the case to a human queue.

@PostMapping("/requests")
public ResponseEntity<?> submitSupportRequest(@Valid @RequestBody SupportTicketRequest request) {
    boolean isBusiness = supportAnalysisService.isItBusinessRequest(request);

    if (isBusiness) {
        return ResponseEntity.status(HttpStatus.BAD_REQUEST)
                .body(ErrorResponse.businessRequest());
    }

    SupportTicket ticket = ticketService.createTicket(request);
    LogCauseReport logCauseReport = supportAnalysisService.analyzeLogs(request);

    String classCode = ragService.findRelevantClass(null, logCauseReport.repoName(), null, logCauseReport.className()).stream()
            .map(SupportAgentController::stripMetadataLines)
            .filter(content -> !content.isBlank())
            .collect(Collectors.joining(System.lineSeparator() + System.lineSeparator()));

    String solution = supportAnalysisService.suggestFix(classCode, logCauseReport.causeOfProblem());

    ticketService.updateTicket(ticket.getId(), SupportStatus.SUBMITTED,
            "Cause of the problem: " + logCauseReport.causeOfProblem() + "\n\nSuggested fix: " + solution);

    return ResponseEntity.status(HttpStatus.CREATED).body(SupportTicketResponse.from(ticket));
}

The ChatClient and prompts as contracts

Spring AI centers on the ChatClient API, a structured way to talk to language models. Instead of building HTTP requests by hand, developers use a fluent API that wraps prompt construction, execution and response handling.

The first attempt at a ticket classifier in the Support Agent project looked like this.

String result = chatClient.prompt("Analyze if incoming support ticket is about some kind of issue or is it a business request. "
        + "If it's a business request, the text will ask you to create something new or ask for estimation, if it's a support request "
        + "the text will say that something is not working properly, something is down or needs fixing. Answer with true if it's a "
        + "business request and false if it's a support request" + request.description())
        .call()
        .content();

It resembles an ordinary API, except the contract is defined by the prompt and no longer by a JSON schema.

Would it produce accurate results in production? Probably not. It mixes lasting instructions with untrusted user input in one string, doesn't delimit the ticket content, and assumes the model will return something Java can parse as a boolean. A production-oriented version reads more like an application contract, with system-level instructions kept apart from user input.

String result = chatClient.prompt()
        .system("""
                You classify incoming tickets for an L3 support workflow.
                Return exactly one token: true or false.
                Return true when the request asks for new functionality, estimation, planning, or project work.
                Return false when the request reports broken, degraded, unavailable, or incorrect production behavior.
                Do not explain your answer.
                """)
        .user(u -> u.text("""
                Ticket title: {title}
                Ticket source: {source}
                Ticket description:
                ---
                {description}
                ---
                """)
                .param("title", request.title())
                .param("source", request.source())
                .param("description", request.description()))
        .call()
        .content();

System instructions set behavior and limits, while the user message holds the changing input. Prompts end up as part of the design itself, recording assumptions, setting tone and affecting how requests get read.

That brings a new discipline, prompt engineering. A prompt isn't a throwaway string anymore. It's an artifact to be versioned, tested and maintained, and treating it that way is necessary for stable systems. The Spring AI reference pages on prompts and on the ChatClient are the natural next reading.

Structured output

An early challenge in adding AI to backend systems is the unstructured response. Models produce free-form text naturally, whereas enterprise applications need structured data.

Spring AI closes the gap by mapping responses straight into Java objects. That removes manual parsing and leaves less room for ambiguity in how results are read.

Rather than receiving a paragraph saying something failed, the Support Agent uses a Java record to state which repository and class are probably involved in an incident.

@Service
@RequiredArgsConstructor
public class SupportAnalysisService {

    private final ChatClient chatClient;
    private final LogRetrievalTool logRetrievalTool;

    public LogCauseReport analyzeLogs(SupportTicketRequest request) {

        var converter = new BeanOutputConverter<>(LogCauseReport.class);
        var format = converter.getFormat();

        String response = chatClient.prompt("Analyze the support problem and provide solution, use available MCP tools to access logs. "
                + "Incoming support request description: " + request.description()
                + " \n provide response in this format " + format).tools(logRetrievalTool).call().content();

        LogCauseReport report = converter.convert(response);

        return report;
    }
}

public record LogCauseReport(String causeOfProblem, String repoName, String className) {
}

Here the model is steered to produce a response convertible directly into a strongly typed object, shifting integration from loose text processing to explicit contracts.

The gain is large. Unstructured output forces fragile parsing that breaks when the wording shifts slightly. Structured output can be validated, logged and passed around like any other data type. The official docs describe the converter approach.

Newer Spring AI releases also offer typed entity mapping and, with some providers, native structured output, both worth a look. Enforcing the contract nearer the model means less parsing code for the application to own.

Tool calling

Generating text is useful, but the real value appears when AI can interact with the system by retrieving data, triggering workflows or running business logic.

Tool calling makes that possible by exposing application functions that the model can invoke based on its reasoning. The model shouldn't be treated as an operator with direct infrastructure access. It requests a capability, and the application decides whether that capability exists, how arguments are validated, what data comes back and how the execution is recorded.

Inside the project, SupportAnalysisService exposes a LogRetrievalTool for the log-analysis step. Defining a tool takes an @Tool annotation, and its description parameter helps the model judge when to call the function.

@Tool(name = "analyze_logs", description = "Analyzes provided logs and summarizes potential issues")
public LogCauseReport analyzeLogs(
        @ToolParam(description = "Application name") String applicationName) {
    var converter = new BeanOutputConverter<>(LogCauseReport.class);
    var format = converter.getFormat();

    String prompt = """
            You are a support AI specialized in analyzing application logs to identify potential issues.
            Given the application name "%s", analyze the relevant logs and provide a structured report.
            Respond ONLY with a single JSON object that strictly matches this format:
            %s
            Do NOT return the schema itself, only valid JSON with values filled in.
            """.formatted(applicationName, format);

    String jsonResponse = chatClient.prompt()
            .user(prompt)
            .call()
            .content();

    return converter.convert(jsonResponse);
}

When the model decides it needs more information, it calls the function, folds in the result and carries on generating. In effect, the model becomes an orchestrator of system capabilities.

As systems grow, though, hardcoded integrations become limiting, because each new tool adds coupling between model and application.

Discovering tools with MCP

The Model Context Protocol addresses that. MCP standardizes how tools and services are exposed to models, so they can be discovered dynamically and don't have to be embedded in the application.

MCP turns fixed integrations into a dynamic ecosystem where capabilities can be added or modified with no change to the core application, a basic step toward scalable agent-based architectures. Spring AI's tool-calling documentation describes the programming model and the execution flow. When tools need dynamic discovery or must be shared between processes, its MCP support is the logical follow-up.

Retrieval-augmented generation

Models are powerful but know nothing about your systems' data. Customers, products and internal processes stay unknown unless the information is supplied.

RAG handles this by adding a retrieval step before generation. Relevant documents come out of a vector store and get injected into the prompt, grounding the output in enterprise data such as runbooks, production logs and application code.

For the Support Agent, RAG supplies a code fix for a given production issue, using the Java class named in the application logs.

The first step has the SupportAgentController retrieve source chunks through GithubRagService, which works from the LogCauseReport holding the suspected repository and class.

String classCode = ragService.findRelevantClass(null, logCauseReport.repoName(), null, logCauseReport.className()).stream()
        .map(SupportAgentController::stripMetadataLines)
        .filter(content -> !content.isBlank())
        .collect(Collectors.joining(System.lineSeparator() + System.lineSeparator()));

In the second step, the retrieval service builds metadata filters and runs a similarity search against the Qdrant vector store.

SearchRequest.Builder searchRequestBuilder = SearchRequest.builder();
if (expressions != null) {
    searchRequestBuilder = searchRequestBuilder.filterExpression(expressions.build());
}

Set<String> results = Objects.requireNonNull(vectorStore.similaritySearch(searchRequestBuilder.topK(10).build()))
        .stream()
        .map(Document::getFormattedContent)
        .collect(Collectors.toSet());

The third step injects the retrieved code into a prompt that asks for a fix.

String prompt = """
        You are a Senior Java Developer. You have the following class code:
        %s
        The code has the following problem: %s
        Provide a detailed suggestion on how to fix the problem in the code.
        Format your response in markdown with clear sections.
        """.formatted(classCode, problemDescription);

String suggestion = chatClient.prompt(prompt).call().content();

The effect is to move the model from a general assistant to one that knows the domain. RAG mainly cuts hallucinations and keeps answers in line with internal knowledge, at the price of added complexity.

It adds embedding models, vector databases and retrieval logic, all of which need management and tuning. For enterprise use that complexity is often unavoidable, since answer quality hinges on several factors: how fresh the ingested data is, metadata quality, chunking strategy, authorization and evidence tracking. Spring AI's documentation on RAG, vector stores and Qdrant covers the concepts in this section.

Observability, evaluation and operational controls

Observability for AI works differently. Conventional logging and monitoring presume deterministic behavior, yet AI brings extra dimensions to follow. Tracking response times and error rates falls short, and teams also need to see prompts, model outputs, token usage and tool interactions.

The Support Agent uses Micrometer as its observability framework, capturing:

• identifiers and context for each request, such as ticket, caller and source system
• the model provider and name, prompt version, latency, token usage and finish reason
• whether conversion to structured output succeeded
• each tool call, with normalized arguments, result size and latency
• which repositories, files, vector scores and chunks retrieval returned
• the ticket's final status, and whether the human reviewer accepted or changed the suggestion

That level of detail is vital for debugging and optimization because it explains why the system behaves as it does. In a production application, sensitive prompts and outputs shouldn't be logged verbatim. A sturdier alternative is structured event logging with the request id, model id, token counts and policy outcome.

Output evaluation adds another layer. Systems can assess response quality against predefined criteria and don't have to assume answers are right. That enables patterns like evaluator-optimizer loops, where a response is refined before reaching the user.

It also changes how the system evolves. Whenever a prompt, embedding model, chunking approach or model version changes, the result can be compared against known cases, so the team can tell whether the change improves the workflow and doesn't depend on a few impressive examples.

Before opening the assistant to other teams, a set of operational controls were considered for the project:

• hard timeouts around calls to the model, the tools and the vector store
• token budgets per request and per caller, limiting cost and failure modes
• retries only where they're safe and bounded
• fallback states for when the model, a tool or Qdrant is down
• redaction of secrets, credentials and personal data ahead of prompts and logs
• content logging off by default, or on only under approved access controls
• explicit retention rules for prompts, completions, embeddings and tool output
• dashboards and alerts for conversion failures, empty retrievals, tool errors and reviewer rejections

The reviewer rejection rate says the most about real usage. When engineers regularly rewrite or discard the generated recommendation, the assistant adds toil. A production system ought to learn from that signal. Fine-tuning automatically on everything isn't the way. The better route is operational: improve prompts, tools, retrieval and policies using measured evidence. Spring AI's observability documentation covers the built-in tooling.

Designing bounded agentic workflows

Combine everything above and a new pattern emerges: the agentic system.

Traditional applications follow predefined workflows. Agentic ones run an iterative loop of observing, reasoning and acting. They keep state, use memory and decide dynamically which actions to take.

State, memory and autonomy become first-class concerns, and each maps onto a capability Spring AI already provides:

• state is what's known in the current interaction
• memory is what persists across interactions
• tools define which actions are allowed
• guardrails mark what's forbidden or needs confirmation
• termination criteria say when the loop must end

This shifts the application's role compared with a classic enterprise workflow. Rather than defining every step, the system defines capabilities and constraints and lets the model orchestrate behavior within those limits.

Within the Support Agent, the loop starts with a request. The agent reasons about it, calls tools and fetches code. It then drafts a recommendation and updates the ticket. It's not an autonomous production operator, and it shouldn't turn into one. In financial technology that boundary decides which actions the agent may take in a given context. The model can be trusted to summarize logs well before it's trusted to modify a ticket.

Building effective agents is hard, and Spring AI's documentation on the topic is a sound starting point.

From experiments to enterprise systems

AI isn't just another feature. It adds a new layer to enterprise systems, one that's probabilistic, dynamic and tightly bound to external dependencies. Without structure, that layer quickly leads to fragile, unpredictable behavior.

Spring AI offers a way to integrate it while keeping the engineering practices that make enterprise systems reliable. Familiar abstractions and well-defined integration patterns let teams introduce AI without abandoning explicit contracts, controlled interactions and predictable behavior.

So the change isn't merely about adding AI capabilities. It's about evolving systems so AI becomes a useful, governed and observable part of the architecture. The next wave of enterprise applications will be judged by how well they integrate AI, and how they use it will matter less.