Agent or Workflow? Splitting Reasoning from Execution in Enterprise AI

Agentic AI is the phrase of the moment. Hand an LLM some tools, add memory, wire it into enterprise systems, let it decide what to do next, and the result gets called an AI agent. There is a catch, though. Enterprise systems cannot run purely on probabilistic decisions. When an AI touches supplier records, financial processes, ERP data, approvals or other critical operations, some choices can stay flexible and others cannot.

A supplier-management project built around RAG, LangGraph, MCP, human approval and Amazon Bedrock led to a single guiding idea: the agent should decide what needs to happen, and the application should control how it happens. Everything else in the design follows from that split.

Why the agent should not do everything

Picture a supplier onboarding request: create this supplier, validate its details, check procurement policy and send it to SAP. A fully autonomous agent could, in theory, interpret the request, search internal documents, validate the data, decide whether approval is needed, connect to SAP, create the record and retry on failure.

That is technically possible and architecturally unappealing. An LLM is probabilistic. The same input can follow slightly different reasoning paths, and models, prompts and context all drift. That flexibility is wonderful for interpreting what a person means. It is a poor fit for executing critical business transactions. A model should not decide whether a database transaction commits, invent its own retry strategy, judge whether authorization can be skipped or hold SAP credentials. Reasoning and execution therefore get separated.

Workflows versus agents

The two terms are often swapped, but the difference is useful. An AI workflow follows a mostly predefined path:

Request
   ↓
Validate
   ↓
Retrieve Policies
   ↓
Evaluate
   ↓
Approval
   ↓
Create Supplier
   ↓
Publish Integration Event

The application knows every state and every transition. An AI agent has more freedom to pick what comes next:

User Request
      ↓
     LLM
      ↓
Reason about intent
      ↓
Choose next capability
   ↙    ↓     ↘
Search Validate Request Approval

That freedom has value, and it turns risky when the agent reaches transactional infrastructure. The preferred design combines both models.

Agentic reasoning on top, deterministic execution below

At a high level the architecture looks like this:

USER
                      │
                      ▼
                 FastAPI API
                      │
                      ▼
                 LangGraph
                Agent Workflow
                      │
          ┌───────────┼───────────┐
          ▼           ▼           ▼
        RAG        Reasoning     Tools
          │
          ▼
   Amazon OpenSearch
          │
          ▼
     AWS Bedrock
                      │
                      ▼
                MCP Boundary
                      │
                      ▼
               Application API
                      │
                      ▼
              Domain / Use Cases
                      │
              ┌───────┴───────┐
              ▼               ▼
          PostgreSQL         AWS SQS
                                │
                                ▼
                               SAP

One boundary stands out. The LLM sits above the application layer. It can interpret requests, reason, retrieve knowledge, recommend actions and ask for tools. It does not own the transaction.

Where agents earn their place

Probabilistic reasoning pays off in a few specific spots.

Understanding intent

People do not write perfectly structured API calls. A user might say they need to onboard ACME for a Brazilian operation and want to know whether everything is compliant. That sentence mixes intent, context and ambiguity, and an LLM is very good at turning it into something the application can consume:

{
  "intent": "supplier_onboarding",
  "supplier": "ACME",
  "country": "BR",
  "requires_policy_validation": true
}

From there, deterministic software takes over.

Retrieving enterprise knowledge with RAG

Onboarding depends on internal policy: required documents, tax rules, country-specific requirements, procurement restrictions and approval thresholds. Rather than stuffing those documents into prompts or hardcoding each rule, the system can retrieve them:

Question
   ↓
Embedding
   ↓
Amazon OpenSearch
   ↓
Relevant Policy Chunks
   ↓
LLM Context
   ↓
Reasoning

A second boundary applies here. Retrieval supplies context and must not own business decisions. Finding a policy that says suppliers above a threshold need approval is helpful. Enforcing that approval has to remain ordinary application logic.

Where workflows do better

Once the request is understood, suppose a supplier must be created. The process may involve steps like these:

Validate Supplier
      ↓
Check Duplicate
      ↓
Check Authorization
      ↓
Persist Supplier
      ↓
Publish Integration Event
      ↓
Wait for SAP Result

An LLM should not improvise this. The workflow should define the execution guarantees: validation, authorization, idempotency, duplicate prevention, retries, audit logs, transaction boundaries and error handling. Those are software engineering concerns, and prompt engineering has nothing to say about them.

Why LangGraph

For orchestration, LangGraph was chosen because it offers explicit state transitions, in place of an open loop where an agent keeps deciding which tool to call next. Conceptually the graph looks like this:

START
  │
  ▼
Understand Intent
  │
  ▼
Retrieve Policies
  │
  ▼
Evaluate Request
  │
  ├───────────────┐
  │               │
  ▼               ▼
Approval Needed?  No Approval
  │               │
  ▼               │
Human Approval    │
  │               │
  └───────┬───────┘
          ▼
     Request Action
          │
          ▼
         END

Inside chosen nodes the model keeps its freedom to reason, while the graph caps how far its conclusions can steer the system. The result is bounded autonomy: freedom inside clearly drawn architectural lines.

Human-in-the-loop has more than one meaning

Enterprise AI talk often reduces to "just add a human in the loop." Yet people can step in for different reasons, and at least two kinds of approval are worth separating.

Business approval

The business process itself demands sign-off, whether or not AI is involved:

Supplier
   ↓
Compliance Check
   ↓
Manager Approval
   ↓
Creation

AI tool approval

A second approval arises because the agent wants to perform a sensitive action:

Agent
  ↓
"Create supplier"
  ↓
Tool requires confirmation
  ↓
Human approves
  ↓
Application executes

These two concerns differ. One belongs to the domain and the other to AI governance. Blending them causes architectural confusion.

Why MCP matters

An agent that works with enterprise systems needs tools, but exposing infrastructure straight to the LLM adds needless risk. The better route runs through an MCP tool:

Agent
  ↓
MCP Tool
  ↓
Application API
  ↓
Use Case
  ↓
Domain
  ↓
Infrastructure

and avoids connecting the agent to a database:

Agent
  ↓
PostgreSQL

or directly to SAP:

Agent
  ↓
SAP

With the first shape, the application still controls authentication, authorization, validation, business rules, auditing, transactions and idempotency. MCP becomes a boundary between AI reasoning and enterprise capabilities. The agent expresses intent and the application owns execution.

SAP should not depend on the LLM staying up

External systems fail. SAP can be down, the network can drop, a downstream service can time out, and the LLM provider can go offline too. Running everything synchronously inside an agent loop makes resilience hard. That is the reason for asynchronous messaging with AWS SQS. Instead of this:

Agent
  ↓
SAP
  ↓
Wait...
  ↓
Timeout

the flow becomes:

Agent
  ↓
Application
  ↓
Persist State
  ↓
AWS SQS
  ↓
Integration Worker
  ↓
SAP

The business transaction no longer waits on SAP to answer immediately. A worker handles retries, backoff, dead-letter queues, observability and idempotent processing, and those guarantees come from deterministic software and not from a model.

A hybrid architecture

After these boundaries are in place, calling the system just an AI agent no longer fits. It is a hybrid:

┌─────────────────────────────┐
│     PROBABILISTIC LAYER     │
│                             │
│ LLM                         │
│ RAG                         │
│ Intent Understanding        │
│ Reasoning                   │
│ Tool Selection              │
└──────────────┬──────────────┘
               │
        Controlled Boundary
               │
┌──────────────▼──────────────┐
│     DETERMINISTIC LAYER     │
│                             │
│ Application Use Cases       │
│ Domain Rules                │
│ Authorization               │
│ Validation                  │
│ Transactions                │
│ Idempotency                 │
│ Messaging                   │
│ Integrations                │
└─────────────────────────────┘

That distinction is essential for enterprise agentic AI.

Software architecture still matters

A common mistake with generative AI is to assume traditional architecture stopped mattering. It matters more. A system that can reason dynamically and choose actions needs stronger limits on what those actions can do. DDD, CQRS, hexagonal architecture, authentication, authorization, messaging, observability, resilience and idempotency all still apply. AI removes none of them. It adds one more probabilistic component that must run safely beside them.

Agent or workflow?

The answer is both. Use agents where uncertainty and reasoning add value, and use deterministic workflows where predictability and guarantees count. A handy mental model:

AI AGENT
                 │
                 │ decides WHAT
                 ▼
        ┌─────────────────┐
        │ Controlled Tool │
        │    Boundary     │
        └────────┬────────┘
                 │
                 │ requests
                 ▼
          APPLICATION
                 │
                 │ decides HOW
                 ▼
             DOMAIN
                 │
                 ▼
        INFRASTRUCTURE

The LLM should not become the architecture. It should take part in it.

Where autonomy pays off

The useful question is not how much autonomy a model can be given. It is where autonomy creates value. Autonomy fits interpreting a vague request, searching a large pile of internal documents, reasoning over context and proposing the next action. It does not fit overriding business rules, managing transactions or carrying production database credentials.

The aim is controlled autonomy, which in enterprise systems means combining probabilistic AI reasoning with deterministic engineering. The next question follows naturally: when an agent has to reach SAP or a database, what should that boundary actually look like?