CrewAI, AutoGen, LangGraph, OpenAI Agents SDK and smolagents Compared
An engineer's comparison of five open-source agent frameworks across orchestration, memory, tools, evals and observability, plus recurring failure patterns.
This article is published in English.
Anyone who builds LLM applications for long enough hits the same wall: one prompt and one tool-call loop doesn't cut it. Real systems need several steps, branching, shared memory, retries, human checkpoints, and some way to work out why an agent misbehaved at 3 a.m. Agent frameworks exist to cover that ground.
As of early 2026, the open-source scene offers a few serious contenders: CrewAI, Microsoft's AutoGen, LangChain's LangGraph, OpenAI's Agents SDK and smolagents from Hugging Face. Their marketing sounds similar, yet architecture, ergonomics and production behavior diverge considerably. This is a working engineer's comparison, not a benchmark, since reliable agent benchmarks are still hard to produce. All five are actively maintained, documented in public and found in real production stacks. They're judged on five production concerns: orchestration model, memory and state, tool integration, evaluation, and operations plus observability. Latency numbers are deliberately absent, because they swing with provider, prompt size and tool fan-out. Measure them on your own workload.
The five frameworks
CrewAI
CrewAI frames things as a crew of agents, each with a role, a goal and a backstory. Agents get tasks and tools, and the framework manages delegation, handoffs and either a sequential or hierarchical process.
It's the easiest to read aloud. A Crew made of a researcher, an analyst and a writer is something a product manager can follow immediately. The price is a leaky abstraction: when delegation goes wrong, the thing to debug is a conversation among agents, not a deterministic graph. Over the past year it gained flows and memory features, and the crewai run CLI plus telemetry hooks make it friendlier to production than it used to be. Still, the LLM does the orchestration reasoning, so token use and non-determinism run higher than with graph-based designs.
It suits content pipelines, research workflows and any case where several personas should mirror real organizational roles.
AutoGen
AutoGen was substantially rewritten in its v0.4 line, shifting to an event-driven actor model with a real runtime and a message broker. Many people still picture the older v0.2 GroupChat style, but the current direction resembles a distributed actor system, where agents trade messages and different runtimes can be swapped in.
The result is the most systems-minded framework of the group, offering detailed control over message routing, termination conditions and agent lifecycle. Complexity is the price. Even a plain two-agent chat needs more boilerplate than CrewAI, and "events, topics and runtime" asks more of the reader than a crew does. It also has a strong record with multi-agent research patterns like debate, reflection and society-of-mind, since Microsoft Research designed it with those in mind.
It fits research-heavy multi-agent setups, custom message routing and teams at ease with actor-style thinking.
LangGraph
LangGraph takes the firmest stance on control flow, modeling every workflow as a state graph. Nodes can be plain functions or agent calls, edges can be fixed or conditional, and a typed state schema travels through it all. Checkpointing is a first-class feature, which makes persisting state, replaying runs and adding human-in-the-loop interrupts clean.
It's the pick when correctness and auditability outrank demo speed. The graph is explicit, which means it can be tested, drawn and reasoned about. The drawbacks: it inherits LangChain's reputation for layers of abstraction, deeply nested runnables remain painful to debug even with LangSmith tracing, and the learning curve is steeper than CrewAI's define-agents-and-tasks approach.
It's best for production workflows needing deterministic flow, approval gates, replay and audit trails, and for anyone already inside the LangChain ecosystem.
OpenAI Agents SDK
This open-source Python package formalizes patterns from OpenAI's earlier Swarm experiment. The model is light: agents carry instructions, tools and handoffs, guardrails wrap inputs and outputs, and tracing is built in.
It feels the least like a framework. No graph engine and no actor runtime exist, only agents, tools, handoffs and a runner loop. That's an advantage for people who want to see exactly what's happening and a limitation for anyone needing complex state management. Defaults lean toward OpenAI models, via the Responses API and a specific tracing format, though other providers work through model adapters. The guardrails primitive is genuinely useful, and elsewhere it tends to show up later as an add-on and not a core piece.
It suits OpenAI-centered stacks, teams wanting a thin layer over the model loop, and designs where handoffs between specialist agents are the main pattern.
smolagents
Hugging Face's smolagents is the odd one out. In place of the usual ReAct loop of calling a tool, receiving JSON and parsing it, it uses code agents: the LLM writes and runs Python to use tools. The framework is deliberately tiny, with a very small codebase, and offers both a CodeAgent and a more conventional ToolCallingAgent.
Code agents bring real benefits. There are no JSON parsing failures, tool calls compose naturally and the model can write loops and conditionals directly. The evident risk is running LLM-generated code, which smolagents softens through sandboxing options such as E2B and local interpreters. Of the five, it's the likeliest to surprise, for better and worse.
A good match for research and data jobs where code is the natural way to act, for lightweight deployments, and for teams wanting a compact, auditable codebase.
Implicit versus explicit orchestration
The widest architectural divide separates implicit orchestration (CrewAI, AutoGen's GroupChat) from explicit orchestration (LangGraph, handoffs in OpenAI's SDK, the smolagents runner).
With implicit orchestration the LLM picks who acts next. That's flexible and looks great in a demo, but in production it injects non-determinism: identical input can produce different agent sequences, which destabilizes evals, complicates debugging and spends extra tokens on coordination chatter.
Explicit orchestration through graphs, handoffs or code keeps control flow inspectable. It can be drawn, tested and fitted with approval gates in the right spots. The cost is design effort up front, since the graph has to be thought through before shipping.
A reasonable rule: for "research and produce a deliverable" workflows, implicit orchestration is acceptable. When a workflow touches money, customer data or irreversible actions, choose explicit orchestration and keep a human in the loop, with no exceptions.
Memory and state
Memory is still a weak spot in most of them. CrewAI has short-term and long-term primitives with swappable backends. AutoGen's actor model keeps message history and state per agent, though lasting memory across sessions is largely left to the developer. LangGraph's checkpointing is the strongest, with serializable state, replay and time-travel debugging included. The OpenAI Agents SDK stays simple, holding session state in a runner context while long-term memory is on you. smolagents keeps it minimal on purpose. Beyond a single session, expect to supply your own vector store or database, because built-in memory is a starting point, not a finished feature.
Tool integration
All five support function-calling tools, and they differ in convenience:
- CrewAI uses a decorator,
@tool, which is tidy for simple cases. - AutoGen treats tools as typed callables, though the actor model adds some indirection.
- LangGraph binds tools to nodes, letting you compose LangChain tools or write plain functions.
- The OpenAI Agents SDK has a
@function_tooldecorator that's straightforward and infers schemas well. - smolagents tools are Python objects that code agents call directly, skipping JSON serialization.
For standard REST or Python calls, any of them works. With dozens of tools and complex schemas, LangGraph plus LangChain's tool ecosystem offers the most plumbing, with the upsides and downsides that implies.
Evaluation
Evaluation is the most disappointing area across the board, because none of the frameworks ships a strong story. The options look like this:
- LangSmith integrates tightly with LangGraph and is the most mature tracing-plus-eval pairing, though it's commercial with a free tier.
- Tracing in the OpenAI Agents SDK offers spans and a dashboard, useful for inspection but short of a full eval harness.
- CrewAI includes training and telemetry hooks, but leans on running it many times and watching, not on structured evals.
- AutoGen and smolagents mostly leave evals to the developer.
Serious projects mean building a custom harness atop promptfoo, DeepEval or an LLM-as-judge setup of your own, and running it in CI. Scoring agents is tougher than scoring single turns, because both the final output and the steps along the way need grades. Set aside real time for this.
Operations and observability
Production agents need tracing, retries, cost tracking and guardrails. In brief:
- LangGraph with LangSmith has the most complete observability, including step-level traces and replay.
- The OpenAI Agents SDK counts tracing and guardrails as first-class primitives, a clear plus.
- CrewAI offers telemetry and integrates with tools like AgentOps and Langfuse.
- AutoGen v0.4's event-driven runtime emits structured events that can go to any backend.
- smolagents is minimal, so logging is yours to wire up.
On cost, none of these frameworks prevents overspending. Agent loops can burn plenty of tokens on retries and inter-agent chatter, so set maximum iterations, a token limit per step and a cost ceiling per run, and alert on each. A misconfigured crew can spend $40 on a single run when one agent keeps asking another to clarify.
Failure patterns that keep recurring
- Token explosions from inter-agent chatter, worst in frameworks with implicit orchestration. Cap iterations and turns.
- Silent tool failures, where an agent receives a 500 error and reasons around it instead of surfacing it. Make tool errors loud and retryable.
- Runs that can't be reproduced. Without checkpointing, as in LangGraph, or fixed seeds, the same input yields different traces, and debugging gets very hard.
- Guardrail gaps. Teams typically add output validation late, and the guardrails primitive in the OpenAI Agents SDK is a good pattern to copy even without using the SDK.
- Stale tool results. When a tool returns market data or any time-sensitive value, the timestamp belongs to the result, and the tool wrapper should refuse stale data. An agent shouldn't act on a quote that's 20 minutes old. If quotes are missing or stale, remove that instrument or model from the pipeline instead of letting the agent guess.
Choosing
No single winner emerges, but a rough guide helps:
- Already on LangChain, needing audit trails or human-in-the-loop: LangGraph.
- An OpenAI shop wanting minimal abstraction: OpenAI Agents SDK.
- Multi-agent research patterns in an actor style: AutoGen.
- Readable role-based crews and content workflows: CrewAI.
- Small footprint, code as the action, data tasks: smolagents.
In practice, production agent systems usually boil down to a slim framework layer over plenty of hand-written glue. No framework supplies retries, evals, cost caps, guardrails or observability for free. Pick the framework whose orchestration model fits the problem, then put the effort into the unglamorous infrastructure around it, which is where production agent engineering truly happens.