One Agent, Many Specialists: Why a Personal AI Should Route Work, Not Do It All
A personal AI assistant works better as an orchestrator: it routes tasks to coding agents, models and plain scripts, with graduated permissions underneath.
This article is published in English.
Sources: I Stopped Trying to Build One AI Agent That Does Everything
Qwen, Nemotron, Gemma, DeepSeek. Local inference, AWS Bedrock, Azure. Different runtimes, different sizes, different prices, different degrees of disappointment. A long run of experiments with Hermes, Codex, OpenCode, local models and cloud APIs tends to start from the belief that one correct model exists to be found. Sooner or later that turns out to be the wrong problem. A second assumption hides behind it, and it is the more interesting one: why should the personal agent itself do everything?
The Jarvis Dream
The first architecture is almost embarrassingly simple: a person, then one agent, then everything.
Messages to that agent might read: check on the projects, fix an issue in one of the repositories, list tomorrow's calendar, report whether the Linux server is healthy, research a topic and return the important parts. The agent is meant to handle all of it.
It sounds wonderful, and it is exactly where things begin to go wrong, because everything is an enormous API surface.
Better Tools Already Exist for Some Jobs
Coding is the clearest case. Codex and OpenCode are not just chat windows attached to a model. They work inside a coding environment. They understand repository structure, read and search files, edit several files at once, run commands, inspect errors, run tests, look at diffs and iterate. That is a great deal of specialized machinery.
Now say the personal agent is told to implement a feature in a project. Two approaches are possible.
Approach A. Give the agent a very powerful model, filesystem access and shell access. Teach it repository operations. Build context management, diff handling, command execution, test loops and error recovery. Keep refining the coding workflow. Congratulations: a coding agent is being rebuilt from scratch.
Approach B. The agent recognizes a coding task and hands it to Codex or OpenCode. That is plainly more sensible.
An Assistant Need Not Be the Best at Everything
This is a real mental shift. A personal assistant does not have to be the best programmer, researcher, DevOps engineer, scheduler and knowledge system in the world. It has to know who should do the work. That is a different responsibility, and possibly a more important one.
Human organizations work the same way. A chief executive does not write every backend function, configure every Kubernetes cluster, research every market, build every spreadsheet, book every meeting or answer every support request. Their value comes in part from understanding what has to happen, who should handle it and whether the result is acceptable. A personal AI orchestrator can be thought of in those terms. It does not perform every task. It understands intent well enough to route work correctly.
A Different Shape of System
Replace the chain of person, agent, one giant model and everything with something like this:
┌──────────────┐
│ Me │
└──────┬───────┘
│
▼
┌─────────────────┐
│ Google Chat │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Hermes │
│ Orchestrator │
└────────┬────────┘
│
┌─────────────────┼──────────────────┐
│ │ │
▼ ▼ ▼
Coding Agents AI Models Tools
Codex/OpenCode Local / Cloud APIs / Servers
│ │ │
└─────────────────┼──────────────────┘
│
▼
Final Result
The agent's job becomes much clearer. It understands the person, understands the capabilities on offer, picks the right one, supplies the necessary context, watches the result and reports back. That is the assistant actually worth wanting.
Chat as a single front door
A chat app such as Google Chat gains meaning in this design. It should not expose the complexity underneath. From the user's side there is one conversation. A message like check whether anything important happened with Project X today might make the agent consult the task system, look at GitHub, inspect server state, read relevant material, perhaps ask another model to analyze something, and then summarize. The user should not need to know any of that, or open five applications. The interface is intent, and everything below is implementation.
Coding as the model case
Picture being away from the computer. A phone message says there is a problem in the authentication flow of Project X and asks for a fix if it is straightforward. In principle the chain runs like this: the agent works out which project is meant, gathers enough context to find the repository, recognizes that code must change and delegates to a coding agent. Codex or OpenCode opens the repository, investigates, edits the relevant files, runs tests, checks the diff and returns the result. The orchestrator receives it and reports something like: the refresh-token validation used the wrong expiry condition, the coding agent fixed it, the relevant tests pass, and here is what changed.
Which model produced which token along the way hardly matters. What matters is that the job was done correctly.
Models Turn Into Infrastructure
Once the agent is an orchestration layer, questions such as whether to use Qwen, DeepSeek, a local model, Bedrock or Azure stop excluding one another. The answer can simply be yes, with different intelligence for different work. A small local model handles simple routing. A cheap cloud model writes summaries. A stronger reasoning model takes the hard decisions. A coding agent handles repositories. Another specialist covers some other workload. Models become infrastructure, and infrastructure should be replaceable.
Avoiding dependence on one vendor
Models change at absurd speed. Today's obvious pick can look mediocre six months on. Prices move, providers change, open models improve, context windows grow, tool calling gets better and new inference architectures arrive. If the whole personal setup assumes that Model X does everything, switching hurts. A better shape is:
Task
↓
Router
↓
Best available capability
Today the last step might be one provider and tomorrow another, and the workflow should not care.
The router may matter more than the model
A great deal of time goes into comparing models: A scored 74, B scored 78, C has a million tokens of context, D costs 60 percent less. That information is useful. Now imagine a system that knows A is best for a certain coding-related reasoning task, B is fast and cheap for classification, C is unnecessary because the request can be answered locally, another request needs no LLM at all and just an API call, one request should go straight to Codex, and one needs a question to the user before anything happens.
Here part of the intelligence lives in the system that decides how intelligence is used, which is far more interesting than another leaderboard.
Not every step needs an LLM
Agents make this easy to forget. Asked whether a server is online, an agent does not need a 200-billion-parameter reasoning model to ponder whether the server exists. It runs a health check, interprets the result and answers. Asked how much disk space is left, it runs:
df -h
then parses the output and replies. An expensive reasoning model on every step is not intelligence. Sometimes it is just bad engineering.
Permissions
Once an agent can delegate actions, a harder problem arrives: what may it do? Reading a calendar differs from sending an email. Reading a repository differs from merging a pull request. Checking a server differs sharply from deleting files. A useful personal agent therefore needs a permission model along with its tools, roughly like this:
- Reading happens without asking.
- Preparing drafts and plans happens without asking.
- Low-risk actions may run on their own.
- Anything consequential waits for a human yes.
Asked to find why a service is down, the agent can just get on with it. If it decides the database is corrupted and plans to delete and recreate it, a confirmation should come first. Autonomy should not be binary. It should be graduated by risk.
Memory and Context
Another reason against turning every specialized tool into the personal assistant is context. Codex should understand the repository, but not the user's whole life. An infrastructure tool should know the servers, not every conversation ever held. The calendar knows meetings and the task manager knows tasks. The orchestrator can become the layer that understands how those systems relate.
Take the question of what is happening with a particular project. No single database query answers it. A good reply draws on recent development work, GitHub activity, tasks, earlier decisions, deployment status and what the user planned to do next. This cross-system context is where a personal orchestration layer becomes truly valuable.
The Overengineering Trap
The danger is obvious. Thinking this way makes it easy to draw a diagram like this:
User
↓
Intent Classifier
↓
Planning Agent
↓
Model Router
↓
Memory Agent
↓
Permission Agent
↓
Coding Agent
↓
Validation Agent
↓
Reflection Agent
↓
Supervisor Agent
↓
Another Agent Watching the Other Agents
and then to spend three months building an AI organization so that one can ask what is on the calendar today. That helps nobody. The lesson from the whole experiment is that complexity has to earn its place. Twelve agents because an architecture diagram looks cool is a bad reason. Capabilities that repeatedly save time are a good one.
Delegation Before Autonomy
Fully autonomous agents get plenty of attention: hand one a goal, leave it running for hours, let agents converse and decide among themselves, and come back once everything is done. That is interesting, but it is not the place to start for a personal system. Highly reliable delegation should come before maximum autonomy. The aim is to say do X and have the system dependably choose the correct tool or agent and report what happened. Once that works extremely well, autonomy can rise gradually.
More Specialized Tools, Not Fewer
At the start the goal looks like consolidation: too many AI tools, so build one AI that replaces them. That turns out to be wrong. The end state may well involve more specialized AI systems, with the difference that they no longer need individual management, which is what the orchestration layer solves. The future may be less one AI to rule them all and more one interface to coordinate them all. The difference is subtle and large.
What the Agent Is For
After running such a system, breaking things, wiring up a chat interface, trying models and thinking hard about specialized coding agents, the role of the personal agent comes into focus. It does not have to be the smartest AI in the stack. It has to be the one that knows the person and knows their systems. It should understand what is being asked, which context matters, which capability suits, what it may do, when it needs confirmation and how to bring the result back. That is enough, and everything else can be delegated.
The Architecture Worth Wanting
The destination looks roughly like this:
ME
│
▼
GOOGLE CHAT
│
▼
┌─────────────────┐
│ HERMES │
│ │
│ Intent │
│ Context │
│ Memory │
│ Permissions │
│ Orchestration │
└────────┬────────┘
│
┌──────────────────┼───────────────────┐
│ │ │
▼ ▼ ▼
AI MODELS CODING AGENTS TOOLS
│ │ │
Local / Cloud Codex/OpenCode GitHub/APIs
Fast / Strong Servers/Tasks
│ │ │
└──────────────────┼───────────────────┘
│
▼
HERMES
│
▼
ME
No individual box is the point. Everything beneath the orchestrator should eventually be replaceable. A better coding agent appears: swap it in. A better local model appears: swap it in. Cloud inference gets cheaper: route more tasks there. A provider turns unreliable: route around it. Redesigning how the assistant is used every time the AI industry shifts should never be necessary.
Personal AI, Possibly Backwards
Most AI products compete on how smart the model is, which is understandable. As models keep improving, however, another layer grows in importance: coordination. That means knowing what the person means, their context, which systems matter, which tools exist, which model fits, which actions are safe, when to ask, and when not to use AI at all. The assistant worth wanting is not necessarily the one with the highest benchmark score. It is the one that makes all the intelligence and software around a person feel like a single system.
The Short Version of the Conclusion
The starting point is a wish for a personal AI agent. Deploying one shows that a persistent assistant is really an infrastructure problem. Hunting for the perfect model shows that model selection is really a routing problem. Deciding what the agent should do shows that a personal assistant may be an orchestration problem at heart. The architecture shifts from hunting for the cleverest model and handing it every tool toward a system that grasps intent and dispatches work to the proper place. It is a much less magical description of Jarvis, but a far closer one to something that can actually be built and used every day.