Five Levels of AI Engineering Projects, From Chatbot to Self-Improving Agents

Hiring managers for AI engineering roles see the same portfolio over and over: a chatbot with a nice interface. The gap between that and what teams behind products like Claude Code and Codex actually build looks enormous, but the climb from one to the other follows a fairly clear sequence of five levels. Knowing which one a project sits on tells you what to build next, and many people stall because they keep making more projects for the first rung.

Level 1: a model behind a simple interface

At the bottom, a project calls a language model API (usually OpenAI or Anthropic) and puts something quick like a Streamlit app in front of it. Think of a chatbot that answers customer questions or a small feature that summarizes articles. There's prompt engineering, though the prompts usually aren't tracked or tested. There might be structured output, a few tests to confirm it parses, and some fallback logic for when the model flakes.

A beginner can get here by watching free tutorials to see what tools exist and how they fit, then spending a weekend on a personal problem, such as a meal-prep coach.

The limit is that it only works when it happens to work. Prompts aren't versioned, so there's no telling whether a change helped. There are no metrics for output quality. And the bot knows only what the model was trained on plus whatever it finds on the web, which leaves out the context your company holds.

Level 2: retrieval and evaluation

Once those limits start to hurt, you're at level 2. Say someone asks for a system that can use the team's internal documentation. The answer is retrieval-augmented generation, which lets the model consult an internal knowledge base.

Plugging in RAG is the easy part. The difficult part is knowing whether the answers are right and, when they aren't, finding out why. That calls for knowing how to chunk documents, how to embed them for a vector database, and which of the many retrieval approaches will surface the right chunks for a new question. It also means building evaluations that are specific to each component as well as the whole system. You have to choose metrics, know what good looks like, and measure whether responses serve what the customer needs.

To climb from level 1, take the chatbot you already built and connect it to something the model has never seen. For the nutrition coach, that could be your own food logs or favorite family recipes. When you get this right, you can ask questions about your own data, get tailored answers and trust that the system is accurate.

Then comes the next itch: getting the AI to do something with that information.

Level 3: agents

At level 3 the system takes action instead of only answering. An agent is a model that can use tools and make decisions. You give it a goal and it works in a loop: decide, act, check the result, choose the next move, repeat until finished. Its tools might search the web, call APIs, run code or update a database. A research agent might pull from dozens of sources to help write a brief, and a support agent might look up orders and process requests alone.

This is arguably the most fun part of AI engineering today, and one of the riskiest. Each capability you hand over is another place for things to go wrong. At level 2, evaluation meant scoring one answer. Now you're grading a string of decisions, and a wrong turn at any step poisons everything after it. The consequences are also real when the agent touches your credit card, your code or your email, so the stakes are higher by nature.

A practical way to feel this: a level 2 bot that gives a bad answer wastes a minute of someone's time. A level 3 agent that misreads an instruction can send the wrong email, charge the wrong card or push broken code, and nobody sees it until the damage is done.

An agent that handles that responsibly can complete multi-step tasks start to finish and save hours. To get there from level 2, give the existing system a real toolset but keep it read-only at first. The coach could search meal logs, use a nutrition calculator and search the web, with no ability to change anything. Then get the loop working across those tools, with observability in place. Only after the agent proves reliable and secure should it receive more power.

There's still a gap between one agent that helps you, or serves as an internal tool, and a multi-agent system that millions of people can rely on.

Closing it means thinking beyond whether the agent works on your laptop. It has to keep working when thousands of requests arrive at once and when users do things nobody planned for.

Level 4: enterprise-scale systems

This is the territory behind roles said to pay over $300,000 a year. Instead of one agent doing everything, specialized agents hand work to each other. One breaks down the request and routes it, others each handle a part, and another checks the work before it goes out. A support setup might have one agent triage and route the ticket, separate billing and technical agents answering their share, and a final agent reviewing the whole reply before a customer sees it, all of it running for millions of people simultaneously.

Coordinating a team of agents is a different animal from running one. Each added agent is another thing that can fail, one agent's mistake flows straight to the next, and since this is live in front of customers, bad responses are public and costly.

A look at the Claude Code codebase showed that the AI itself is a tiny sliver of it. Nearly everything else is scaffolding that keeps the product reliable.

Models are the part everybody talks about, yet the hours at this level go into plumbing: retries, permissions, logging, state handling and the checks that stop one bad step from spreading. So most of the work at this level surrounds the agents: tracing, so you can see what each agent did when something breaks; guardrails, so nobody can trick an agent into leaking data; and control over cost and latency, on top of the prompts, RAG and evaluation from earlier levels.

To practice the jump from level 3, split the nutrition coach into a small team. One agent plans meals, one builds the grocery list (and perhaps orders groceries), and a checker reviews the plan before you see it. Then try to break it deliberately: slip a fake ingredient to the planner and see whether the checker catches it or waves it through. The millions-of-users part is something you'll only do on the job, but you can still build something that could scale by using cloud services and thinking about cost and reliability from the start. Teams of agents that hold up at scale are what companies run their business on, and the salaries reflect that.

Level 5: agents that build the system

Even at level 4 the engineers are the bottleneck. At level 5 you step out of the way. This is the most advanced work in the field: instead of constructing the system by hand, you give agents the job of improving it.

That goes beyond using coding agents to write code, which happens at every level, and beyond spec-driven development where an agent implements what you describe. The top teams building Claude Code and Codex no longer steer agents through each change. They're building systems that improve themselves, using a cycle where an agent examines performance, proposes a change, makes it, tests whether it helped, then keeps or discards it, and goes around again for hours or days with no human involved.

Two data points show where this is. In one project by Andrej Karpathy, a single agent ran about 700 experiments over two days on its own and found twenty ways to speed up model training. Anthropic has said that most of the code being merged into its own codebase is now written by Claude.

The whole cycle depends on the agent judging its own work correctly, again and again, unchecked, and that's hard. If scoring is even slightly off, the agent won't improve the product. It'll find the flaw in the scoring and exploit it, collecting a great score while the real product gets worse in ways nobody thought to measure.

So the skills here are serious. You must design an evaluation the agent can't cheat and tie its performance to what makes the product good, so it can't just game an easy number. You must instrument everything, since you can't watch it live and can only read the trail left behind. And you must know exactly where a human still needs to step in. Even the best teams aren't fully hands-off: people pick the problem, design the scoring and approve what reaches production.

This isn't a level you reach through a couple of weeks of self-study. Each skill comes from years of running production systems, watching them break and improving.

The reasoning is simple. You can read about reward hacking in an afternoon, but spotting the moment your own metric is being exploited takes seeing it happen on something that matters, and then rebuilding the measurement until it holds up.

Most people don't need level 5. Plenty of production AI work is realistically level 2 or 3, and sometimes a well-made prompt with simple evaluation is enough to build something useful. Engineers are hired to be useful, not fancy.