Nine Engineering Habits for Shipping Your First AI Agent

One story circulates in agent-engineering circles about a bill of $47,000. It's retold without much detail, so treat it as an illustration and not an audited case. An agent behaved flawlessly in every demo. Then it was pointed at a real customer workflow, ended up in a refund loop, firing the identical action over and over for six hours with nobody watching. The model wasn't too dim. The people who built the system had skipped everything around the model that tells it when to stop.

That's the uncomfortable truth about doing engineering work on AI systems. You can know Python, RAG, agents, vector databases, prompting and every model release, and still be unready for production. The skills that rescue you after a demo breaks rarely feature in tutorials. They're dull and easy to skip, and they separate an impressive-looking system from one you can trust. Here are nine working habits worth having before you ship anything autonomous.

Why knowing more doesn't make you senior

A few figures show how often projects stall between demo and deployment. They're secondhand and methods vary, so read them as a rough picture:

• Gartner forecast in mid-2025 that over 40% of agentic AI projects would be cancelled before the end of 2027, pointing to rising costs, fuzzy value and weak risk controls.
• A 2025 survey by S&P Global Market Intelligence reportedly found enterprises dropping about 46% of their AI proofs of concept before production.
• MIT's NANDA initiative, after studying more than 300 deployments, is said to have found that only around 5% produced measurable financial impact.

No model suddenly got 40% worse at reasoning. Something breaks between the demo and the launch. The working theory here is that production failures are often caused or magnified by the system surrounding the model and not by the model's output alone: missing guardrails, bad data, weak validation, thin observability and uncontrolled spending.

A benchmark points the same way without proving the cause. On tau-bench, which tests tool-using agents on jobs such as reserving flights and resolving support tickets, one summary said that even top models finished fewer than half of them, and that repeating a task saw consistency fall to roughly a quarter. That measures capability and doesn't diagnose anything. It says agents are unreliable, not why.

Hold on to one distinction, because it runs through every habit below: the loop above wasn't a wrong answer. Nothing stopped it. The gap between a model's output and what the system lets happen with that output is where most of these habits live.

1. Write the eval before the feature

Beginners write a prompt, try it a handful of times, like what they see and ship. More experienced engineers first write down what "correct" means as a set of test cases, and only then touch the feature.

An eval can stay simple. Collect 20 to 30 real examples from actual use, each with its expected outcome, and run your prompt against all of them whenever something changes. If you can't tell whether a change helped without staring at outputs and guessing, you have a feeling and not an eval. This one habit is the quickest way to stop shipping regressions you can't see.

2. Spend context like money

The beginner habit is to cram everything in: the whole document, the entire chat, five examples "just in case". More context feels safer and usually isn't. Each extra token competes for the model's attention and gives it one more wrong detail to fixate on. Treat the window as a budget:

• Cut retrieved documents down to the relevant parts.
• Summarise older conversation turns and don't replay them in full.
• Remove examples that aren't pulling their weight.

If you don't know what's filling the window, print the whole prompt you send and read it. Plenty of people are surprised by how much is dead weight.

3. Put a leash on autonomy before launch

This is the habit that would have stopped the refund loop. One review of costly agent failures, by a group called Operator Collective, sorted the cases into five categories: no limits on autonomous action, stale or bad data, trusting output without verification, security as an afterthought, and failures nobody saw because nothing was watching. None of the five is "the model reasoned wrongly". That's a single review and not a law of nature, but it indicates something real. A wrong output rarely does harm alone. The harm comes when nothing downstream catches it, caps it or prevents it repeating.

The arithmetic makes this non-negotiable. If each step of an agent workflow succeeds 95% of the time and the task has 20 steps, the whole thing doesn't finish correctly 95% of the time, because the odds compound. At 90% per step, a 20-step chain succeeds only around 12% of the time. A demo that runs three steps cleanly says almost nothing about a production flow with twenty.

Before any autonomous system goes live, give it a step limit, a cost cap, a timeout and a way to stop itself. The reason isn't that the model can't be trusted. It's that probability never loses.

4. Log as if you'll debug blind at 2 a.m.

The most dangerous failure doesn't crash noisily. The agent takes the wrong action, reports success, the dashboard stays green, and a customer loses money or gets bad advice before anyone notices.

Beginners add logging after something breaks. Experienced engineers build structured tracing from day one: every input, tool call and intermediate reasoning step, timestamped and searchable. Three weeks later you want to replay exactly what the agent saw and decided instead of piecing it together from memory. If your present setup would have you guessing in the small hours, close that gap first.

5. Climb the ladder: prompt, retrieve, then fine-tune

When accuracy disappoints, newcomers often go straight to fine-tuning because it feels like "real" engineering. It's usually the wrong first step, and a well-known case shows why. Microsoft researchers took GPT-4, added a bundle of prompting techniques called Medprompt, and compared it with Med-PaLM 2, Google's medical specialist, tuned on clinical data and paired with prompts written by experts. Measured on MedQA, the fine-tuned Med-PaLM 2 scored 86.5% and prompting-only GPT-4 with Medprompt scored 90.2%. That amounts to a 27% reduction in error rate, with an order of magnitude fewer model calls than the fine-tuning route needed.

The sensible order runs from cheap and reversible to costly and slow:

Prompting        Retrieval (RAG)      Fine-tuning
(cheap, fast,  →  (add missing    →   (expensive, slow,
reversible)        knowledge)          last resort)

Fix the prompt first and measure it. If the model lacks facts, add retrieval. Fine-tune only when you hold a large, clean dataset and a specific behaviour that prompting truly can't produce. Most teams never need to, because the earlier steps solve the problem.

6. Keep the brain apart from the hands

A beginner's agent is often one huge prompt that plans, decides and executes in a single breath. When it fails, nothing separates "the model thought wrongly" from "the code did the wrong thing", because it's all tangled together.

Experienced builders split the two. One part, the model, acts as the brain and chooses the next move. Another part, plain deterministic code that works as the harness, performs it: it calls the API, checks the response, retries within limits and enforces the guardrails from habit three. That split is what makes a system debuggable. You can tell at once whether the model picked the wrong action or the execution code botched a right one, which are different bugs with different fixes.

7. Treat prompts as code and version them

A prompt that needed four rewrites is production logic. Handle it like scratch paper and you'll lose the working version the moment someone "improves" it. Keep prompts in version control like any other file, run your eval set from habit one against every change before it merges, and record a short changelog of what moved and why. It sounds too basic to mention, which is exactly why most teams skip it, until a prompt regression reaches production and no one can say what changed.

8. Price the feature first

Here's one measured example. An unconstrained agent working a single software task burned $5 to $8 in API fees, while whole production sessions, retries and tool calls included, cost between $10 and $100. Figures will differ by model, task and optimisation, but the shape holds. A cost that looks trivial per task stops looking trivial at real volume, and a pilot that seemed cheap in testing can quietly become the reason a project is shelved.

Beginners build first and read the bill later. Experienced engineers estimate cost per task before writing any code: rough tokens in, rough tokens out, expected calls per task and expected daily volume. If the number looks off at ten users, it won't look better at ten thousand. Cost isn't a finance matter to handle after launch. It's an architecture decision you've already taken, whether or not you noticed.

9. Diagnose before you guess

When something breaks, the beginner reflex is to point at the model and swap in a bigger one or write a longer prompt. Don't. The failures in the review mentioned under habit three traced back to missing guardrails, bad data, unchecked trust, weak security or a failure nobody was watching for, and not to the model erring by itself. That's an observation from one set of cases and no guarantee about yours, but it's a cheap check to run before an expensive one. Before touching the prompt, ask whether the retrieved data was stale or wrong, whether a tool call failed silently and whether retries had any limit. The fix often lives in the surrounding system, and checking costs five minutes against a rewrite.

When to skip half of this

None of the nine habits is free. Evals take time to build, guardrails add code and cost modelling eats an afternoon you'd rather spend building. If you're prototyping over a weekend and you're the only user, skip most of it. Bring in the eval suite, guardrails and versioned prompts when other people start relying on what you made, not earlier. Running a one-person experiment with production ceremony burns time better spent finding out what works. The discipline pays off where the refund loop happened: systems that touch real users, real money or real decisions.

The shift in one sentence

According to that account, no smarter model fixed the loop. What fixed it was a set of written rules: caps on autonomy, checks ahead of each action and a person reading the logs. Governance, not intelligence. Beginners try to out-think failure with a better model. Experienced engineers build a system that fails safely when the model is wrong, because sooner or later it will be.