Articles for people who
ship the stack
Rewrites on React, Node.js, TypeScript and AI. A translated body is shown when it exists. If it does not, you see the English text and a note that says so.
Tagged: Reliability
Why Agent Accuracy Scores Mislead: pass^k, Perturbations and Fault Injection
Why pass@1 hides unreliable AI agents, and how pass^k, input perturbation sets and fault injection reveal consistency, stability and fault-tolerance gaps.
9 min readThe Naked Agent: What Breaks When Nothing Guards the Model
Build the same unprotected travel agent in the Claude Agent SDK and LangChain Deep Agents, then watch three failures show which harness controls are missing.
9 min readWhy Production AI Agents Need More Than a Good Prompt
Production AI agents need more than a good prompt: context, tools, memory, failure handling, observability, testing and autonomy matched to the actual risk.
7 min readTesting a Rubric Gate That Retries an Agent's Wrong Final Action
A 30-case experiment: an LLM judge plus one retry cut wrong support actions from 6 to 2, and why detecting a violation doesn't always mean the agent fixes it.
7 min readWhy the Agent Harness, Not the Model, Decides What Becomes Real
A timed-out CRM update shows why agents need tool contracts, unknown-outcome states, idempotent retries and verified completion, plus controls worth keeping.
6 min readBefore an AI Agent Gets Production Authority: Contracts, Claim Graphs, Shadow Runs and Canaries
An evidence-first release process for AI agents: scoped contracts, stateful scenarios, fault injection, uncertainty bounds, staged canaries and machine-enforced promotion.
23 min readHarness, Loop, Graph: Three Layers Behind Most Production Agent Failures
Production agent failures rarely come from the model. Separate the tool harness, the loop's stop condition and the LangGraph graph topology, each with an owner.
24 min readWhy Agents That Dazzle in Demos Fail in Production, and What to Build Instead
Why AI agents that dazzle in demos fail in production: where they break, single vs multi-agent trade-offs, retries and fallbacks, and the real running cost.
9 min readAgent Evals With a Travel Bot: Outcome vs Trajectory, Gates and pass^k
A worked travel-booking example of evaluating agents: outcome and trajectory axes, deterministic gates before weighted scores, a 2x2 matrix and pass@k vs pass^k.
8 min read
About these articles
Request a 24h estimate
Need the same stack in a production operator layer? Send the brief — estimate within 24 hours.