From Models to Agents: Reworking the Three Lines of Defense

Agentic AI breaks the governance playbooks written for conventional software and conventional models, but that claim is easier to see in a concrete case. Years of MLOps work, covering model inventories, validation pipelines, drift monitoring and approval workflows in enterprise and financial-services settings, do not get discarded when an organization adopts LLMs and agents. What changes is narrower and more specific than most people assume.

Imagine a model risk team handling an ordinary intake request. The form asks its usual questions about training data, version number, approval status and the date of the last drift check, and the team has processed plenty of them. Then the system description lands: four cooperating agents, six tools shared among them, a retrieval index rebuilt every day, and an outcome produced by a chain in which every individual step looked fine. Nothing on the form fits.

A 2026 McKinsey AI Trust survey covering about five hundred organizations paints a similar picture: roughly a third of enterprises say their governance maturity is adequate for the autonomous agents already running. The rest keep using the old intake form and hope it is good enough. It usually is not, so the useful task is to pinpoint where it fails.

What the existing program already covers

MLOps governance is the collection of controls keeping a traditional machine learning model trustworthy from training through retirement. In practice:

• Model versioning, so it is always clear which model runs in production and why.
• Drift monitoring that catches performance decay as input data shifts.
• Data lineage back to the training and validation sets.
• Approval gates ahead of any production release.
• The CI/CD and change-management discipline expected of any production software, which sits inside this program and not beside it.

None of these enforce themselves. Someone must own each control, and someone else must verify that the owner did the job.

Responsibility is usually assigned through the three lines of defense, a framework that banks and other regulated industries have long used to allocate accountability. One accuracy point: the Institute of Internal Auditors renamed it the Three Lines Model in 2020, tilting the emphasis toward creating value and away from pure defense. Finance and model-risk circles still say three lines of defense, and so does this text, though the official title has changed.

In a conventional MLOps program the split looks like this:

• The first line, covering product, data and engineering, owns development, testing, deployment and documentation of models.
• The second line, covering compliance and risk, sets policy for model use, monitors for bias and reviews high-risk applications before launch.
• The third line, internal audit, has the narrowest brief: independently confirm that the first two lines do what they claim, without building or approving anything itself.

This structure is mature and well understood, and it works for the systems it was designed around, namely models whose structure is fixed, validated once and monitored between review cycles.

How each line has to change for agents

AgentOps governance begins from the same three lines, but the substance each one inspects differs, because the systems behave differently. The gaps below come down to five specific differences.

One artifact becomes many. A classic model is a single thing to version. The four-agent system is a bundle: the base model, four system prompts, six tool definitions, and a retrieval layer refreshing on its own clock. Each of those can alter behavior, so the first line must version and test every piece. That looks less like a registry entry and more like prompts and tool schemas treated as source code, with version control, pull-request review and canary tests against a holdout set before anything is promoted. Tracking only the model leaves roughly three quarters of the behavioral surface unmanaged.

Correct output becomes acceptable range. Give a traditional model the same input and it returns the same output, which is why validation is simple. An LLM may produce two different but defensible answers across runs. So the second line stops asking whether an answer is right and starts asking whether answers stay within an acceptable band most of the time. That calls for an evaluation gate with stated thresholds, for example a minimum pass rate on a labeled test set before release, and a human-review trigger when confidence or evaluation scores dip under a floor. Agreeing with compliance on what acceptable most of the time means for a given use case is a project of its own, not something finished in an afternoon.

Single owner becomes a chain. A lone model making a lone prediction has an obvious owner. A four-agent system has none. When the final result is wrong but no step was individually wrong, diagnosis needs a decision log capturing the intermediate handoffs as well as the end product: inputs per agent, each decision, and what was passed on. Lacking it, nobody can say which agent is accountable, and the first and second lines argue about it after an incident instead of agreeing beforehand.

Drift monitoring is not behavior monitoring. Drift checks compare predictions with a known baseline. They are blind to what an agent does mid-run: the tools it invoked, the actions it took, whether it stayed within scope. Agent observability platforms target that blind spot by recording the whole execution trace, down to tool calls, parameters and the way one agent's output feeds the next. Many rely on open standards such as OpenTelemetry, which keeps traces portable between vendors. For this system the trace is the sole artifact from which an auditor can rebuild events afterwards. Absent it, the third line audits a black box and labels it independent verification.

Calendar retraining becomes continuous change. Traditional models retrain on a schedule: monthly, quarterly, whenever enough new data accumulates. A system built on RAG or continuous fine-tuning can change its effective behavior as often as its knowledge base updates, possibly daily. If the retrieval layer refreshes nightly, a third line built around quarterly reviews checks the system about ninety times less often than it changes.

Where the two programs meet

Compared directly, the two programs share a skeleton. The same three lines apply, and both require versioning, testing, review and independent audit. The differences lie in granularity, frequency and the object each control inspects.

MLOps governance is not mistaken here. Its questions were simply built on the premise that model behavior holds steady between reviews, and a system whose retrieval index rebuilds nightly refuses to stay put that long. All five differences come from that one premise failing: what used to change monthly can now change daily, occasionally with no one touching the model at all.

Closing the gap does not require a new governance function built from scratch. It requires extending the existing lines with capabilities they did not need before, and being candid about the cost of each:

• First line: version control for prompts and tools, with canary testing before promotion. This is mostly an engineering practice change and not a new team.
• Second line: an evaluation gate that states acceptance thresholds, along with a written escalation route for decisions spanning several agents. This is a real policy-writing project, and the first pass will involve genuine negotiation with legal and compliance.
• Third line: runtime observability tooling and the skills to audit a decision log in place of a static model card. For most model risk functions that means a new hire or serious training, not a checkbox on an old review template.

None of these is optional if the aim is to answer credibly which of the four agents made the call.

What regulators have said so far

No single regulatory framework has fully caught up with agentic AI, but three existing models from different industries show where things are heading.

• The EU AI Act applies across sectors. Article 50 transparency duties start on August 2, 2026, whatever else happens. The more demanding Annex III high-risk duties shared that date at first, but the proposed Digital Omnibus package, agreed in principle in May 2026 and still awaiting formal adoption, would delay them to December 2027. A compliance calendar should list that delay as pending.
• The FDA's treatment of AI-enabled medical devices is vertical and product-centered, and its signature tool is the Predetermined Change Control Plan. Under a PCCP, the original approval already blesses certain later model updates, provided they stay inside stated performance limits. It comes with Good Machine Learning Practice guidance and a quality system moving toward ISO 13485. In effect, change control becomes part of the approval route, not an obligation bolted on afterwards.
• Finance is instructive partly for what it leaves out. The 2026 interagency guidance SR 26-2, which superseded SR 11-7, explicitly puts generative and agentic AI outside its formal scope. A mature framework is conceding that tools designed for one generation of models do not stretch automatically to the next, and firms working under it are already building agent governance ahead of supervisors.

None of the three works as a template to copy. Read together, they suggest a direction: change boundaries authorized in advance, risk tiers, and an openly acknowledged mismatch between current rules and agent behavior.

Back to the intake form

The existing intake form can stay. Alongside it, the team needs a separate questionnaire for multi-agent systems, organized around the three lines it already relies on, and it should expect that answering honestly takes actual effort.

Which step will a program catch up on first? Probably versioning, since it resembles a version-control problem that engineers already know how to solve. Accountability for chained decisions will likely linger longest. It is an organizational design problem in technical clothing, and no evaluation gate repairs that.