AI Agent Audit Logs Need 12 Fields: What LangChain, CrewAI and Four Others Record

A trace looks like evidence but isn't

Picture an agent invoking a tool: funds transfer, a database row gets deleted, or a message is sent. Afterward, somebody has to answer one question, namely whether the agent's actions can be proven. The trace gets opened, and it shows a span carrying a name, a start time, an input and an output. That looks like evidence, but it's a debugging aid wearing evidence's clothing.

To measure the gap, six open-source agent frameworks and instrumentation specs were cloned at their latest commits on 4 September 2026: the OpenAI Agents SDK, LangChain, CrewAI, LlamaIndex, smolagents and OpenInference. The fields on each project's tool-call record were pulled out and compared with the twelve mandatory fields in the newest agent audit trail format. The headline: the typical framework captures five of the twelve fields an audit record needs, and none of the six has a single tamper-evidence field.

What frameworks capture suits debugging. Audit records serve another purpose, and the distance between the two can be counted.

Five findings from the count

• Across the five frameworks the median coverage is 5 of 12 mandatory fields: smolagents has 1, LlamaIndex 3, OpenAI Agents SDK 5, CrewAI 5 and LangChain 6.
• The string trust_level shows up in zero files across all six codebases. A search through every Python and TypeScript file in every repository finds nothing.
• prev_hash, previous_hash and hash_chain also return zero files. No framework links one record to the next, so a trace store can be edited later without leaving a mark.
• OpenInference, the fullest instrumentation spec in the ecosystem at 119 namespaced span attributes, still reaches only 8 of 12 and includes no tamper-evidence field.
• Just one field, timestamp, is logged by all five frameworks. agent_id, session_id and record_phase each show up in one of five, and agent_version, outcome and trust_level in none.

Frameworks get good instrumentation for debugging and weak instrumentation for evidence. The two resemble each other closely enough that the substitution goes unseen until someone asks for proof. The absent fields aren't exotic. They establish who acted, under whose authority and whether the record was altered afterward, questions a debugging trace never had to answer.

How the count was done

Commits were fixed per project: OpenAI Agents SDK (openai-agents-python) at 89c02c8, LangChain at 0d50cbd, CrewAI at 92eb5f9, LlamaIndex at d2ac544, smolagents at 30bb116, and OpenInference at 479d168. The field list for each came from the class that logs a single tool call, meaning the span envelope together with its tool-specific payload, and it was read from class definitions and not from the docs. Afterward the fields were matched against the twelve mandatory ones in the current audit-trail draft.

Where a field satisfies a requirement only in part, it was scored as partial with a reason attached. LangChain's error reports that something failed, for instance, but doesn't separate failure from timeout, denial or escalation.

The capture shows the run itself: six repositories at head, every tool-call field extracted and scored against the twelve fields. trust_level and prev_hash come back zero everywhere.

The twelve fields

The yardstick is the agent audit trail Internet-Draft, revision 02, which Raza Sharif of CyberSecAI published on 3 September 2026. It's an individual submission and not an endorsed IETF standard, so it counts as one carefully reasoned proposal and not settled law. Even so, it's the most concrete published answer available, and its twelve mandatory fields are hard to argue with:

• record_id is a unique identifier for the record.
• timestamp gives the event time in RFC 3339 format.
• agent_id names the agent instance that acted, and agent_version names the version of its software.
• session_id ties the action to a session.
• action_type uses a controlled vocabulary and not free text, and action_detail holds the type-specific payload.
• outcome is one of success, failure, timeout, denied or escalated.
• trust_level states the authority the agent operated under, from L0 to L4.
• parent_record_id points to the previous record in the chain, and prev_hash holds a SHA-256 hash of it.
• record_phase says whether the record was written before, during or after execution.

Put that list next to a span object and the problem's outline shows up. The first eight fields are ones a debugger might plausibly want, while the final four matter only to an auditor, and those four are exactly the missing ones.

Counts by framework

Field counts on the tool-call record, with coverage against the twelve:

• CrewAI (ToolUsageFinishedEvent plus BaseEvent): 25 fields, with 5 full, 3 partial and 4 absent.
• LangChain and LangSmith (RunTree 0.12.1): 20 fields, with 6 full, 1 partial and 5 absent.
• OpenAI Agents SDK 0.22.0 (SpanImpl plus FunctionSpanData): 13 fields, with 5 full, 2 partial and 5 absent.
• LlamaIndex (SimpleSpan plus BaseEvent): 10 fields, with 3 full, 1 partial and 8 absent.
• smolagents 1.26.0 (Timing plus TokenUsage): 6 fields, with 1 full, none partial and 11 absent.
• OpenInference semantic conventions define 119 namespaced span attributes (distinct dotted attribute-name constants at commit 479d168, leaving out another 43 that are enum values), and rate 8 full, 1 partial and 3 absent.

Raw field count doesn't forecast auditability. CrewAI logs the most fields but covers fewer requirements than LangChain, since much of its twenty-five describe crew context, like agent_role, from_task, plan_step_number and delegations. They're handy and still answer no audit question. Instrumentation richness and evidential fitness barely correlate.

The chart shows it plainly: CrewAI holds more fields than any other framework yet satisfies fewer requirements than LangChain, because most of its extras are crew context and not evidence.

One thing sets CrewAI apart. It emits ToolUsageStartedEvent and ToolUsageFinishedEvent as separate classes, so it alone has a true record_phase, a record that exists before the tool runs. This matters more than it seems, because a log written purely after the fact can show something happened but can't show it was approved beforehand.

What no framework records

Four of the twelve fields are missing from all five frameworks, and two of those stay missing even when the instrumentation spec is counted.

• trust_level turns up in no file of any of the six repositories. Without it, nothing records the authority an agent acted under, so a trace can't separate a permitted action from a forbidden one.
• prev_hash appears in zero files, as do previous_hash and hash_chain. Nothing chains records, and a trace store is an ordinary database whose rows can be changed.
• agent_version is missing from all five frameworks. It exists only in OpenInference as a mirror of OpenTelemetry's gen_ai.agent.version, something that can be set but that, on this reading, the frameworks themselves basically never set.
• outcome as a classification earns no full score from any framework. All have an error field, yet none tells denied apart from failed, the split an auditor values most.

Searches for non_repudiation and pre_execution also return zero files.

Why the fields are missing

These tools were designed for another question. A trace helps an engineer discover why something ran slowly or wrongly, and input, output, duration and a parent pointer come close to sufficing for that. Evidence must hold up against an adversary, including whoever controls the log, and none of these systems was built with that threat in mind. The gaps aren't an oversight. Nobody expected traces to double as evidence until this year.

The strongest counter-argument

Could OpenTelemetry fill the gap? Partially. OpenInference covers 8 of 12, the top result in the set, and OpenTelemetry's GenAI conventions define gen_ai.agent.id and gen_ai.agent.version, which shuts the identity gap in theory. Applied consistently, adoption alone would settle four of the twelve with no new work.

Yet the two toughest requirements remain open. Tamper-evidence can't be added to a span as an attribute, since it belongs to the store and not the record. A hash chain has meaning only if it's written where the writer can't rewrite it. And record_phase needs a record emitted before the action executes, which is an architectural choice about timing and not a field appended later. Neither falls under semantic conventions.

Limits of the measurement

A number is only useful if its edges are honest, so the caveats matter:

• This is static analysis and not runtime capture. Fields the classes define were counted. Six agents weren't run with their emitted payloads compared, and a framework might populate a field through an extension mechanism that wasn't traced.
• metadata, tags and extra are escape hatches and were scored as absent. Most of these frameworks accept arbitrary keys, so a determined team could store a hash chain in extra. The count measures defaults and not ceilings.
• The closest thing to a signature isn't one. OpenInference defines message_content.signature, but its docstring describes an opaque vendor-issued value stored as received, like Gemini's thoughtSignature. It vouches for the model's reasoning content and not the integrity of the record, and only the provider can check it. Scoring it absent for tamper-evidence was intentional.
• The twelve-field reference is one draft from one author at revision 02, so it isn't ratified, and people will reasonably dispute trust_level. If the requirement list moves, coverage numbers move too.
• Six projects aren't the whole ecosystem. Pydantic AI, Semantic Kernel, AutoGen and commercial platforms weren't counted.
• Presence says little about quality. timestamp counted as satisfied in all five, yet a plain datetime.now() lacking a timezone or outside anchoring is much weaker than an RFC 3161 timestamp.

What this can't decide is whether any of it matters for a given system. An agent that reads docs and drafts text is well served by a debugging trace, and audit-grade records would be overkill. The gap hurts only when an agent's actions carry consequences someone could later contest.

Three changes in order of payoff

1. Write a record before execution. That means before the call and not after, noting the record ID, timestamp, agent ID and version, the action about to run and the authority behind it. CrewAI's started/finished split is the template, and other stacks need it built by hand.
2. Chain records into append-only storage. Hashing the previous record with SHA-256 is nearly free. What counts is the place it's written, not the hash itself.
3. Give outcome a vocabulary. With five values (success, failure, timeout, denied, escalated), the difference between a tool error and a refusal becomes queryable and doesn't stay buried in a string.

No new framework is required, only about two hundred lines around the existing one. It mirrors the pattern of silent agent failures that dashboards miss, where an absence looks like health because nothing was recorded.

Quick answers

How does an agent trace differ from an audit log? Traces are written for engineers who want to explain behavior afterward, and they presume an honest writer. Audit logs are written for a party in dispute and must establish identity, authority and integrity, meaning who acted, with what permission and whether the entry has changed since. By comparison, the frameworks hold between 6 and 25 trace fields each yet satisfy a median of only 5 audit requirements.

Can LangChain traces be made audit-ready? Only in part, and by building around the framework instead of configuring it. LangSmith's RunTree supplies 6 of the 12 directly, including session_id and parent_run_id. Still to add are who the agent is and which version ran, a fixed set of outcome values, a trust level, a record written ahead of execution, and a hash chain kept in append-only storage. Custom data can ride in the extra field, though nothing fills it automatically.

Does OpenTelemetry solve it? It covers the identity half. Tamper-evidence and pre-execution recording stay open, since they depend on when and where records are written and not on which attributes a span holds.

Which framework is most audit-friendly today? LangChain via LangSmith covers the most with 6 of 12, while CrewAI uniquely offers a real pre-execution record plus a first-class agent_id. Neither ships ready for audits. If auditability is non-negotiable, build the evidence layer separately and treat the framework's trace as raw input.

Is a hash chain excessive for an internal tool? Typically yes. The deciding test is whether anyone beyond the team could ever depend on the log, such as a customer, a counterparty, an insurer or an incident review with real stakes. If not, a good trace suffices. If perhaps, adding the chain now costs far less than reconstructing it later.

Arguments about agent oversight have run for months on adjectives. Counting settles part of the question, because a field name either sits in the class or doesn't.