Architecting an AG-UI Frontend: Event Streams, Widget Registry, and Local LLMs

Demo chatbots that only paint Markdown look polished until someone asks for a table, a chart, a calendar, or a form. At that point the product stops being a chat wrapper and becomes an application—and Markdown is a poor application format.

A better framing emerged after several dead ends: stop asking the model to invent UI. Ask it to choose actions. The frontend already knows how to render; the backend already knows where data lives. The model should decide what happens next.

That split produced an assistant that renders sixteen production React widgets, supports backend-driven UI, runs against a local LLM, added no new frontend dependencies, and extends by registering one component.

In brief: refuse model-authored markup. Let the backend emit a sequenced list of typed events—prose, tool status, and widget intents—while the client keeps a registry and paints those events in order. The model’s sole job is selecting which backend capability should answer. That yields steadier output, reusable components, and a rendering path that does not churn as the catalog grows.

The Problem With AI-Generated UI

Many AI apps follow a narrow path:

User
  ↓
LLM
  ↓
Markdown
  ↓
React renders Markdown

It is simple and limiting. Users eventually ask to see active members, compare monthly revenue, list appointments, or open a profile. Those answers are interfaces, not paragraphs.

Asking the model to emit HTML or JSX fails when it invents missing components, drops required props, or brands a widget that does not exist. The frontend then crashes on imaginary markup. Generated markup also cannot be typed, unit-tested, reused outside chat, or reviewed in a pull request. The component library’s guarantees disappear. Determinism is the goal instead.

The Core Idea: the Model Chooses Actions, Not Components

The backend never returns HTML, JSX, or React component names. Every response is an ordered event stream:

{
  "events": [
    { "type": "tool",   "tool": "list_members", "status": "completed" },
    { "type": "widget", "widget": "table", "props": { "rows": [] } },
    { "type": "text",   "text": "I found 24 active members." }
  ]
}

The backend communicates intent; the frontend decides presentation. Responsibilities stay clean: AI decides what, React decides how, widgets decide appearance. The string "table" is a semantic key, not a class name—desktop can virtualize a grid while mobile stacks cards without the backend caring.

The Event Protocol

The contract stays small enough to hold in working memory. Typed events keep it safe:

type TextEvent = {
  type: 'text';
  text: string;
};

type ToolEvent = {
  type: 'tool';
  tool: string;
  status: 'running' | 'completed' | 'failed';
  error?: string;
};type WidgetEvent = {
  type: 'widget';
  widget: WidgetName;              // union of registered keys
  props: Record<string, unknown>;  // validated at the boundary
};export type AssistantEvent = TextEvent | ToolEvent | WidgetEvent;

Three event kinds cover the assistant’s responses. Order matters: a tool chip can appear before the widget it produced, and a summary sentence can follow. The transcript reads like a narrated workflow.

One Rendering Pipeline, No Switch Statements

Every message travels one path:

AiDialog
   ↓
ChatMessages
   ↓
ChatMessage
   ↓
MessagePart
   ↓
WidgetRenderer
   ↓
widgetRegistry[widget]

No ad-hoc switches litter the tree. One renderer owns each event kind: text becomes chat copy, tool becomes progress, widget becomes a React component. Adding capability does not rewrite the pipeline—it registers another widget. The renderer that handled widget one still handles widget sixteen.

Open for Extension, Closed for Modification

New widgets register; the pipeline stays closed. That is the open/closed principle applied to chat UI. Product growth becomes a catalog problem, not a rewrite problem.

Never Trust the Model: Validate Props at the Boundary

Model output is untrusted input. Before a widget mounts, props pass through schema validation (Zod in the sample). Invalid payloads become safe error UI instead of runtime exceptions deep in the tree.

const registry = {
  table: {
    schema: z.object({
      columns: z.array(z.object({ key: z.string(), label: z.string() })),
      rows: z.array(z.record(z.unknown())).default([]),
    }),
    component: TableWidget,
  },
  // ...
};

function WidgetRenderer({ widget, props }: WidgetEvent) {
  const entry = registry[widget];
  if (!entry) return <UnknownWidget name={widget} />;  const parsed = entry.schema.safeParse(props);
  if (!parsed.success) return <WidgetError name={widget} issues={parsed.error.issues} />;  const Widget = entry.component;
  return <Widget {...parsed.data} />;
}

Validation at the boundary keeps crashes out of leaf components and gives operators a clear failure mode when the model mis-shapes data.

Building a Real Widget Library

Widgets live in a registry keyed by semantic names:

export const widgetRegistry = {
  table,
  profile,
  chart,
  timeline,
  calendar,
  markdown,
  json,
  stats,
  progress,
  alert,
  fileList,
} satisfies Record<string, ComponentType<any>>;

export type WidgetName = keyof typeof widgetRegistry;

WidgetName is derived from the registry so TypeScript rejects unknown keys. Tables, profiles, charts, and forms become ordinary components the rest of the app can reuse outside chat.

Why I Added Zero Dependencies

The chat shell reused the existing design system and React stack. No new UI kit entered package.json. That constraint forced the assistant to consume the same buttons, tables, and tokens the product already ships—so chat and non-chat screens stay visually aligned.

Interactive Widgets Without React Callbacks

Widgets must not import the chat shell. They emit DOM CustomEvents for actions such as refresh or open-profile:

// Inside a widget — it knows nothing about the app
element.dispatchEvent(
  new CustomEvent('ai:action', {
    bubbles: true,
    detail: { action: 'open_member', payload: { id: member.id } },
  })
);

Widget
  ↓
CustomEvent
  ↓
AiDialog
  ↓
AI Agent

The dialog listens once, maps events to tool calls, and keeps widgets portable. The same table can sit on a members page and inside chat without chat-specific props.

The AI Doesn’t Decide the UI

The model never chooses colors, layouts, or component trees. It chooses tools. Backend orchestration turns tool results into widget events with validated props. Design stays owned by frontend engineers.

Replacing Hardcoded Endpoints With a Tool Catalog

Early prototypes hard-coded URLs. The mature shape ships a tool catalog the host can discover:

{
  "message": "Show me all active members",
  "tools": [
    { "name": "list_members", "endpoint": "/api/v1/members",        "method": "GET" },
    { "name": "get_member",   "endpoint": "/api/v1/members/{id}",   "method": "GET" },
    { "name": "member_stats", "endpoint": "/api/v1/members/stats",  "method": "GET" }
  ]
}

Each tool names an endpoint and schema. The model picks a tool; the backend executes it; widget events carry the result. Swapping infrastructure becomes a catalog edit, not a prompt rewrite.

Streaming Without Breaking the Contract

Streaming still emits the same event types. Partial tool status can flip from running to completed while text tokens arrive. The renderer appends or updates events without inventing a second protocol. Users see progress chips before final widgets land.

Building a Chat Experience That Feels Modern

The host page remains an ordinary listing. The assistant hides behind a Chat with AI control and answers using the same data and components the page already uses. Continuity matters more than a separate “AI product” skin.

Feature Flags Matter

A single flag gates the experience:

VITE_ENABLE_AI_CHAT

Flags let teams dark-launch the dialog, disable it under load, or expose it only to staff without redeploying widget code.

Testing a Widget Framework

Tests target three layers: schema validation rejects bad props, registry lookup resolves known widgets, and event ordering renders tool → widget → text as expected. Widget unit tests run outside chat. End-to-end tests open the dialog, ask for members, and assert a table mounts.

When This Architecture Is Overkill

Pure FAQ bots that only need paragraphs can stay on Markdown. The event protocol earns its keep when answers are interfaces, when widgets must be reused outside chat, and when the team refuses to let the model invent component trees.

Design Principles That Guided Every Decision

• Model chooses actions, not markup.
• One rendering pipeline for every message.
• Widgets register; the pipeline does not fork.
• Validate props before mount.
• Widgets communicate through CustomEvents, not chat imports.
• Tools live in a catalog, not hard-coded URLs.
• Stream the same event types you store.
• Gate with feature flags.
• Prefer zero new frontend dependencies when a design system already exists.

The Result

The assistant streams typed events, renders sixteen widgets through one pipeline, validates every prop, and extends by registration. Local LLM backends remain swappable because the contract is events, not vendor SDKs in the UI.

Final Thoughts

Teaching models to paint interfaces fights the wrong battle. Teaching them to select capabilities—and letting a disciplined frontend render those capabilities—preserves design systems, testability, and growth. The chat becomes another surface for the same components the product already trusts.

The members screen in the sample remains an ordinary product page. The assistant is not a second product; it sits behind a Chat with AI control and answers with the same data plane and the same components that page already uses. That continuity is deliberate. When the chat invents a parallel visual language, operators stop trusting it. When chat and page share widgets, every improvement to the table lands in both places.

Ordering inside an event array is part of the product language. A tool event with status running can appear first so the user sees work begin. A widget event can follow with rows. A text event can close with a one-line summary. Reordering those events changes how the conversation feels without changing any React component. Treat the array as a storyboard the backend authors and the frontend stages.

Schema validation is not optional theater. Models omit fields, coerce numbers to strings, and occasionally rename keys. Zod (or an equivalent) at the registry boundary turns those mistakes into a controlled error widget instead of a white screen. Logging the validation failure beside the raw payload makes prompt and tool fixes faster than guessing from a user report.

CustomEvents keep ownership clear. A widget that imports chat hooks cannot be reused on a static page. A widget that only dispatches member:open or table:refresh can live anywhere. The dialog becomes an adapter that translates those events into tool calls. That adapter is the only place that knows about the assistant session.

Streaming fits the same contract. As tokens and tool statuses arrive, the client appends or patches events in place. There is no second “streaming format.” If a tool fails mid-flight, a failed status replaces running and an optional text event explains the failure. Users see continuity instead of a spinner that dies silently.

Feature flags buy operational safety. Shipping sixteen widgets behind VITE_ENABLE_AI_CHAT lets teams disable the dialog under incident load without ripping code out. Combine the flag with per-tool authorization so a read-only catalog can stay up while destructive tools stay dark.

Testing strategy mirrors the architecture. Unit tests cover schemas and pure widget rendering. Integration tests feed a fixed event array into the renderer and snapshot the structure. End-to-end tests ask a real or mocked backend for members and assert that a table widget mounts with expected columns. Because widgets are ordinary components, existing visual regression suites can include them without a special AI harness.

Overkill cases still exist. If the product only answers definitional questions with short paragraphs, Markdown remains cheaper. The event architecture pays for itself when answers are interfaces, when the same widgets appear outside chat, and when the team refuses to let the model invent component trees. Those three conditions usually arrive together once an internal tools product matures.

Zero new frontend dependencies was a deliberate constraint, not a purity contest. Adding another chart library for chat alone would fork the design system. Reusing the existing chart and table packages forced the assistant to speak the product’s visual dialect. The cost was occasional adapter code; the benefit was one look-and-feel and one accessibility baseline.

Local LLM support matters for privacy-sensitive deployments. Because the UI only understands events, swapping a cloud model for a local runtime is a backend concern. The registry, validators, and CustomEvent bus stay identical. That separation is what lets the same frontend demo against Ollama in development and a hosted model in production without forking the chat shell.

Extending the system is intentionally boring. Add a widget file, register it with a schema, map a tool result into props, and write a test. No renderer changes. No new event types unless the product truly needs a fourth kind—and so far three have been enough. Boredom here is a feature: it means juniors can contribute widgets without learning a bespoke AI UI framework.

The end state feels less like a chatbot glued onto a product and more like a conversational router into the product’s existing capabilities. That is the architectural win. The model becomes a dispatcher. The design system remains the source of truth for presentation. The backend remains the source of truth for data and permissions.

Operationally, keep a changelog of widget registrations beside tool catalog changes. When a tool’s response shape shifts, the matching widget schema must shift in the same pull request. That pairing prevents the classic failure where the agent still answers correctly in text while the widget silently renders empty columns. Reviewers should reject tool PRs that forget the schema companion.

Accessibility follows from reusing production widgets. If the members table already supports keyboard navigation and screen-reader labels, the chat-mounted table inherits that work. Generating ad-hoc HTML from a model would throw that investment away. The event architecture is therefore not only a reliability choice; it is an accessibility preservation strategy.

Key Takeaways

• Prefer event streams (text, tool, widget) over model-authored HTML/JSX.
• Keep a single renderer and grow a widget registry.
• Validate widget props at the boundary.
• Decouple widgets from chat via CustomEvents.
• Drive behavior from a tool catalog and feature flags.
• Reuse the existing design system instead of adding a parallel UI stack.

Taken together, the event stream, registry, validators, and CustomEvent bus form a small but complete framework for AI-assisted product UI without surrendering design control to the model. Keep iterating on widgets, not on one-off markup prompts, and the assistant stays maintainable as the product grows.