Second Brain: Turning Meeting Transcripts into a Queryable Knowledge Graph

The Problem

Nearly every organization depends on meetings to move work forward — planning discussions, calls with clients, sprint retrospectives, workshops for discovery. Each of these generates a steady flow of decisions, action items, risks, and commitments. And almost without exception, that information simply evaporates afterward.

The transcript gets dropped into a shared drive somewhere. Action items sit in someone's notebook for a while, then fade from memory. Weeks later, someone asks what was actually decided about a particular approach, and nobody can give a clear answer.

This isn't really a storage issue — the data exists somewhere. The real problem is that meeting content is unstructured, scattered across tools, and effectively invisible to any kind of search.

What was needed was a system capable of ingesting transcripts at scale, pulling out structured knowledge automatically, and letting people query that knowledge using plain English — all while building a living map of relationships that grows richer with every meeting added. This system was named Second Brain.

The Vision: A Living Knowledge Graph Per Project

The underlying idea is straightforward. Each time a meeting transcript is uploaded, the system parses it to determine who talked about which project, what decisions got made, which risks surfaced, and who is responsible for which follow-up action. All of this gets written into Azure Cosmos DB. From there, you can ask a question in natural language and receive a bulleted, cited answer.

As more transcripts accumulate, a knowledge graph takes shape naturally — an evolving network connecting people, decisions, actions, and risks across every recorded session.

The Solution Interface

The system exposes an interactive UI that lets users upload transcripts, browse extracted entities, ask natural-language questions, and explore the resulting knowledge graph visually.

A Closer Look at the Components

1. Extract_Entities_Tool — Parallelized Extraction

This tool does the heavy lifting. Given a raw text string, it breaks the transcript into chunks grouped by paragraph, runs entity extraction on each chunk concurrently using a ThreadPoolExecutor, and then merges the results across chunks by scoring every distinct value according to how confident the extraction was and how often it appeared.

Seven entity types are defined using a pipe-delimited format that the language model can follow consistently.

As a result, the model produces output like "Review architecture | Owner: Archana | Due: Next sprint" — a string that the application can then parse into structured {task, owner, due} dictionaries, ready for detailed rendering and for building edges in the graph.

Autonomous Discovery of Domain Terms

One important design choice was dropping the domain_context input parameter altogether and instead letting the model discover domain-specific vocabulary on its own. Every prompt used for chunk extraction includes a dedicated section that asks the model to identify domain terms, abbreviations, and system names it encounters in the text.

2. Text2SQL_CosmosDB_Tool — Recall in Natural Language

This component accepts a plain-English question along with a hint describing the Cosmos schema, converts that into a Cosmos SQL query, executes it, and produces a cited answer from the results. The tricky part is that Cosmos DB's NoSQL SQL dialect doesn't let you call CONTAINS() directly on array fields. The workaround was to give the tool an explicit schema hint describing how arrays should be queried instead.

3. Cosmos DB as the Brain Itself

This database sits at the core of the whole system. It isn't functioning as a cache or a log store — Cosmos DB effectively is the Second Brain, the persistent memory layer where every extracted piece of knowledge is stored, accumulated, and made queryable.

Each meeting-extraction document stores every entity twice: once as a flat array of strings, which supports SQL-based querying, and once as an array of structured objects, which supports detailed UI rendering and derivation of graph edges. This duplication adds a modest amount of extra storage per document, but it avoids having to parse anything at the application layer after the data comes back from the database.

The Brain Analogy — Two Modes of Memory

Human memory operates in two distinct modes: episodic memory, which captures what actually happened during an event, and semantic memory, which captures general facts and definitions — what an abbreviation stands for, who a particular person is. The Cosmos data model was deliberately built to mirror this split, storing meeting-specific episodic records separately from the accumulating semantic glossary of people, terms, and systems.

Key Engineering Decisions

Several deliberate choices shaped how the system behaves, from the decision to drop manual domain configuration in favor of automatic discovery, to storing entities redundantly for both SQL and UI needs, to keeping the LLM's role narrowly scoped to extraction and query generation rather than letting it own formatting or business rules.

Lessons Learned

1. Streamlit's rerun behavior demands deliberate state handling. Every button click causes the entire script to execute again from the top. Anything that must survive across interactions needs to be written into st.session_state before the button that triggers the rerun is drawn, not afterward.
2. Cosmos DB's SQL dialect isn't standard ANSI SQL. The CONTAINS() function only works against strings, not arrays, so the model needs explicit direction in the schema hint — including at least one example of the wrong way to write the query.
3. Language models can't be trusted to always follow output format instructions. Even with clear synthesis instructions, the model occasionally returns nothing but a brief introductory sentence instead of the full answer. Because of this, a deterministic fallback that builds a response directly from the raw retrieved data isn't optional polish — it's a required safety net.
4. Tools should have narrow, well-defined responsibilities. It's tempting to fold formatting logic, business rules, or domain knowledge directly into a tool, but that temptation should be resisted — a tool should be responsible for exactly one job.
5. Rendering Pyvis graphs inside Streamlit requires writing to a temporary file rather than passing an in-memory string. Use tempfile.NamedTemporaryFile and remember to clean it up afterward. It's also worth noting that st.components.v1.html is scheduled for deprecation starting 2026-06-01, so st.iframe should be used going forward.

The Core Design Insight

The real intelligence in a system like this doesn't come from the language model — it comes from the memory store behind it.

The model itself has no memory at all; it's stateless by nature. It reads a chunk of transcript and produces a set of entities. It reads a question and produces a SQL query. Nothing persists between calls — each invocation starts completely fresh.

Cosmos DB is where the actual memory resides. The instant something is saved to the brain, what was a temporary extraction from the model turns into a durable, structured, searchable record. The glossary of domain terms keeps growing. The web of relationships expands. Knowledge builds on itself over time.

The whole system runs on infrastructure that was already in place — Azure VMs, Azure Functions, Azure Cosmos DB, and Azure OpenAI — coordinated through the AGF Hub. No new services were introduced, and no elaborate pipeline was necessary. What made it work was a carefully designed memory store, clear boundaries around what each tool is responsible for, and a language model whose job is simply to read meetings so people don't have to.