Home / Articles

Four Memory Strategies for AI Agents: Rolling Windows, FAISS and GraphRAG

Compare ways to give an LLM agent memory, from full history and rolling windows to inverted indexes, FAISS semantic search and a GraphRAG knowledge graph.

Publisher: REACTAPP.TOP7 min read1497 words

This article is published in English.

Sources: Building Intelligent Context Memory for AI Agents

An agent built from just an LLM and a few tools works, after a fashion. What makes it dependable and personal is memory: a store of earlier exchanges and context that lets the agent answer more accurately as time passes. Memory also lets context carry across interactions, including when several agents cooperate on one task, so the system keeps hold of what the user actually wants.

Given the relevant background plus the current input, an agent can choose among a few moves: call a tool, ask the user a clarifying question or reply outright. Memory is what turns it from a purely reactive responder into an assistant that stays consistent.

Below are four memory strategies, ordered from the simplest to the most capable: sending the full history, a rolling window, similarity search over past messages, and a knowledge graph.

Option 1: send the entire history

Current foundation models accept very large context windows. The GPT-5.4 family, for instance, goes up to 1.05 million tokens, enough for whole codebases or big document sets. That makes a lazy approach tempting: append every earlier message, possibly from several sessions, to each new request and let attention sort it out.

messages: Annotated[List[BaseMessage], add_messages]

It's easy to build and fine for short-lived tasks. As conversations lengthen, though, problems pile up:

  • Too much irrelevant context can overwhelm the model.
  • Attention is opaque, so outputs may turn inconsistent.
  • Engineers struggle to explain why near-identical inputs give different replies.
  • Token usage grows with each turn, and so does cost.
  • Bigger contexts need more processing, so latency climbs.

Per-query cost and delay rise steadily as the history grows, and in production agent systems the approach becomes wasteful and hard to manage.

Option 2: a rolling context window

The alternative caps how many recent interactions get sent. This is a rolling context window, and it can be sized by message count or by tokens.

Including only the latest N interactions gives developers control over token spend, lower latency and predictable bills. It suits tasks driven mostly by recent turns. When an agent needs older or long-term facts, though, a truncated window leads to partial understanding and wrong outputs.

WINDOW_SIZE = 6
#Added a wrapper method for return window messages
def rolling_add_messages(old, new):
  combined = add_messages(old, new)
  return combined[-WINDOW_SIZE:]

class AgentState(TypedDict):
  messages: Annotated[List[BaseMessage], rolling_add_messages]

Option 3: retrieving by words, then by meaning

An inverted index is a data structure built for very fast full-text search. It maps words or numbers straight to the places they occur in one or more documents.

Applying it to chat history means indexing past messages, then pulling those that best match the words of a new query. The model gets only the matching pieces, which cuts the context sharply while keeping relevant material, even from the early part of a conversation.

The weakness is literal matching. A query can use different words for the same idea, which leaves context out and invites hallucinations, and similar questions may end up with quite different answers. A basic index can be built and queried like this:

def build_inverted_index(messages):
    index = defaultdict(set)

    for i, msg in enumerate(messages):
        words = msg.lower().split()
        for word in words:
            index[word].add(i)
    return index

def search_context(query, index, messages):
    query_words = query.lower().split()

    matched_ids = set()

    for word in query_words:
        if word in index:
            matched_ids.update(index[word])

    return [messages[i] for i in matched_ids]

Semantic search removes that weakness by interpreting intent and meaning and not just keywords. It cuts hallucination and makes answers more reliable. The pipeline runs like this. A sentence splits into tokens, and a lightweight embedding model turns them into vectors. Each token vector is adjusted using its neighbors, and the token vectors are pooled into one vector for the full sentence. These vectors live in a vector database.

A query goes through the same conversion and is compared with the stored vectors by cosine similarity, which is the cosine of the angle between two vectors. A score near 0.9 means very similar meaning, and below 0.3 usually means unrelated. Applications can set their own threshold. The best matches are then handed to the foundation model.

The sample code pairs a small pretrained Sentence Transformers model with FAISS, an in-memory vector database, to retrieve context by meaning.

from sentence_transformers import SentenceTransformer
import faiss
import numpy as np

# Sample data
sentences = [
    "schedule meeting tomorrow",
    "send project report",
    "prepare meeting agenda"
]

# Load embedding model
model = SentenceTransformer("all-MiniLM-L6-v2")

# Convert sentences into vectors
embeddings = model.encode(sentences)

# Create FAISS index
index = faiss.IndexFlatL2(embeddings.shape[1])
index.add(np.array(embeddings))

# Search
query = "plan meeting"
query_vector = model.encode([query])

distances, indices = index.search(np.array(query_vector), k=2)

print([sentences[i] for i in indices[0]])

Without such retrieval, each query starts from nothing and ignores what earlier exchanges taught, giving a thin, impersonal experience. With it, the system searches past interactions for semantically close context whenever a query arrives. The context window gets split so there's room for both the latest conversation and the retrieved history. The agent can then respond in line with user preferences and earlier exchanges, which feels more consistent and familiar. In effect, previously acquired knowledge serves as experience that improves the current answer.

Option 4: a knowledge graph with GraphRAG

Finding the right context gets hard when knowledge is scattered across interactions or documents. Pure semantic search may capture only part of what's needed, and it can skip details that differ in wording yet connect logically.

Take the question "Why was the payment delayed?" Semantic search may surface messages about a payment delay or a transaction pending, and still miss "approval from finance team is pending" or "compliance verification not completed". Both explain the delay, but their wording shares little with the question, so similarity alone won't find them.

GraphRAG helps here. It models information as a graph where each document or concept is a node and relationships are edges, which supports more structured and connected retrieval. The cost is upkeep: a graph that must change at runtime as new facts arrive adds heavy computation and design overhead.

Building a knowledge graph generally goes through four steps:

  1. Identify the nodes, meaning entities or concepts.
  2. Identify the relationships between them.
  3. Define an ontology, the structure and schema of the graph.
  4. Populate the graph with data.

The resulting structure returns richer, better-linked context than similarity alone, with fewer missed connections.

Microsoft's GraphRAG library can detect entities and relationships in content automatically and assemble the graph.

Step 1, set up the project. Install GraphRAG and initialize a working directory.

pip install graphrag
graphrag init --root ./graphrag_demo

The command generates a folder with input/, settings.yaml and prompts/. Source documents go into input/.

Step 2, build the graph. The indexing command extracts entities, finds relationships and constructs the graph.

graphrag index --root ./graphrag_demo

Step 3, query it. The generated graph can then be queried.

graphrag query \
 — root ./graphrag_demo \
 — method global \
 “Why was the payment delayed?”

The system gathers connected context across the graph, which allows multi-hop reasoning instead of leaning only on semantic similarity.

How the pieces fit

Accurate, personalized agents need more than keyword matching or isolated similarity. An inverted index trims context efficiently, and semantic search keeps intent intact, yet both can overlook relationships spread over several pieces of knowledge. GraphRAG fills that gap by organizing information as linked entities and relationships, so reasoning can hop across context rather than collecting similar-looking fragments. That shifts agents from surface retrieval toward a grasp of how concepts relate.

Together, rolling windows, semantic search, experience memory and graph retrieval produce agents that recall relevant details and reason over them, giving fuller answers, fewer hallucinations and a steadier, more personal experience. Agent memory is heading from storing information toward connecting knowledge, and that move lets AI assistants grow from context-aware to context-reasoning.