Bedrock Agents: Memory Summarization, Versioned Guardrails and Knowledge Base

Failures that make no noise

Plenty can go wrong in an AI agent system, and the worst failures raise no error and trigger no alert. Quality just erodes. Session memory piles up noise in place of signal, safety rules live somewhere outside version control, and the agent calls tools for content a knowledge base could serve directly.

Three pieces address those problems on Amazon Bedrock: a MEMORY_SUMMARIZATION override that keeps meaningful user signals across sessions, a Bedrock Guardrail versioned in Terraform, and an optional Knowledge Base backed by S3.

Prompt engineering can't carry the whole load. Real systems need controls in three places:

1. Session memory quality. Summaries have to retain the right user signals.
2. Safety governance. Guardrails should be explicit and versioned.
3. Retrieval boundary. The agent needs a clear rule for when to use tools, the Knowledge Base, or both.

Without them, a system tends to look stable in demos and then degrade once real, varied traffic arrives.

How session memory works

Memory summarization keeps a user's history available from one session to the next, so old messages needn't be re-extracted and reprocessed live. On Classic Agents, Bedrock provides a basic form via enabled_memory_types = ["SESSION_SUMMARY"] on aws_bedrockagent_agent, and GetAgentMemory in Agent Runtime reads what's stored.

With memory enabled, the work happens asynchronously after a session closes. Bedrock summarizes the finished conversation, stores the summary and injects it into later sessions that share the same memoryId, all without holding up the user's reply.

Bedrock ships a default prompt that defines how the summarization behaves:

{
  "messages": [
    {
      "role" : "user",
      "content" : "We will be given a conversation between a user and an AI
        assistant.
        When available, in order to have more context, we will also be given
        summaries we previously generated.
        Our goal is to summarize the input conversation.
        When we generate summaries we ALWAYS follow the below guidelines:
        <guidelines>
          - Each summary MUST be formatted in XML format.
          - Each summary must contain at least the following topics: 'user
            goals', 'assistant actions'.
          - Each summary, whenever applicable, MUST cover every topic and be
            placed between <topic name='$TOPIC_NAME'></topic>.
          - We ALWAYS output all applicable topics within <summary></summary>
          - If nothing about a topic is mentioned, DO NOT produce a summary
            for that topic.
          - We summarize in <topic name='user goals'></topic> ONLY what is
            related to the user, e.g., user goals.
          - We summarize in <topic name='assistant actions'></topic> ONLY what
            is related to the assistant, e.g., assistant actions.
          - NEVER start with phrases like 'Here's the summary...', provide
            directly the summary in the format described below.
        </guidelines>The XML format of each summary is as follows:
        <summary>
          <topic name='$TOPIC_NAME'>
            ...
          </topic>
          ...
        </summary>
        Here is the list of summaries we previously generated.
        <previous_summaries>
          $past_conversation_summary$
        </previous_summaries>
        And here is the current conversation session between a user and an AI
        assistant:
        <conversation>
          $conversation$
        </conversation>
        Please summarize the input conversation following the above guidelines
        plus the below additional guidelines:
        <additional_guidelines>
          - ALWAYS strictly follow the above XML schema and ALWAYS generate
          well-formatted XML.
          - NEVER forget any detail from the input conversation.
          - We also ALWAYS follow the below special guidelines for some of the topics.
          <special_guidelines>
            <user_goals>
              - We ALWAYS report in <topic name='user goals'></topic> all details the user
              provided in formulating their request.
            </user_goals>
            <assistant_actions>
              - We ALWAYS report in <topic name='assistant actions'></topic> all details
              about actions taken by the assistant, e.g., parameters used to invoke actions.
            </assistant_actions>
          </special_guidelines>
        </additional_guidelines>"
        }
    ]
}

Its instructions mostly sit in the <guidelines/> section, which tells the agent to return its result inside a <summary/> tag. The summary is split by topic. Anything about user goals goes under a topic named user goals, the agent's own actions go under assistant actions, and every topic uses a <topic name='$TOPIC_NAME'></topic> wrapper. The expected output looks like this:

<summary>
  <topic name='$TOPIC_NAME'>
    ...
  </topic>
  ...
</summary>

Bedrock fills <previous_summaries/> with earlier summaries through the $past_conversation_summary$ variable, and the live conversation arrives inside <conversation>$conversationlt;/conversation>. Two more sections, <additional_guidelines/> and <special_guidelines/>, let you add detail to the summarization strategy.

The lifecycle goes like this:

1. Memory is turned on in the agent configuration with memory_configuration and enabled_memory_types = ["SESSION_SUMMARY"].
2. The caller invokes the agent with a stable memoryId for each user.
3. The session ends, either by endSession = true or by timeout.
4. Bedrock generates the summary in the background.
5. In later sessions with the same memoryId, Bedrock places memory context into orchestration placeholders such as $memory_content$.

Two consequences follow. Summaries aren't instant and can show up some time after a session closes. And if the memoryId changes from request to request, continuity is gone.

Warning: Unstable memoryId values, such as connection IDs where user IDs belong, break memory continuity. Use an identifier that stays fixed for each user or conversation entity.

Making summaries useful

The default summary is a generic recap. That's fine for a broad assistant but often too noisy for a specialized one. For a small news assistant, a strong summary keeps hold of:

• recurring topics such as AI news, climate or geopolitics;
• favored regions and entities;
• stories a user follows across sessions;
• explicit dislikes, like "do not include crypto."

Without that structure, memory grows while relevance falls.

Enabling it starts with the agent resource in Terraform:

resource "aws_bedrockagent_agent" "agent" {
  # ... existing code above ...
  memory_configuration {
    enabled_memory_types = ["SESSION_SUMMARY"]
    storage_days = 30
    session_summary_configuration {
      max_recent_sessions = 5
    }
  }
  # ... existing code below ...
}

Bedrock agents accept only the SESSION_SUMMARY type here. AgentCore goes further, with summaries, user preferences, semantic facts and custom memory strategies. storage_days sets how long summaries are kept, and max_recent_sessions controls how many recent sessions get summarized.

Every invocation must carry a memoryId, here the user_id, so conversations link to the same entity:

# Invoke with stable memoryId and explicit session end
resp = bedrock_agent_runtime.invoke_agent(
  agentId=agent_id,
  agentAliasId=alias_id,
  sessionId=conversation_id,
  memoryId=user_id,     # stable per user
  inputText=message_text,
  endSession=is_last_turn
)

Retrieval uses the same memoryId:

# Retrieve summaries later (classic Agents API)
memory = bedrock_agent_runtime.get_agent_memory(
  agentId=agent_id,
  agentAliasId=alias_id,
  memoryId=user_id,
  memoryType="SESSION_SUMMARY"
)

Overriding the summarization prompt

With memory on, Bedrock lets a prompt override replace the MEMORY_SUMMARIZATION stage: prompt_type = "MEMORY_SUMMARIZATION" with prompt_creation_mode = "OVERRIDDEN". Classic agents use two dedicated placeholders in it: $past_conversation_summary$ for stored summaries tied to the same memory identity, and $conversation$ for the current session text. In orchestration templates, memory context comes through $memory_guideline$ for strategy, $memory_content$ for persistence and $memory_action_guideline$ for system prompts.

A tailored version of the default prompt spells out what the agent should capture and what it should ignore, so only helpful details survive:

prompt_override_configuration {
  prompt_configurations = [{
    prompt_type = "MEMORY_SUMMARIZATION"
    prompt_state = "ENABLED"
    prompt_creation_mode = "OVERRIDDEN"
    parser_mode = "DEFAULT"
    base_prompt_template = <<-JSON
{
  "system": "
  We are summarizing a news research conversation for future sessions.
  Capture:
    - recurring topics,
    - region focus,
    - entities and stories the user tracks,
    - explicit dislikes.
  Ignore:
    - greetings and small talk,
    - implementation questions about the assistant itself.
  Past summaries:
  <previous_summaries>
    $past_conversation_summary$
  </previous_summaries>
  Current session:
  <conversation>
    $conversation$
  </conversation>
  Return concise bullet points focused on user signal.",
  "messages": [
    {
      "role": "user",
      "content": [{ "text": "Generate a focused memory summary for personalization." }]
    }
  ]
}
JSON
  inference_configuration = [{
    temperature = 0.2
    top_k = 64
    top_p = 0.9
    max_length = 512
    stop_sequences = []
  }]
  # ... existing code below ...
}
]
}

Override the stage when a generic recap doesn't improve later answers, when summaries need a high signal-to-noise ratio, or when the memory token budget has to stay tight and predictable. If the assistant is generic and doesn't rely on personalization, the default is fine.

Typical mistakes include:

• unstable memoryId values, again, like connection IDs standing in for user IDs;
• sessions that never end, so no summary is ever produced;
• verbose summary prompts that rebuild what is practically a full transcript;
• mixing summary goals with formatting goals meant for post-processing.

Guardrails: policy that prompts can't enforce

Users sometimes are rude, misuse the agent, or ask for internal details, secrets and other people's personal data. Guardrails handle that. The principle is short: prompts guide behavior, guardrails enforce policy. Use prompts for decisions and formatting, and guardrails for content-safety limits on both user input and assistant output.

Tip: Try guardrails in DETECT mode before BLOCK. Auditing what would have been stopped leaves testers unaffected.

Bedrock includes several guardrail capabilities that need no extra setup beyond adding them to the list:

• Content filters cover hate, insults, violence, sexual content, misconduct and prompt attacks, with input and output strengths and actions. They suit always-on baseline safety for ordinary user traffic.
• Denied topics block off-limits subject areas, useful for product or regulatory boundaries such as "never discuss this domain."
• Word filters take managed and custom blocked terms, handy for exact business terms, profanity or hard-block words.
• Sensitive information filters match PII entities and custom regex, covering privacy controls for account identifiers, emails and phone numbers.
• Contextual grounding checks enforce grounded-answer policies in retrieval flows, which matters in RAG and Knowledge Base setups where hallucination control counts.
• Prompt attacks sit inside content filters as a category (jailbreaks, injections, prompt leakage) on the Standard tier.
• Automated reasoning checks validate consistency with policy rules, for strict logic in regulated responses.

Most teams do best rolling these out in phases: content and sensitive-information filters first, topic and word filters next, then grounding and reasoning once retrieval policies mature.

Order of evaluation

Guardrails examine both directions:

1. Input. The prompt from the user gets evaluated first, with all configured policies running in parallel. A block makes Bedrock return your blocked_input_messaging and skip inference.
2. Output. When input passes, the model answers and its output is checked. A block swaps the response for blocked_outputs_messaging, or masks it where sensitive filters are configured.
3. If both pass, the output goes back untouched.

The order matters for cost. A blocked input costs only the guardrail evaluation and not model inference.

Each content filter has two independent settings. Strength (inputStrength and outputStrength) takes NONE, LOW, MEDIUM or HIGH and controls how aggressively violations are detected. Action (inputAction and outputAction) is either BLOCK, which blocks the content and substitutes the blocked messaging, or NONE, which does nothing but records detections in trace responses, effectively a detect-only mode.

A sensible default strategy:

• begin with MEDIUM on high-risk categories such as hate, violence and misconduct, for input and output alike;
• in testing or staging, start with outputAction = "NONE" so false positives can be counted;
• switch to BLOCK only after the traces and test-set results look acceptable;
• keep low-risk categories at LOW or MEDIUM, and don't set everything to HIGH on day one.

Content filters also come in two safeguard tiers. CLASSIC is baseline filtering, while STANDARD brings stronger protection with code-domain analysis and prompt-attack category support. An assistant that handles developer-style prompts, snippets or mixed text and code is usually safer on Standard.

Versioning

Guardrail releases should be explicit and immutable, and every policy change should become a new version. aws_bedrock_guardrail_version supports this: create a version and attach it to aws_bedrock_guardrail through guardrail_configuration.

resource "aws_bedrock_guardrail" "guardrail" {
  name = "${var.environment}-news-guardrail"
  description = "Safety policies for news assistant"
  blocked_input_messaging = "Your request violates our usage policy."
  blocked_outputs_messaging = "This response was blocked by our safety policy."
  content_policy_config {
    filters_config {
      type = "HATE"
      input_strength = "MEDIUM"
      output_strength = "MEDIUM"
    }
    filters_config {
      type = "VIOLENCE"
      input_strength = "MEDIUM"
      output_strength = "MEDIUM"
    }
    filters_config {
      type = "MISCONDUCT"
      input_strength = "MEDIUM"
      output_strength = "MEDIUM"
    }
  }
}

resource "aws_bedrock_guardrail_version" "guardrail_version" {
  guardrail_arn = aws_bedrock_guardrail.guardrail.guardrail_arn
  description = "release-${substr(sha256(var.guardrail_policy_fingerprint), 0, 12)}"
  skip_destroy = true
}

resource "aws_bedrockagent_agent" "news_agent" {
  # ... instruction, model, prompt overrides ...
  guardrail_configuration {
    guardrail_identifier = aws_bedrock_guardrail.guardrail.guardrail_id
    guardrail_version = aws_bedrock_guardrail_version.guardrail_version.version
  }
  # ... existing code below ...
}

The split allows a clean rollback. Prompts stay as they are, and only the guardrail version moves when policy tuning is needed.

Knowledge Base for internal content

Suppose the agent should answer from company policies or knowledge gathered over time, and avoid unnecessary tool calls. Bedrock uses an attached knowledge base to generate answers from retrieved documents, and it's on by default once one is configured. It lets agents connect securely to private company data so foundation models give more relevant, accurate and tailored answers through Retrieval Augmented Generation (RAG).

It fits when the source of truth is internal or curated documentation, when minute-level freshness isn't needed, or when grounded answers with citation-like traceability are required.

Note: Real-time, externally indexed information is better served by API tools.

Bedrock Knowledge Base works as a managed RAG pipeline over your own content. Rather than building retrieval infrastructure yourself, you let Bedrock take in data from a connector, parse and chunk the documents, create embeddings, keep vectors in a supported store and fetch the relevant chunks when a request arrives. During KNOWLEDGE_BASE_RESPONSE_GENERATION, those chunks feed grounded responses.

Two flows are involved. Ingestion runs offline or on sync: documents are parsed, split into chunks according to the chunking strategy, converted to vectors by an embedding model and written to a vector index with metadata mappings. Runtime retrieval runs per request: the query is embedded, the vector store returns semantically relevant chunks, Bedrock adds them to the model's context, and the generation stage produces grounded output, or plain retrieval results in retrieve-only patterns.

The split has operational meaning, because poor retrieval quality is often an ingestion or chunking issue, not a prompt issue.

Nothing is "inserted as a row" like in a SQL cache. You add or update files, or connector-backed sources, and then run sync.

Warning: Sync isn't automatic. After adding documents to S3, trigger it explicitly before the agent can retrieve them.

Runtime controls

• numberOfResults sets how many source chunks come back (default up to 5, range 1 to 100).
• overrideSearchType picks SEMANTIC for vector-only search or HYBRID for vector plus raw-text search, supported only on compatible stores and indices. Unset, Bedrock chooses.
• Metadata filter and logical groups (andAll, orAll) allow precise retrieval.
• Optional reranking improves the final ordering of chunks.

With hierarchical chunking, the number of results can be lower than requested, since child chunks may be replaced by their parents at response time.

Chunking

Chunking strongly affects retrieval quality. Bedrock offers default chunking of about 300 tokens along sentence boundaries, fixed-size chunking with explicit token size and overlap, hierarchical parent and child chunks for precision plus broader context, semantic chunking with model-driven boundaries, and no chunking, which treats each document as one chunk and mostly suits pre-segmented content.

A reasonable path: begin with default or fixed chunking, since it's simple. Switch to hierarchical once answers need more surrounding context, and to semantic if precision stays poor after prompt tuning. Compare against a retrieval eval set before and after every change.

Vector stores

Knowledge Bases work with several vector backends, such as OpenSearch Serverless or a managed cluster, Aurora PostgreSQL-compatible, Pinecone, Redis Enterprise Cloud, MongoDB Atlas and Neptune Analytics, depending on provider support at deploy time. The choice affects whether HYBRID search exists, which metadata filters are possible, and the operational complexity and cost. OpenSearch Serverless is usually the quickest start. A Terraform wiring example follows:

resource "aws_bedrockagent_knowledge_base" "news_kb" {
  name = "${var.environment}-news-kb"
  role_arn = aws_iam_role.bedrock_execution_role.arn
  knowledge_base_configuration {
    type = "VECTOR"
    vector_knowledge_base_configuration {
      embedding_model_arn = "arn:aws:bedrock:${data.aws_region.current.name}::foundation-model/amazon.titan-embed-text-v2:0"
    }
  }
  storage_configuration {
    type = "OPENSEARCH_SERVERLESS"
    opensearch_serverless_configuration {
      collection_arn = aws_opensearchserverless_collection.kb.arn
      vector_index_name = "news-kb-index"
      field_mapping {
        vector_field = "vector"
        text_field = "text"
        metadata_field = "metadata"
      }
    }
  }
  # ... existing code below ...
}

resource "aws_bedrockagent_data_source" "news_docs" {
  knowledge_base_id = aws_bedrockagent_knowledge_base.news_kb.id
  name = "${var.environment}-news-docs"
  data_source_configuration {
    type = "S3"
    s3_configuration {
     bucket_arn = aws_s3_bucket.news_docs.arn
    }
  }
  # ... existing code below ...
}

resource "aws_bedrockagent_agent_knowledge_base_association" "news_agent_kb" {
  agent_id = aws_bedrockagent_agent.agent[0].agent_id
  agent_version = "DRAFT"
  knowledge_base_id = aws_bedrockagent_knowledge_base.news_kb.id
  description = "Internal news assistant knowledge base for curated context and editorial guidelines"
  knowledge_base_state = "ENABLED"
  # ... existing code below ...
}

Routing between the Knowledge Base and tools

Attaching a Knowledge Base gives the agent a second retrieval path. To avoid ambiguity, put routing rules for KB versus API tool in the orchestration prompt, keep the POST_PROCESSING contract identical whichever source answered, and say when the agent should ask clarifying questions because retrieval confidence is low.

A practical policy for this setup: use the KB for internal, static context such as policy documents, editorial standards and historical notes; use the action group for live external data; and don't call both in one turn unless the request truly needs cross-checking.

Guardrails also apply in retrieval and generation operations, which matters because grounded retrieval doesn't remove safety risk on its own. A common recommendation is to evaluate the KB path with guardrails in detect mode first, tune false positives and negatives from the traces, then move to blocking or masking in production.

Getting content in

• Initial corpus: for a one-time load of policies, FAQs and style guides, upload files to the S3 bucket wired to the data source and start a sync.
• Ongoing updates: when product or legal text changes on a schedule, a nightly job can upload new files to S3 and trigger StartIngestionJob for the data source, or rely on automatic sync if it's enabled.
• Many small documents: store one topic per file, such as kb/topics/eu-ai-act-overview.md, so chunk boundaries match topics and citations are clearer than with one giant PDF.
• Connectors: SharePoint, Confluence, web crawlers and the like fit content already living there. Bedrock pulls and syncs on a schedule, and exact options vary with connector type and region.
• Programmatic pipelines: when a CMS or ticket system exports HTML or JSON, a Lambda can convert it to plain text or markdown, write it to S3 and call StartIngestionJob so the KB picks up the new objects.

Pitfalls

• Treating the KB as a real-time feed. It's only as fresh as the ingestion sync schedule.
• Chunking too much or too little, which hurts precision.
• Sloppy metadata, which leaves filters useless.
• Packing retrieval and formatting duties into the same prompt stage.
• Believing that rewording prompts can rescue a weak retrieval index.

What this adds up to

Summaries work best when they're tuned to the domain and not left as generic recap text. Prompts and guardrails operate at different layers, so version them separately. A Knowledge Base is one retrieval choice and doesn't replace real-time APIs, which means prompts should state plainly when a tool provides freshness and when the KB provides curated context.

Together the pieces complete the agent's internal pipeline: a MEMORY_SUMMARIZATION prompt that condenses long conversations into reusable summaries, a Bedrock Guardrail that blocks harmful content categories, and a Knowledge Base on an S3 data source for retrieval-augmented generation.