AgentSRE: Composable AI Agents for Autonomous Incident Remediation on AWS

The incident loop that eats an SRE's day

Much of a site reliability engineer's day goes to one repeating cycle. An alert fires, somebody digs in, a root cause emerges, a fix goes out and a ticket gets closed. Every step takes time, and mean time to resolution (MTTR) can stretch from minutes to hours. As cloud estates grow, that manual routine turns into a bottleneck.

AgentSRE is an autonomous incident management platform on AWS aimed at that bottleneck. It carries an incident from detection through to resolution with composable AI agents, each specialized in a different kind of remediation task. The design treats remediation as a set of small cooperating agents and not as one monolithic automation script, and the reasons are laid out below.

What a typical incident looks like

Picture a CloudWatch alarm that flags high read latency on a database. The on-call engineer gets paged, opens a dashboard, checks metrics and reads slow query logs, and eventually finds that a missing index is forcing full table scans. They then write and run a CREATE INDEX statement, confirm the fix worked and update the ticket.

All of those steps are necessary, yet most are mechanical. The engineer isn't solving a new puzzle. They're following a familiar diagnostic and repair pattern, and the cost is real. Switching context, navigating dashboards and running queries by hand add up while the incident stays open longer than it should.

If the diagnostic pattern is understood and the fix is well defined, the whole cycle can be automated with a person still approving the change. AgentSRE covers the full incident lifecycle in five stages: detection, investigation, decision, execution and closure.

The distinguishing feature is how execution is organized. A single sprawling script that attempts every possible fix is replaced by specialized agents, one per category: a database agent for SQL remediation and a queue agent for messaging trouble. Every agent is its own state machine, so it can be built, tested and released separately, and new agent types slot in with no edits to existing ones. Composability is the central design principle. It reflects how a real operations team splits problems among specialists, and it lets the platform grow as new kinds of remediation come up.

Architecture at a glance

The platform runs entirely on AWS managed services, which keeps operational overhead low and scaling automatic.

In the architecture diagram, everything starts at the left with a CloudWatch alarm feeding an event-driven pipeline. SNS topics spread the notification, an alerting Lambda opens the ServiceNow incident, and the AWS DevOps Agent begins investigating by itself. What it finds appears in a React front end divided into Chat, Incidents and Reports. Along the bottom, three topology views show operators how each incident looks visually. A single figure thus covers the whole route from detection to visualization, and no stage waits for a manual handoff.

Detection

The first signal is a CloudWatch alarm for an anomaly, for example high read latency, exhausted connections or a dead letter queue that keeps growing. It publishes to an SNS topic, and that triggers the alerting Lambda, which in one pass opens a ServiceNow incident and starts the AWS DevOps Agent's investigation. This stage needs no human at all.

Investigation

The AWS DevOps Agent is a managed service that runs the investigation autonomously. It looks at CloudWatch metrics, RDS Performance Insights and slow query logs, and it decides for itself which data matters given the alarm. An asynchronous Lambda collects the findings and writes them to DynamoDB.

Amazon Bedrock, running Claude, then condenses the findings into a short root-cause statement, an impact assessment and a recommended mitigation. This summary sits at the top of the investigation view, with the full analysis available below it.

The screenshot shows the investigation findings view, summary first and details underneath.

Decision and governance

Safety checks run before any change is applied. Guardrails inspect each SQL statement and reject destructive ones, such as dropping a table or deleting rows without a WHERE clause. The action gets a risk level and goes to a human for sign-off. The confirmation dialog lays out the exact command, the target resource and how to roll back.

The human step is deliberate. Full autonomy is powerful, but changes in production deserve a moment of human judgment, particularly for higher-risk operations.

The dialog above is what an operator sees when asked to confirm a fix.

Routing a fix to the right agent

Once a fix is approved, the orchestrator classifies the remediation category and sends the request to the matching agent. That's where composability pays off.

The diagram follows remediation in detail. When someone approves a fix, the orchestrator takes over and does three jobs. It classifies the root cause, applies the SQL safety guards, and hands the request to the correct specialist while logging the operation in DynamoDB. From there, two branches exist. On the database side, a LangGraph state machine on Amazon Bedrock AgentCore validates the SQL with an AI analyst, runs it against the database via a gateway Lambda in the VPC, and writes the result to Jira together with an audit entry. The queue side works the same way for jobs like emptying a dead letter queue or sending messages back to where they came from.

No agent reaches a resource itself. Each calls gateway tools: a gateway function connecting to RDS for the database agent, and one doing SQS operations for the queue agent. The design keeps agents small and makes resource access controlled and traceable. After either branch finishes, a shared result pipeline stores the outcome in DynamoDB, adds a note to ServiceNow, resolves the ticket and refreshes the topology, so every incident ends consistently regardless of which agent worked on it.

How classification works

The intelligence that makes this work sits in the orchestrator. When the AWS DevOps Agent finishes, it produces a root-cause analysis in natural language, and the orchestrator reads it and assigns a remediation category before any agent runs.

Classifying happens in two steps. A structured parser first searches the analysis for a predefined remediation block that states the type of fix, the target resource and the planned action. Lacking that block, the orchestrator turns to a language model classifier on Amazon Bedrock, which reads the free-form cause together with the original alarm context and decides the category. The alarm context helps connect the symptom with the fix. High read latency points to adding a database index, exhausted connections point to ending a blocking query, and a growing dead letter queue points to a purge or redrive.

Routing is deterministic once the category is set: database goes to the database agent, queue goes to the queue agent. Keeping classification apart from execution is what makes later additions easy, since a new category just maps to a new agent and the routing logic doesn't change.

The two production agents

AgentSRE ships with two production agents, each built for one category of remediation. Both run as containerized runtimes on Amazon Bedrock AgentCore and share an internal design: a LangGraph state machine with a few well-defined nodes, each instrumented so that its execution time and outcome are recorded.

The database agent

This agent covers every SQL-based fix. Its design follows a real database operations workflow, organized as a state machine with a routing layer that picks the next step from the current state.

When a request comes in, the agent first picks up the target database and the suggested SQL from the incident. Next comes validation, in which an AI analyst running on Bedrock inspects the statement to confirm it's well formed and safe. That semantic check is added to the deterministic guards the orchestrator already applied, so a faulty or dangerous query faces a second barrier.

Once validation passes, the agent executes the statement, though not directly. It calls a gateway Lambda inside the VPC that connects to the target RDS instance with IAM authentication. Database credentials and network access therefore stay confined to a single controlled component, while the agent remains lightweight with no direct database connection.

One pipeline serves a range of remedies. The agent can create a missing index to fix read latency, terminate a blocking query that holds locks and exhausts connections, or run other corrective statements as needed. Afterward it records the outcome, including rows affected and execution time, and creates a matching Jira entry plus a full audit record. Since the same validate, execute and audit pipeline serves every SQL operation, supporting a new kind of database fix doesn't require changing the agent's structure.

The queue agent

The queue agent handles messaging problems, most often a dead letter queue that has piled up failed messages. It follows the same three-stage machine of validate, execute and audit, but with logic specific to queues.

Validation shows why safety matters most for an autonomous queue agent. Before it does anything destructive, it makes sure the target queue is in fact a dead letter queue, first by checking the ARN format and then by asking its gateway tool about the queue's attributes. A queue that isn't a dead letter queue stops the agent right there with a reported safety failure, rather than gambling on a live production queue. Wiping the wrong queue would destroy messages in transit, and that price justifies a dedicated check.

Once validation succeeds, one of two actions follows. Purging wipes all accumulated messages from the dead letter queue, which fits messages known to be unrecoverable poison. Redriving sends them back to the source queue for another attempt, with a configurable rate limit so that downstream consumers aren't flooded. A status check is available as well, to monitor a redrive in progress.

Like the database agent, the queue agent sends every operation through its own gateway function, which performs the boto3 SQS calls, and logs all actions for audit. It accepts either a structured request or plain language and pulls out the action and queue ARN from either.

Independence by design

Neither agent has any knowledge of the other. Each brings its own state machine, gateway tool and validation tuned to its field. Only the orchestrator sees both, and it does nothing except classify and route.

This independence is what composability means in practice. A third agent for restarting a container, rotating a credential or changing a scaling policy would follow the same recipe, with a LangGraph state machine going through validate, execute and audit and a gateway tool of its own. It registers under a fresh category with the orchestrator, and the older agents continue unaffected. Growth comes by adding pieces, never by editing them.

Reporting and closure

Once the work is done, the agent reports whether it succeeded, how many rows changed and how long it took. The outcome is saved to DynamoDB and added to the ServiceNow incident as a work note, and a successful run resolves the ticket without further input. All steps are logged, which yields a full audit trail running from the first alarm to the resolution.

The operations interface

Automation does the work, but operators still need somewhere to watch, steer and understand it. AgentSRE has a custom web interface, built with React and hosted on AWS Amplify, as a single pane of glass for the whole incident lifecycle. Access runs through Microsoft Entra ID single sign-on, so the organization's identity provider governs who can use operational controls. The interface has three areas, each serving a different need.

Chat

Chat is a conversational front end to the AWS DevOps Agent. Operators ask in plain language about the health of a particular database, whether any alarms are active or how an investigation is going. Typing an incident number starts a full investigation right from the chat. The interface keeps conversation context across messages and copes with slow responses by retrying in the background, so operators get an answer and not an error.

The screenshot shows the chat view.

Incidents

Incidents is the dashboard for day-to-day operations. Each incident appears with its status, whether anyone has investigated and whether a fix went in. An operator can open an investigation from there, read the root cause, look at the topology and start remediation. Fixing takes one click and then a confirmation dialog that shows the precise change before it executes.

This is the incidents dashboard.

Reports

Reports gives analytics across all incidents: investigation counts, resolution rates, MTTR and breakdowns by priority and resource. It converts single-incident data into operational insight over time.

The report view appears above. Inside the investigation view, the interface also offers topology visualization, covered below, which turns dense analysis into interactive graphs that communicate at a glance.

ServiceNow integration

An autonomous platform that ignores existing IT service management would create more problems than it solves. So AgentSRE integrates directly with ServiceNow, which means every incident it handles is reflected in the system of record that operations teams already depend on.

Information flows in both directions throughout the lifecycle. A firing CloudWatch alarm makes the alerting Lambda create a ServiceNow incident on its own, pre-filled with alarm details, priority and category, after first looking for an open incident to avoid a duplicate. So the ticket is there from the start with nobody having to file it.

As the incident moves along, AgentSRE posts updates to the ticket. Investigation findings go in as work notes, so anyone viewing the ticket sees the full root-cause analysis in context. When a fix runs, the result, covering the action taken, the outcome and a timestamp, lands as another work note. After a successful fix, the platform moves the ticket to resolved and attaches resolution notes.

What results is a complete, auditable record inside ServiceNow that mirrors the whole automated workflow. Teams keep working in the tool they know while automation does the heavy lifting, and respecting the existing process makes the platform adoptable in an enterprise and not a parallel system that people must learn separately.

This is a ServiceNow ticket after the platform has updated it.

Seeing what happened: topology views

Knowing what happened during an incident matters nearly as much as fixing it, and a picture communicates more than dense text. AgentSRE offers three topology views, each rendered as an interactive graph.

Investigation topology

This view shows the blast radius: which resources are affected and how they connect. The root-cause resource gets a distinct highlight, and animated edges trace the failure path as the problem spreads. The graph is generated dynamically from the investigation findings, so different incidents produce different topologies, since each graph reflects what the agent actually found.

The view above shows an investigation topology.

Infrastructure topology

This map shows the complete architecture: each API gateway, Lambda function, agent, database, queue and monitoring part, and the links among them. It gives operators the full deployed picture and the path requests take.

The infrastructure map shows the same system.

Composable agent topology

This view follows the remediation path step by step: the user trigger, orchestrator classification, agent invocation, gateway tool call, resource change and the final result posted to ServiceNow. Each node stands for a real component invoked during the fix, so the composable architecture stops being an abstract idea and becomes something operators can see and verify.

The agent topology shows one remediation run.

The topology is extracted with a hybrid method. A rule-based extractor builds the graph immediately and for free when the incident matches a known pattern. For anything unfamiliar, the system asks Bedrock Claude to extract the topology from the investigation text. Common cases stay fast and unusual ones still get handled.

Why composability pays off

Building remediation as composable agents and not as a monolith brings several practical benefits.

• Extensibility. A new remediation category, such as restarting a container, adjusting a scaling policy or rotating a credential, means building one agent and registering it with the orchestrator, and the existing agents stay untouched.
• Testability. A single state machine with a defined input and output can be tested in isolation, so a change in one agent is less likely to break another.
• Observability. Every agent follows a defined state machine, so its execution can be traced and visualized, and the composable agent topology view comes straight from this design.
• Reusability. One agent can handle many incidents. For instance, the database agent covers index creation, query termination and other SQL fixes unchanged.

Technology stack

The platform combines these AWS services and open-source frameworks, grouped by role:

• Monitoring and alarms come from Amazon CloudWatch, and event-driven orchestration from Amazon SNS with AWS Lambda.
• Investigation is handled by the AWS DevOps Agent, while Amazon Bedrock with Claude covers reasoning, summaries and topology extraction.
• Amazon Bedrock AgentCore hosts the agents, and LangGraph supplies their state machines.
• Amazon DynamoDB keeps state and audit data, and ServiceNow manages the ticket lifecycle.
• The front end uses React with React Flow for the interface and topology graphs, hosted on AWS Amplify.

Measured impact

Automating the cycle has a measurable payoff. By hand, an investigation plus fix may take between one and four hours, but AgentSRE gets from alarm to resolution in less than ten minutes. The investigation normally takes three to five minutes, and an approved fix runs in under thirty seconds.

Consistency improves as well as speed. All incidents get the same careful investigation, safety checks and audit trail, so the on-call roster doesn't decide how good the response is.

Where it can go next

The composable foundation opens several directions. More agents could handle infrastructure-level fixes through Systems Manager or Terraform. Approval flows could be routed through collaboration tools such as Microsoft Teams. A knowledge layer could bring in lessons from past incidents to sharpen future recommendations. And since the agent pattern isn't tied to one cloud, the same approach could extend to multi-cloud environments.

The architectural takeaway

Autonomous incident management turns out to be practical as well as feasible when it rests on the right principles: autonomous investigation, composable remediation agents, human approval and rich topology views. Together they cut mean time to resolution sharply while keeping safety and full traceability.

The key lesson concerns architecture. Splitting remediation into specialized, composable agents, instead of one automation, matches how operations teams really operate, keeps the system open to extension and makes each action visible. With cloud environments growing more intricate, that is a scalable route to AI-driven operations.