Skip to content

CloudWatch Monitoring

Overview

In this lab, you'll create a CloudWatch alarm that monitors agent activity by tracking MCP Server API calls. When your agent makes too many API calls in a short window (more than 50 in 5 minutes), the alarm fires — giving you visibility into unusual or runaway agent behavior.

What You'll Learn

  • How to monitor agent activity using CloudWatch metrics and CloudTrail events
  • How to create an alarm that detects high-frequency agent API calls
  • How monitoring complements rules files and IAM policies as a defense-in-depth strategy

Instructions

Explore

Try to accomplish this goal using your agent. Here are some hints if you get stuck:

  1. Create a CloudWatch metric filter on CloudTrail logs that counts events where additionalEventData contains "CalledViaAWSMCP"
  2. Create a CloudWatch alarm on that metric that triggers when more than 50 API calls occur in a 5-minute period
  3. The alarm name should include "agent-activity" so the dashboard validation can find it
Step-by-step Walkthrough
  1. First, ask your agent to create a CloudWatch metric filter that tracks MCP Server API calls:

Create a CloudWatch metric filter on the CloudTrail log group that counts API calls made through the MCP Server. Filter for events where additionalEventData contains "CalledViaAWSMCP". Use the metric namespace "AgentToolkit" and metric name "MCPApiCallCount".

  1. Next, create a CloudWatch alarm on that metric:

Create a CloudWatch alarm called "agent-activity-high-rate" that fires when the MCPApiCallCount metric in the AgentToolkit namespace exceeds 50 within a 5-minute period. Use the Sum statistic and set the alarm action to the workshop SNS topic.

  1. The alarm configuration should be:
  2. Metric namespace: AgentToolkit
  3. Metric name: MCPApiCallCount
  4. Statistic: Sum
  5. Period: 300 seconds (5 minutes)
  6. Threshold: Greater than 50

  7. Evaluation periods: 1

  8. Verify the alarm was created:

Show me the CloudWatch alarms in my account that have "agent-activity" in the name. Include the alarm configuration details.

  1. (Optional) Test the alarm by generating activity. Ask your agent to perform multiple API calls rapidly:

List all S3 buckets, then list all Lambda functions, then list all DynamoDB tables, then describe all EC2 instances, then list all IAM roles. Repeat this sequence 10 times.

After a few minutes, check if the alarm transitioned to ALARM state.

Validation

Open the CloudWatch Dashboard in the AWS Console. The Module 4 widget checks:

  • ✅ A CloudWatch alarm exists with a name containing "agent-activity" or a metric namespace related to agent monitoring

You can verify by asking your agent:

List my CloudWatch alarms and check if any are monitoring agent or MCP activity.

Agent-Specific Tips

Claude Code can create the metric filter and alarm in a single conversation. If the CloudTrail log group name isn't obvious, ask Claude to find it first:

Find the CloudTrail log group in my account, then create a metric filter on it for MCP Server calls, and finally create an alarm on that metric.

Kiro can chain the metric filter and alarm creation. Ask it to verify the CloudTrail log group exists before creating the filter:

First check which CloudTrail log groups exist, then create the metric filter and alarm for MCP API call monitoring.

Cursor may try to write a CloudFormation template instead of making direct API calls. If you want direct creation:

Use the MCP Server to create the CloudWatch metric filter and alarm directly via PutMetricFilter and PutMetricAlarm API calls.

:::alert{type="info"} If Cursor creates a CloudFormation template instead, that's also valid — deploy it with your agent to create the resources.

:::

Codex can handle the CloudWatch API calls through MCP. Be specific about the metric namespace and alarm name for validation:

Create a CloudWatch metric filter with namespace "AgentToolkit" and metric name "MCPApiCallCount", then create an alarm named "agent-activity-high-rate" that fires when the count exceeds 50 in 5 minutes.