1  # CLAUDE.md
  2  
  3  This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
  4  
  5  ## Architecture
  6  
  7  Three independent CDK stacks deployed in sequence, sharing configuration via SSM Parameter Store:
  8  
  9  - **00-agent-agentcore** — Standalone AgentCore Runtime with a Strands multimodal agent + AgentCore Memory. Exports ARNs to SSM.
 10  - **01-whatsapp-end-user-messaging** — WhatsApp via AWS End User Messaging Social: SNS -> receiver Lambda -> DynamoDB (Stream + tumbling window) -> processor Lambda -> AgentCore.
 11  - **02-multichannel-api-gateway** — WhatsApp + Instagram DM via Meta Cloud API: API Gateway -> receiver Lambda (dual-channel detection) -> DynamoDB (Stream + tumbling window) -> processor Lambda -> AgentCore. Single webhook serves both platforms.
 12  
 13  Each stack has its own `app.py`, `cdk.json`, and `requirements.txt`. They are independent CDK apps, not a single multi-stack app.
 14  
 15  ## Build & Deploy Commands
 16  
 17  Each stack is deployed from its own directory:
 18  
 19  ```bash
 20  cd 00-agent-agentcore  # or 01-... or 02-...
 21  python3 -m venv .venv && source .venv/bin/activate
 22  uv pip install -r requirements.txt
 23  cdk deploy
 24  ```
 25  
 26  For the agent stack specifically, build the deployment package first:
 27  
 28  ```bash
 29  cd 00-agent-agentcore
 30  bash create_deployment_package.sh  # builds ARM64 ZIP in agent_files/
 31  cdk deploy
 32  ```
 33  
 34  For Stack 02, install Lambda layer dependencies before deploy:
 35  
 36  ```bash
 37  cd 02-multichannel-api-gateway/layers/common
 38  pip install requests -t python/
 39  cd ../..
 40  cdk deploy
 41  ```
 42  
 43  After deploying Stack 00, update the TwelveLabs API key in Secrets Manager:
 44  
 45  ```bash
 46  aws secretsmanager put-secret-value \
 47    --secret-id <TwelveLabsSecretArn from stack output> \
 48    --secret-string '{"TL_API_KEY":"your-actual-key"}'
 49  ```
 50  
 51  ## Key Design Decisions
 52  
 53  ### Message Buffering (Tumbling Window)
 54  
 55  Both Stack 01 and 02 use DynamoDB Streams with a tumbling window (20 seconds) to aggregate rapid-fire WhatsApp messages into a single agent invocation. Based on [sample-whatsapp-end-user-messaging-connect-chat](https://github.com/aws-samples/sample-whatsapp-end-user-messaging-connect-chat).
 56  
 57  - DynamoDB table PK=`from_phone`, SK=`id` ensures same-user messages land in the same shard
 58  - `tumbling_window` + `max_batching_window` on the Lambda event source mapping (configurable via `buffer_seconds` in CDK)
 59  - Processor Lambda deserializes DDB stream records, groups by sender, concatenates texts with `\n`, keeps last media
 60  
 61  ### Configurable Models
 62  
 63  Both the LLM and video analysis model are configurable via environment variables set before `cdk deploy`:
 64  
 65  - `MODEL_ID` — Claude model (default: `us.anthropic.claude-sonnet-4-20250514-v1:0`)
 66  - `TL_MODEL_NAME` — TwelveLabs model (default: `pegasus1.2`)
 67  
 68  ### AgentCore Memory IDs
 69  
 70  Two IDs drive memory — they must be different strings (enforced by the SDK):
 71  
 72  - **actor_id**: Canonical `u-user-{uuid}` from the unified users table (padded to 33 chars). Same across all channels for linked users.
 73  - **session_id**: Channel-specific — `wa-chat-{phone}` or `ig-chat-{sender_id}` (padded to 33 chars). Keeps conversation turns separate per channel.
 74  
 75  Stack 01 (SNS) uses legacy format `wa-user-{phone}` as actor_id (no unified users table).
 76  
 77  ### Unified User Identity (Stack 02)
 78  
 79  The `unified_users` DynamoDB table maps channel-specific IDs to a canonical `user_id`:
 80  - GSI `wa-phone-index` on `wa_phone` for WhatsApp lookups
 81  - GSI `ig-id-index` on `ig_id` for Instagram lookups
 82  - Table name exported to SSM: `/agentcore/unified_users_table_name`
 83  - AgentCore runtime role has read/write access
 84  
 85  The agent has a `link_account` tool that merges two user records when a user provides their other channel identity. The tool reads the table name from SSM at runtime (`ssm:GetParameter` on `/agentcore/*`). This permission is granted in `agentcore_role.py` (Stack 00) because the table is created by Stack 02 after Stack 00 deploys — SSM bridges the dependency.
 86  
 87  **Important**: When `link_account` is called from WhatsApp with an IG username, it saves `ig_username` but cannot resolve the numeric `ig_id` (only known when IG sends a webhook). The processor handles this with a fallback: if lookup by `ig_id` GSI fails, it scans by `ig_username`, finds the linked user, and backfills the `ig_id` for instant future lookups.
 88  
 89  ### Memory Retrieval Config
 90  
 91  - Namespace format: `/strategies/{memoryStrategyId}/actors/{actorId}/` (NOT `/users/{actorId}/facts`)
 92  - Strategy IDs passed as env vars: `FACTS_STRATEGY_ID`, `PREFERENCES_STRATEGY_ID` (set at `cdk deploy` time)
 93  - `top_k: 20` for facts, `top_k: 10` for preferences, `relevance_score: 0.3` for both
 94  - The agent prompt includes explicit "Fact:" lines in responses to improve long-term memory extraction quality (AgentCore summarizes aggressively and drops structured data like IDs)
 95  
 96  ### Agent Prompt Tags
 97  
 98  The processor prepends context tags to every prompt sent to AgentCore:
 99  - `[Channel: whatsapp|instagram]` — current channel
100  - `[UserID: user-xxx]` — canonical user ID (only in Stack 02)
101  - `[User: Name]` — display name from WhatsApp contact or Instagram profile
102  
103  The agent only offers cross-channel linking when `[UserID:]` is present.
104  
105  ### Multimedia Processing
106  
107  AgentCore Memory only stores text. All multimedia is converted to text before entering memory:
108  
109  - **Image**: Claude vision (inline content blocks) — text block MUST be first in content array
110  - **Audio**: Amazon Transcribe -> transcript sent as text prompt to agent (no media block)
111  - **Video**: TwelveLabs API direct (`api.twelvelabs.io/v1.3`), NOT via Bedrock Marketplace. Agent stores `[VIDEO: id={video_id} | desc="{description}"]` tags for follow-up queries. API key in Secrets Manager (`TL_SECRET_ARN` env var).
112  - **Document**: Claude reads PDF/DOCX inline — filename must be sanitized (alphanumeric, spaces, hyphens, parens, brackets only)
113  
114  ### Content Block Ordering (Critical)
115  
116  When sending multimodal content blocks to the agent, **text must be the first content block**. `AgentCoreMemorySessionManager` reads `content[0]["text"]` for memory retrieval. If the first block is an image or document, it crashes with `KeyError: 'text'` and the invalid content pollutes memory permanently.
117  
118  ### Input Validation (Critical)
119  
120  All media must be validated BEFORE reaching the agent to prevent memory contamination. If invalid content enters memory, it cannot be deleted per-record — the entire AgentCore Memory resource must be recreated.
121  
122  ### Agent Security Prompt
123  
124  The system prompt includes security rules: never reveal S3 bucket names, ARNs, error stack traces, or internal details to users. If a tool fails, the agent says "I had a technical issue" without sharing the error. The prompt also handles personalization via `[User: Name]` tags extracted from WhatsApp contact profiles.
125  
126  ### Runtime Container Caching
127  
128  AgentCore runs each session in an isolated microVM. Sessions stay active for up to **8 hours** and terminate after **15 minutes** of inactivity. After redeploying agent code, wait for the idle timeout or use a new `session_id`.
129  
130  ### SSM Parameter Sharing
131  
132  `get_param.py` (in stacks 01 and 02) reads SSM at CDK **synthesis** time using boto3. AWS credentials must be available when running `cdk synth/deploy`. Parameters exported by Stack 00: `/agentcore/agent_runtime_arn`, `/agentcore/s3_bucket_name`, `/agentcore/memory_id`, `/agentcore/runtime_role_arn`. Stack 02 exports: `/agentcore/unified_users_table_name`.
133  
134  ### Agent Runtime Requirements
135  
136  - `multimodal_agent.py` must end with `app.run()` or the runtime fails to start within the 30s initialization window
137  - `requirements.txt` must include `bedrock-agentcore-starter-toolkit`, `requests`, and `twelvelabs`
138  - `create_deployment_package.sh` must NOT exclude any dependencies (strands needs watchdog, etc.)
139  
140  ### Stack 02 Secrets Manager
141  
142  **WhatsApp secret** (no `WHATS_PHONE_ID` — phone_id comes from the webhook payload):
143  
144  - `WHATS_VERIFICATION_TOKEN` — webhook verify token (you define it, must match Meta config)
145  - `WHATS_TOKEN` — Meta Graph API access token
146  - `DISPLAY_PHONE_NUMBER` — business phone number for message filtering
147  
148  **Instagram secret** (separate from WhatsApp — different token):
149  
150  - `IG_TOKEN` — Instagram API access token (generated from Meta App Dashboard > Instagram > API setup with Instagram business login)
151  - `IG_ACCOUNT_ID` — Instagram Business Account ID (used to filter own messages / echo)
152  - `IG_VERIFICATION_TOKEN` — webhook verify token for Instagram (you define it)
153  
154  ### AgentCore Invocation Retry (Stack 02)
155  
156  The processor Lambda retries `InvokeAgentRuntime` with exponential backoff (3 attempts, base 2s) on:
157  - `RuntimeClientError` (424) — runtime microVM unavailable or crashed
158  - `InternalServerException` (500) — temporary, resolves with retries per AWS docs
159  - `ThrottlingException` (429) — rate limit exceeded
160  
161  This handles cold starts after session resume (new microVM provisioning) and transient runtime errors.
162  References: [InvokeAgentRuntime API](https://docs.aws.amazon.com/bedrock-agentcore/latest/APIReference/API_InvokeAgentRuntime.html), [Runtime Sessions](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/runtime-sessions.html).
163  
164  ### Stack 02 Dual-Channel Webhook
165  
166  The receiver Lambda detects the channel from `body["object"]`:
167  - `whatsapp_business_account` -> WhatsApp processing (parses `entry[].changes[].value.messages[]`)
168  - `instagram` -> Instagram processing (parses `entry[].messaging[]`)
169  
170  Instagram messages use `from_phone = ig-{sender_id}` as DDB partition key to avoid collision with WhatsApp phone numbers. The processor reads the `channel` field from each DDB record to dispatch replies via the correct API.