ECHO_ARCHITECTURE.md
1 ╔══════════════════════════════════════════════════════════════════════════╗ 2 ║ ECHO STANDALONE REPOSITORY - MCP ARCHITECTURE DESIGN ║ 3 ║ Designed by: Senior Architect Agent ║ 4 ║ Date: 2025-11-03 ║ 5 ╚══════════════════════════════════════════════════════════════════════════╝ 6 7 ## 1. SYSTEM ARCHITECTURE 8 9 ### High-Level Architecture 10 11 ┌─────────────────────────────────────────────────────────────────────┐ 12 │ MCP Client Layer (Claude Desktop, Cline, etc.) │ 13 └────────────┬────────────────────────────────────────────────────────┘ 14 │ 15 │ JSON-RPC 2.0 over stdio 16 │ 17 ┌────────┴─────────────────────────────────────────────────┐ 18 │ │ 19 ▼ ▼ 20 ┌─────────────────┐ ┌──────────────────┐ 21 │ Agent MCP │ │ Agent MCP │ 22 │ Servers (9) │◄───────── Redis Pub/Sub ─────────►│ Servers (cont.) │ 23 │ │ │ │ 24 │ • CEO │ │ • Architect │ 25 │ • CTO │ │ • UI/UX │ 26 │ • CHRO │ │ • Developer │ 27 │ • Ops Head │ │ • Test Lead │ 28 │ • Prod Mgr │ │ │ 29 └────────┬────────┘ └────────┬─────────┘ 30 │ │ 31 │ │ 32 └──────────────────┬───────────────────────────────────┘ 33 │ 34 ▼ 35 ┌──────────────────────┐ 36 │ Shared Storage │ 37 │ │ 38 │ • PostgreSQL │ 39 │ - Decisions │ 40 │ - Messages │ 41 │ - Memories │ 42 │ │ 43 │ • Redis │ 44 │ - Message Bus │ 45 │ - Pub/Sub │ 46 │ - Agent Status │ 47 └──────────────────────┘ 48 49 ### Component Breakdown 50 51 **MCP Server Layer (9 Independent Processes)** 52 - Each agent runs as standalone Elixir/OTP application 53 - Implements MCP protocol (JSON-RPC 2.0 over stdio) 54 - Exposes role-specific tools and resources 55 - Stateless (state in PostgreSQL) 56 - Horizontally scalable 57 58 **Message Bus (Redis)** 59 - Pub/Sub channels per agent 60 - Broadcast channels for all-hands messages 61 - Decision coordination channel 62 - Real-time event streaming 63 - TTL-based message expiry 64 65 **Shared Storage (PostgreSQL)** 66 - ACID transactions for decisions 67 - Full audit trail 68 - Cross-agent state synchronization 69 - Ecto for schema validation 70 - Migrations for versioning 71 72 ## 2. MCP TOOL DEFINITIONS 73 74 ### CEO MCP Server Tools 75 76 1. **approve_strategic_initiative** 77 - Input: {initiative: string, business_value: string, budget: number} 78 - Output: {approved: boolean, decision_id: string, rationale: string} 79 - Authority: Autonomous up to $1M, else requires board approval 80 81 2. **allocate_budget** 82 - Input: {amount: number, department: string, justification: string} 83 - Output: {allocated: boolean, tracking_id: string} 84 - Authority: Full control over approved budget 85 86 3. **escalate_to_board** 87 - Input: {decision_id: string, urgency: enum} 88 - Output: {escalation_id: string, human_approval_required: true} 89 90 4. **review_organizational_health** 91 - Input: {metrics: [string]} 92 - Output: {health_score: number, issues: [string], recommendations: [string]} 93 94 ### CTO MCP Server Tools 95 96 1. **review_architecture** 97 - Input: {design: object, components: [string], scalability: string} 98 - Output: {approved: boolean, feedback: string, modifications: [string]} 99 100 2. **select_technology** 101 - Input: {options: [string], criteria: object, timeline: string} 102 - Output: {selected: string, rationale: string, migration_plan: string} 103 104 3. **assess_technical_debt** 105 - Input: {codebase_metrics: object} 106 - Output: {debt_score: number, priority_fixes: [string]} 107 108 4. **approve_deployment** 109 - Input: {release: string, environments: [string]} 110 - Output: {approved: boolean, deployment_plan: object} 111 112 ### Product Manager MCP Server Tools 113 114 1. **define_feature** 115 - Input: {name: string, user_stories: [string], business_value: string} 116 - Output: {feature_id: string, requirements: object, priority: enum} 117 118 2. **prioritize_backlog** 119 - Input: {items: [object], constraints: object} 120 - Output: {prioritized_backlog: [object], rationale: string} 121 122 3. **gather_requirements** 123 - Input: {feature: string, stakeholders: [string]} 124 - Output: {requirements_doc: object, acceptance_criteria: [string]} 125 126 ### Senior Architect MCP Server Tools 127 128 1. **design_system** 129 - Input: {requirements: object, constraints: object} 130 - Output: {architecture: object, components: [object], data_flow: string} 131 132 2. **evaluate_technology** 133 - Input: {technology: string, use_case: string} 134 - Output: {score: number, pros: [string], cons: [string], recommendation: string} 135 136 3. **create_technical_spec** 137 - Input: {feature: string, architecture: object} 138 - Output: {spec_document: object, implementation_guide: string} 139 140 ... (similar tool definitions for remaining 5 agents) 141 142 ## 3. DATABASE SCHEMA 143 144 ```sql 145 -- Organizational Decisions 146 CREATE TABLE decisions ( 147 id UUID PRIMARY KEY, 148 decision_type VARCHAR(100) NOT NULL, 149 initiator_role VARCHAR(50) NOT NULL, 150 participants JSONB, 151 mode VARCHAR(50) NOT NULL, -- autonomous, collaborative, hierarchical, human 152 context JSONB NOT NULL, 153 status VARCHAR(50) NOT NULL, -- pending, approved, rejected, escalated 154 consensus_score FLOAT, 155 outcome JSONB, 156 created_at TIMESTAMP NOT NULL, 157 completed_at TIMESTAMP, 158 159 INDEX idx_decisions_role (initiator_role), 160 INDEX idx_decisions_status (status), 161 INDEX idx_decisions_type (decision_type) 162 ); 163 164 -- Inter-Agent Messages 165 CREATE TABLE messages ( 166 id BIGSERIAL PRIMARY KEY, 167 from_role VARCHAR(50) NOT NULL, 168 to_role VARCHAR(50) NOT NULL, 169 type VARCHAR(50) NOT NULL, -- request, response, notification, escalation 170 subject VARCHAR(255) NOT NULL, 171 content JSONB NOT NULL, 172 metadata JSONB, 173 read BOOLEAN DEFAULT FALSE, 174 created_at TIMESTAMP NOT NULL, 175 176 INDEX idx_messages_to (to_role, created_at), 177 INDEX idx_messages_from (from_role, created_at), 178 INDEX idx_messages_unread (to_role, read) 179 ); 180 181 -- Organizational Memory 182 CREATE TABLE memories ( 183 id UUID PRIMARY KEY, 184 key VARCHAR(255) UNIQUE NOT NULL, 185 content TEXT NOT NULL, 186 tags TEXT[] NOT NULL, 187 metadata JSONB, 188 created_by_role VARCHAR(50), 189 inserted_at TIMESTAMP NOT NULL, 190 updated_at TIMESTAMP NOT NULL, 191 192 INDEX idx_memories_key (key), 193 INDEX idx_memories_tags (tags) USING GIN, 194 INDEX idx_memories_role (created_by_role) 195 ); 196 197 -- Collaborative Decision Votes 198 CREATE TABLE decision_votes ( 199 id BIGSERIAL PRIMARY KEY, 200 decision_id UUID REFERENCES decisions(id), 201 voter_role VARCHAR(50) NOT NULL, 202 vote VARCHAR(20) NOT NULL, -- approve, reject, abstain 203 rationale TEXT, 204 confidence FLOAT NOT NULL, 205 voted_at TIMESTAMP NOT NULL, 206 207 UNIQUE(decision_id, voter_role), 208 INDEX idx_votes_decision (decision_id) 209 ); 210 211 -- Agent Health Status 212 CREATE TABLE agent_status ( 213 role VARCHAR(50) PRIMARY KEY, 214 status VARCHAR(20) NOT NULL, -- running, stopped, error 215 last_heartbeat TIMESTAMP NOT NULL, 216 version VARCHAR(20), 217 capabilities JSONB, 218 metadata JSONB, 219 220 INDEX idx_agent_status_heartbeat (last_heartbeat) 221 ); 222 ``` 223 224 ## 4. MESSAGE BUS PROTOCOL 225 226 ### Redis Pub/Sub Channels 227 228 ``` 229 # Per-Agent Channels 230 messages:ceo # Private messages to CEO 231 messages:cto # Private messages to CTO 232 messages:chro 233 messages:operations_head 234 messages:product_manager 235 messages:senior_architect 236 messages:uiux_engineer 237 messages:senior_developer 238 messages:test_lead 239 240 # Broadcast Channels 241 messages:all # All-hands announcements 242 messages:leadership # CEO, CTO, CHRO, Ops 243 244 # Decision Coordination 245 decisions:new # New decision initiated 246 decisions:vote_required # Vote needed from participant 247 decisions:completed # Decision finalized 248 decisions:escalated # Escalated to higher authority 249 250 # System Channels 251 agents:heartbeat # Agent health checks 252 agents:status # Agent status updates 253 ``` 254 255 ### Message Format (Redis) 256 257 ```json 258 { 259 "id": "msg_abc123", 260 "from": "ceo", 261 "to": "cto", 262 "type": "request", 263 "subject": "Q3 Technology Strategy Review", 264 "content": { 265 "question": "What are our top 3 priorities?", 266 "deadline": "2025-11-10", 267 "context": "Board meeting preparation" 268 }, 269 "metadata": { 270 "priority": "high", 271 "thread_id": "thread_xyz", 272 "timestamp": "2025-11-03T12:00:00Z" 273 } 274 } 275 ``` 276 277 ## 5. DEPLOYMENT ARCHITECTURE 278 279 ### Docker Deployment 280 281 ``` 282 docker-compose.yml: 283 284 services: 285 # Infrastructure 286 postgres: 287 image: postgres:16-alpine 288 volumes: [./data/postgres:/var/lib/postgresql/data] 289 environment: 290 POSTGRES_DB: echo_org 291 POSTGRES_USER: echo 292 POSTGRES_PASSWORD: ${DB_PASSWORD} 293 294 redis: 295 image: redis:7-alpine 296 volumes: [./data/redis:/data] 297 298 # Agent MCP Servers 299 echo-ceo: 300 build: ./agents/ceo 301 depends_on: [postgres, redis] 302 environment: 303 ROLE: ceo 304 DATABASE_URL: ${DATABASE_URL} 305 REDIS_URL: ${REDIS_URL} 306 307 echo-cto: 308 build: ./agents/cto 309 depends_on: [postgres, redis] 310 environment: 311 ROLE: cto 312 DATABASE_URL: ${DATABASE_URL} 313 REDIS_URL: ${REDIS_URL} 314 315 # ... (7 more agent services) 316 ``` 317 318 ### Kubernetes Deployment (Production) 319 320 ```yaml 321 apiVersion: v1 322 kind: Namespace 323 metadata: 324 name: echo-org 325 326 --- 327 # StatefulSet for each agent 328 apiVersion: apps/v1 329 kind: StatefulSet 330 metadata: 331 name: echo-ceo 332 namespace: echo-org 333 spec: 334 serviceName: echo-ceo 335 replicas: 1 336 selector: 337 matchLabels: 338 app: echo-ceo 339 role: ceo 340 template: 341 metadata: 342 labels: 343 app: echo-ceo 344 role: ceo 345 spec: 346 containers: 347 - name: ceo 348 image: ghcr.io/pranavj17/echo-ceo:latest 349 env: 350 - name: DATABASE_URL 351 valueFrom: 352 secretKeyRef: 353 name: echo-secrets 354 key: database-url 355 - name: REDIS_URL 356 valueFrom: 357 secretKeyRef: 358 name: echo-secrets 359 key: redis-url 360 ``` 361 362 ## 6. REPOSITORY STRUCTURE 363 364 ``` 365 echo/ 366 ├── .github/ 367 │ └── workflows/ 368 │ ├── ci.yml # Tests + builds 369 │ ├── docker-publish.yml # Push to GHCR 370 │ └── release.yml # GitHub releases 371 │ 372 ├── agents/ 373 │ ├── ceo/ 374 │ │ ├── Dockerfile 375 │ │ ├── mix.exs 376 │ │ ├── lib/ 377 │ │ │ └── echo/ 378 │ │ │ ├── mcp_server.ex 379 │ │ │ ├── tools.ex 380 │ │ │ └── decision_logic.ex 381 │ │ └── config/ 382 │ │ 383 │ ├── cto/ 384 │ ├── chro/ 385 │ ├── operations_head/ 386 │ ├── product_manager/ 387 │ ├── senior_architect/ 388 │ ├── uiux_engineer/ 389 │ ├── senior_developer/ 390 │ └── test_lead/ 391 │ 392 ├── shared/ 393 │ ├── lib/ 394 │ │ └── echo/ 395 │ │ ├── storage.ex # PostgreSQL interface 396 │ │ ├── message_bus.ex # Redis pub/sub 397 │ │ ├── schemas/ 398 │ │ │ ├── decision.ex 399 │ │ │ ├── message.ex 400 │ │ │ └── memory.ex 401 │ │ └── mcp/ 402 │ │ ├── protocol.ex # MCP JSON-RPC 403 │ │ └── base_server.ex 404 │ └── priv/ 405 │ └── repo/ 406 │ └── migrations/ 407 │ 408 ├── workflows/ 409 │ ├── feature_development.ex 410 │ ├── incident_response.ex 411 │ └── strategic_planning.ex 412 │ 413 ├── docker-compose.yml 414 ├── docker-compose.prod.yml 415 ├── k8s/ 416 │ ├── namespace.yml 417 │ ├── postgres.yml 418 │ ├── redis.yml 419 │ └── agents/ 420 │ ├── ceo.yml 421 │ └── ... 422 │ 423 ├── docs/ 424 │ ├── ARCHITECTURE.md 425 │ ├── MCP_TOOLS.md 426 │ ├── DEPLOYMENT.md 427 │ └── DEVELOPMENT.md 428 │ 429 ├── examples/ 430 │ ├── claude_desktop_config.json 431 │ ├── workflows/ 432 │ └── scripts/ 433 │ 434 ├── README.md 435 ├── LICENSE 436 └── .gitignore 437 ``` 438 439 ## 7. TECHNOLOGY DECISIONS 440 441 ### Core Stack 442 - **Language**: Elixir 1.18 443 - **Runtime**: Erlang/OTP 27 444 - **Database**: PostgreSQL 16 445 - **Message Bus**: Redis 7 446 - **Container**: Docker + Docker Compose 447 - **Orchestration**: Kubernetes (optional) 448 449 ### Key Libraries 450 - **Phoenix**: Web framework (if HTTP mode needed) 451 - **Ecto**: Database ORM 452 - **Redix**: Redis client 453 - **Jason**: JSON encoding/decoding 454 - **Telemetry**: Metrics and monitoring 455 456 ### Rationale 457 1. **Elixir/OTP**: Perfect actor model for agents 458 2. **PostgreSQL**: ACID guarantees for decisions 459 3. **Redis**: Low-latency pub/sub for real-time coordination 460 4. **Docker**: Consistent deployment across environments 461 5. **MCP Protocol**: Standard interface for AI agents 462 463 ## 8. IMPLEMENTATION PHASES 464 465 ### Phase 1: Foundation (Week 1) 466 - [ ] Shared library with MCP protocol implementation 467 - [ ] Database schemas and migrations 468 - [ ] Redis message bus implementation 469 - [ ] Base MCP server behavior 470 - [ ] CEO agent (reference implementation) 471 472 ### Phase 2: Core Agents (Week 2) 473 - [ ] Implement remaining 8 agent MCP servers 474 - [ ] Tool definitions for each role 475 - [ ] Inter-agent communication tested 476 - [ ] Docker Compose setup 477 - [ ] Local development environment 478 479 ### Phase 3: Workflows (Week 3) 480 - [ ] Feature development workflow 481 - [ ] Incident response workflow 482 - [ ] Collaborative decision engine 483 - [ ] Human-in-the-loop approvals 484 - [ ] Workflow orchestration 485 486 ### Phase 4: Production Ready (Week 4) 487 - [ ] Kubernetes manifests 488 - [ ] CI/CD pipelines 489 - [ ] Monitoring and observability 490 - [ ] Documentation complete 491 - [ ] Published to GitHub + Docker Hub 492 493 ## 9. RISK ASSESSMENT 494 495 **High Risk** 496 - Redis single point of failure 497 → Mitigation: Redis Sentinel for HA 498 499 **Medium Risk** 500 - Database connection pool exhaustion 501 → Mitigation: Ecto pool sizing + monitoring 502 503 **Low Risk** 504 - MCP protocol evolution 505 → Mitigation: Version pinning + adapter pattern 506 507 ## 10. SUCCESS METRICS 508 509 - [ ] All 9 agents startable via Claude Desktop 510 - [ ] Feature development workflow completes end-to-end 511 - [ ] < 100ms inter-agent message latency 512 - [ ] 99.9% agent uptime (via supervision) 513 - [ ] Zero cost for AI inference (uses Claude Desktop) 514 515 ═══════════════════════════════════════════════════════════════════════ 516 Architecture Review: APPROVED ✅ 517 Signed: Senior Architect Agent 518 Next Step: Begin implementation of Phase 1 519 ═══════════════════════════════════════════════════════════════════════