1  # ECHO Test Suite
  2  
  3  Comprehensive testing infrastructure for the ECHO multi-agent system.
  4  
  5  ## Directory Structure
  6  
  7  ```
  8  test/
  9  ├── README.md              # This file
 10  ├── integration/           # Multi-component integration tests
 11  ├── e2e/                   # End-to-end workflow tests
 12  └── fixtures/              # Test data and fixtures
 13  ```
 14  
 15  ## Test Categories
 16  
 17  ### Integration Tests (`integration/`)
 18  Tests that verify multiple components working together:
 19  
 20  - **Database + Redis** - Message persistence and pub/sub
 21  - **Agent Communication** - Message bus between agents
 22  - **Decision Workflows** - Collaborative decision-making
 23  - **LLM Integration** - Ollama API calls and responses
 24  
 25  **Examples:**
 26  - `test_agent_messaging_test.exs` - Agent-to-agent communication
 27  - `test_decision_workflow_test.exs` - Multi-agent decisions
 28  - `test_redis_pubsub_test.exs` - Redis pub/sub reliability
 29  
 30  ### End-to-End Tests (`e2e/`)
 31  Complete workflow tests simulating real-world scenarios:
 32  
 33  - **Feature Development** - PM → Architect → Dev → Test → CEO
 34  - **Hiring Workflow** - CHRO → CEO collaborative decisions
 35  - **Incident Response** - Multi-agent coordination
 36  
 37  **Examples:**
 38  - `test_feature_development_workflow.sh` - Full feature workflow
 39  - `test_hiring_workflow.sh` - Complete hiring process
 40  - `test_autonomous_mode.sh` - Agents running standalone
 41  
 42  ### Fixtures (`fixtures/`)
 43  Reusable test data for consistent testing:
 44  
 45  - `sample_decisions.json` - Decision test data
 46  - `sample_messages.json` - Message test data
 47  - `sample_agents.json` - Agent configuration data
 48  - `sample_workflows.json` - Workflow definitions
 49  
 50  ## Running Tests
 51  
 52  ### All Tests
 53  ```bash
 54  cd /Users/pranav/Documents/echo
 55  ./scripts/testing/test_all.sh
 56  ```
 57  
 58  ### Integration Tests Only
 59  ```bash
 60  cd shared
 61  mix test test/integration/
 62  ```
 63  
 64  ### E2E Tests Only
 65  ```bash
 66  cd test/e2e
 67  ./test_feature_development_workflow.sh
 68  ```
 69  
 70  ### Specific Test File
 71  ```bash
 72  cd shared
 73  mix test test/integration/agent_messaging_test.exs
 74  ```
 75  
 76  ## Writing Tests
 77  
 78  ### Integration Test Template
 79  
 80  ```elixir
 81  defmodule Echo.Integration.YourTest do
 82    use ExUnit.Case, async: false  # Database access = no async
 83  
 84    alias EchoShared.MessageBus
 85    alias EchoShared.Schemas.Message
 86    alias EchoShared.Repo
 87  
 88    setup do
 89      # Clean database before each test
 90      Repo.delete_all(Message)
 91      :ok
 92    end
 93  
 94    describe "your feature" do
 95      test "does what you expect" do
 96        # Arrange
 97  
 98        # Act
 99  
100        # Assert
101        assert true
102      end
103    end
104  end
105  ```
106  
107  ### E2E Test Template
108  
109  ```bash
110  #!/usr/bin/env bash
111  set -euo pipefail
112  
113  echo "=== E2E Test: Your Workflow ==="
114  
115  # Setup
116  cd "$(dirname "$0")/../.."
117  source scripts/utils/test_helpers.sh
118  
119  # Test steps
120  step "Start agents" start_test_agents
121  step "Send message" send_test_message
122  step "Verify outcome" verify_test_result
123  
124  # Cleanup
125  cleanup_test_agents
126  
127  echo "✅ Test passed!"
128  ```
129  
130  ## Test Helpers
131  
132  ### `scripts/utils/test_helpers.sh`
133  Common test utilities:
134  
135  ```bash
136  # Start agents in background
137  start_test_agents() { ... }
138  
139  # Stop test agents
140  cleanup_test_agents() { ... }
141  
142  # Send test message
143  send_test_message() { ... }
144  
145  # Wait for condition
146  wait_for() { ... }
147  
148  # Assert equality
149  assert_equal() { ... }
150  ```
151  
152  ## CI/CD Integration
153  
154  ### GitHub Actions Workflow
155  
156  ```yaml
157  name: Test
158  
159  on: [push, pull_request]
160  
161  jobs:
162    test:
163      runs-on: ubuntu-latest
164  
165      services:
166        postgres:
167          image: postgres:16
168          env:
169            POSTGRES_DB: echo_org_test
170            POSTGRES_PASSWORD: postgres
171  
172        redis:
173          image: redis:7-alpine
174  
175      steps:
176        - uses: actions/checkout@v3
177        - name: Set up Elixir
178          uses: erlef/setup-beam@v1
179          with:
180            elixir-version: '1.18'
181            otp-version: '27'
182  
183        - name: Install dependencies
184          run: cd shared && mix deps.get
185  
186        - name: Run tests
187          run: cd shared && mix test
188  
189        - name: Run E2E tests
190          run: ./scripts/testing/test_all.sh
191  ```
192  
193  ## Test Coverage
194  
195  ### Current Coverage
196  ```
197  Agents: 85% coverage
198  Shared Library: 90% coverage
199  Workflows: 70% coverage
200  Integration: 60% coverage (growing)
201  ```
202  
203  ### Coverage Goals
204  - Agents: 90%+
205  - Shared Library: 95%+
206  - Workflows: 85%+
207  - Integration: 80%+
208  
209  ### Generate Coverage Report
210  ```bash
211  cd shared
212  mix test --cover
213  open cover/excoveralls.html
214  ```
215  
216  ## Best Practices
217  
218  1. **Isolate Tests** - Each test should be independent
219  2. **Use Fixtures** - Reuse test data from `fixtures/`
220  3. **Clean State** - Setup/teardown for clean database
221  4. **Descriptive Names** - Test names should describe behavior
222  5. **Fast Tests** - Optimize for quick feedback
223  6. **No Flaky Tests** - Fix intermittent failures immediately
224  
225  ## Debugging Tests
226  
227  ### Failed Test
228  ```bash
229  # Run with verbose output
230  mix test --trace test/path/to/test.exs
231  
232  # Run single test
233  mix test test/path/to/test.exs:42
234  ```
235  
236  ### Database State
237  ```bash
238  # Check database during test
239  psql -U postgres -h localhost -p 5433 -d echo_org_test
240  ```
241  
242  ### Agent Logs
243  ```bash
244  # View agent logs during E2E test
245  tail -f logs/ceo.log logs/cto.log
246  ```
247  
248  ## Related Documentation
249  
250  - **Agent Testing:** [../agents/claude.md](../agents/claude.md)
251  - **Shared Library:** [../shared/claude.md](../shared/claude.md)
252  - **CI/CD:** [../.github/workflows/](../.github/workflows/)
253  
254  ---
255  
256  **Last Updated:** 2025-11-06
257  **Test Count:** Growing (integration tests being added)
258  **Framework:** ExUnit + Bash