/ memory-bank / techContext.md
techContext.md
1 # CREATURE Technical Context 2 3 ## Technology Stack 4 5 ### Core Technologies 6 7 - **Language**: Go (Golang) 1.20+ 8 - **External APIs**: OpenRouter API for language model capabilities 9 - **Data Storage**: JSON-based file storage 10 - **Visualization**: Terminal-based ASCII visualization 11 - **Concurrency**: Go's goroutines and channels 12 13 ### Key Libraries 14 15 - **github.com/google/uuid**: For generating unique identifiers 16 - **github.com/fatih/color**: For terminal color output 17 - **context**: For managing timeouts and cancellation 18 - **encoding/json**: For JSON serialization/deserialization 19 - **sync**: For synchronization primitives 20 21 ## Development Environment 22 23 ### Requirements 24 25 - Go 1.20 or higher 26 - OpenRouter API key 27 - Terminal with Unicode support 28 - Git for version control 29 30 ### Setup 31 32 1. Clone the repository 33 2. Set the `OPENROUTER_API_KEY` environment variable 34 3. Run `go mod tidy` to install dependencies 35 4. Build with `go build -o creature ./cmd/creature` 36 5. Run with `./creature` 37 38 ### Directory Structure 39 40 ``` 41 creature/ 42 ├── cmd/ 43 │ └── creature/ # Main entry point 44 ├── data/ 45 │ ├── thoughts/ # Stored thoughts 46 │ └── plans/ # Stored plans 47 ├── src/ 48 │ ├── api/ # API clients 49 │ ├── models/ # Data models 50 │ ├── systems/ # Core systems 51 │ ├── utils/ # Utility functions 52 │ └── server.go # HTTP server 53 ├── go.mod # Go module definition 54 └── go.sum # Go module checksums 55 ``` 56 57 ## Technical Constraints 58 59 ### API Limitations 60 61 - **Rate Limits**: OpenRouter API has rate limits that must be respected 62 - **Timeout Handling**: API calls can time out and must be retried 63 - **Cost Management**: API usage incurs costs that should be minimized 64 65 ### Performance Considerations 66 67 - **Memory Usage**: The system should manage memory efficiently, especially for large colonies 68 - **Concurrency**: Goroutines should be used judiciously to avoid excessive resource consumption 69 - **JSON Overhead**: JSON serialization/deserialization can be expensive for large data structures 70 71 ### Error Handling 72 73 - **Graceful Degradation**: The system should continue to function even if some components fail 74 - **Logging**: Errors should be logged with sufficient context for diagnosis 75 - **Retry Logic**: Failed operations should be retried with exponential backoff 76 77 ## Dependencies 78 79 ### External Dependencies 80 81 - **OpenRouter API**: Used for thought generation and memory compression 82 - **Terminal**: Used for visualization and user interaction 83 84 ### Internal Dependencies 85 86 - **Models → Systems**: Systems depend on models for data structures 87 - **Systems → API**: Systems depend on API clients for external services 88 - **Utils → All**: Utility functions are used throughout the codebase 89 90 ## Configuration 91 92 ### Environment Variables 93 94 - `OPENROUTER_API_KEY`: API key for OpenRouter 95 - `LOG_LEVEL`: Logging level (debug, info, warning, error, critical) 96 - `DATA_DIR`: Directory for data storage (default: ./data) 97 98 ### Command-Line Flags 99 100 - `--mission`: Mission statement for the colony 101 - `--cells`: Initial number of cells (default: 10) 102 - `--cycles`: Number of cycles to run (default: 100) 103 - `--log-to-file`: Whether to log to file instead of stdout 104 - `--state-file`: Path to state file for loading/saving state 105 106 ## Build and Deployment 107 108 ### Build Process 109 110 ```bash 111 go build -o creature ./cmd/creature 112 ``` 113 114 ### Deployment Options 115 116 - **Local**: Run locally for development and testing 117 - **Server**: Deploy to a server for longer-running simulations 118 - **Container**: Package in a container for portable deployment 119 120 ### Monitoring 121 122 - **Logging**: The system logs to stdout or a file 123 - **Statistics**: The system generates statistics that can be monitored 124 - **Visualization**: The terminal visualization provides real-time feedback 125 126 ## Testing 127 128 ### Testing Approach 129 130 - **Unit Tests**: Test individual components in isolation 131 - **Integration Tests**: Test interactions between components 132 - **System Tests**: Test the entire system end-to-end 133 134 ### Test Coverage 135 136 - **Models**: High coverage for data models 137 - **Systems**: Medium coverage for core systems 138 - **API**: Mock-based testing for API clients 139 140 ## Security Considerations 141 142 ### API Key Management 143 144 - API keys should be stored securely as environment variables 145 - API keys should not be committed to version control 146 147 ### Data Privacy 148 149 - The system does not collect or store personal data 150 - Thought and plan content is stored locally and not shared 151 152 ## Future Technical Directions 153 154 ### Web Interface 155 156 - Develop a web-based visualization and interaction interface 157 - Use WebSockets for real-time updates 158 - Implement a REST API for external integration 159 160 ### Distributed Operation 161 162 - Enable distributed operation across multiple machines 163 - Implement a coordination mechanism for distributed cells 164 - Use a distributed database for state storage 165 166 ### Alternative Storage 167 168 - Implement a database backend for improved performance and scalability 169 - Support multiple storage backends through an interface 170 - Implement data compression for efficient storage