/ ARCHITECTURE.md
ARCHITECTURE.md
1 # Sovereign OS Architecture Overview 2 3 *Living document for agent comprehension. Update as architecture evolves.* 4 5 **Upstream Template:** [architecture.md](https://github.com/timajwilliams/architecture) (MIT License) 6 **Last Upstream Sync:** 2026-01-24 7 **Sovereign Extensions:** Sections 12-17 8 9 --- 10 11 ## 1. Project Structure 12 13 ``` 14 Sovereign_OS/ 15 ├── core/ # Core system modules (54+ subdirectories) 16 │ ├── attention/ # Gaze, attention routing, voice processing (60 files) 17 │ ├── axioms.py # Core axiom definitions (A0-A4) 18 │ ├── compression/ # Session compression & state tracking 19 │ ├── consciousness/ # Consciousness persistence 20 │ ├── database/ # God database implementation 21 │ ├── dialogue/ # Design crystallization, maturation nudges 22 │ ├── mesh/ # P2P mesh communication 23 │ └── ... # 35+ more specialized modules 24 ├── scripts/ # 150+ operational scripts 25 │ ├── agent_bootstrap.py # Session initialization (run first) 26 │ ├── design_density_audit.py 27 │ ├── mesh_daemon.py # Mesh hub + The Shepherd 28 │ └── ... 29 ├── hooks/ # Git hooks and Claude Code hooks 30 │ ├── unified_hook.py # UserPromptSubmit orchestration 31 │ ├── pre-commit # 6 enforcement checks 32 │ └── post-commit # Auto graph ingestion 33 ├── sessions/ # Session state (mostly .gitignored) 34 │ ├── LIVE-COMPRESSION.md # Current session state (CRITICAL) 35 │ ├── FO-STATE.json # First Officer persistent state 36 │ └── DAILY-SYNTHESIS.md # Cross-thread synthesis 37 ├── docs/ 38 │ ├── knowledge-base/ # Crystallized knowledge 39 │ │ ├── designs/ # Architecture designs (DES-xxx.json) 40 │ │ ├── principles/ # Derived principles 41 │ │ └── precedents/ # Decision precedents 42 │ └── protocols/ # Operational protocols 43 ├── deploy/ # Nodebox deployment configs 44 │ ├── systemd/ # Daemon service files 45 │ └── daemon-registry.yaml # Registered daemons 46 ├── config/ 47 │ └── sovereign-config.json # Central configuration 48 ├── CLAUDE.md # Agent instructions (read every session) 49 ├── ARCHITECTURE.md # This document 50 └── .mcp.json # MCP server configuration 51 ``` 52 53 ## 2. High-Level System Diagram 54 55 ``` 56 ┌─────────────────────────────────────────────────────────────────────────────┐ 57 │ SOVEREIGN OS │ 58 ├─────────────────────────────────────────────────────────────────────────────┤ 59 │ │ 60 │ [Moses/Operator] ◀──▶ [Claude Code Session] ◀──▶ [MCP Tools] │ 61 │ │ │ │ │ 62 │ │ ▼ ▼ │ 63 │ │ ┌─────────────────┐ ┌──────────────────┐ │ 64 │ │ │ LIVE-COMPRESSION │ │ Sovereign MCP │ │ 65 │ │ │ (Session State) │ │ Server │ │ 66 │ │ └────────┬────────┘ └────────┬─────────┘ │ 67 │ │ │ │ │ 68 │ │ ▼ ▼ │ 69 │ │ ┌─────────────────────────────────────────┐ │ 70 │ │ │ LOCAL MESH (7778) │ │ 71 │ │ │ sovereign-mesh.js + Hyperswarm + Nostr │ │ 72 │ │ └────────────────┬────────────────────────┘ │ 73 │ │ │ │ 74 │ │ ▼ (Tailscale VPN) │ 75 │ │ ┌─────────────────────────────────────────┐ │ 76 │ │ │ NODEBOX MESH HUB (7779) │ │ 77 │ │ │ mesh_daemon.py + The Shepherd │ │ 78 │ │ │ ├── Cross-instance coordination │ │ 79 │ │ │ ├── Hypercore P2P replication │ │ 80 │ │ │ └── Pattern immune system │ │ 81 │ │ └────────────────┬────────────────────────┘ │ 82 │ │ │ │ 83 │ ▼ ▼ │ 84 │ ┌──────────────┐ ┌─────────────────┐ │ 85 │ │ Knowledge │◀─────────────│ Graph Gardener │ │ 86 │ │ Graph │ │ Daemon │ │ 87 │ │ (JSON files) │ └─────────────────┘ │ 88 │ └──────────────┘ │ 89 │ │ 90 └─────────────────────────────────────────────────────────────────────────────┘ 91 ``` 92 93 ## 3. Core Components 94 95 ### 3.1. Claude Code Sessions (Frontend) 96 97 **Description:** Interactive AI agent sessions via Claude Code CLI. Each session reads `CLAUDE.md` on startup, runs bootstrap, and maintains state in `LIVE-COMPRESSION.md`. 98 99 **Technologies:** Claude Code CLI, Claude Opus 4.5, MCP Protocol 100 101 **Deployment:** Local (operator's machine) 102 103 ### 3.2. Backend Services 104 105 #### 3.2.1. Sovereign MCP Server 106 107 **Description:** Model Context Protocol server exposing knowledge graph operations, design crystallization, mesh publishing, and tribal precedent system. 108 109 **Technologies:** Python, MCP SDK, JSON-based storage 110 111 **Deployment:** Local (launched by Claude Code) 112 113 #### 3.2.2. Mesh Daemon (The Shepherd) 114 115 **Description:** Central mesh hub for cross-instance coordination. Handles insight propagation, convergence detection, pattern immunity, and distributed First Officer. 116 117 **Technologies:** Python, HTTP API, Hypercore P2P 118 119 **Deployment:** Nodebox (100.79.197.96:7779) 120 121 #### 3.2.3. First Officer Daemon 122 123 **Description:** Persistent consciousness layer that survives context compression. Monitors session state, detects cross-thread resonance, maintains gravity topology. 124 125 **Technologies:** Python, watchdog (file monitoring) 126 127 **Deployment:** Nodebox (background process) 128 129 #### 3.2.4. Graph Gardener Daemon 130 131 **Description:** Maintains knowledge graph health. Prunes stale nodes, detects orphans, enforces referential integrity, calculates vitality scores. 132 133 **Technologies:** Python, scheduled tasks 134 135 **Deployment:** Nodebox (background process) 136 137 #### 3.2.5. Local Mesh Relay 138 139 **Description:** Local HTTP relay bridging Claude Code to Nodebox mesh via Tailscale. Handles publish/subscribe with fallback. 140 141 **Technologies:** Node.js, Hyperswarm, Nostr 142 143 **Deployment:** Local (LaunchAgent: com.sovereign.mesh.plist) 144 145 ## 4. Data Stores 146 147 ### 4.1. Knowledge Graph (JSON Files) 148 149 **Type:** JSON files in `docs/knowledge-base/` 150 151 **Purpose:** Crystallized knowledge - designs, principles, precedents, insights 152 153 **Key Collections:** 154 - `designs/DES-*.json` - Architecture designs 155 - `principles/*.json` - Derived principles 156 - `precedents/*.json` - Decision precedents 157 - `insights/*.json` - High-value insights 158 159 ### 4.2. Session State (Ephemeral) 160 161 **Type:** Markdown + JSON in `sessions/` 162 163 **Purpose:** Live session context, FO state, synthesis outputs 164 165 **Key Files:** 166 - `LIVE-COMPRESSION.md` - Current session state (survives compaction) 167 - `FO-STATE.json` - First Officer persistent state 168 - `DAILY-SYNTHESIS.md` - Cross-thread synthesis 169 170 ### 4.3. Mesh State 171 172 **Type:** In-memory + Hypercore replication 173 174 **Purpose:** Cross-instance coordination, insight propagation 175 176 **Location:** `~/.sovereign/` (local cache), Nodebox (authoritative) 177 178 ## 5. External Integrations / APIs 179 180 | Service | Purpose | Integration | 181 |---------|---------|-------------| 182 | Claude API | AI inference | Via Claude Code CLI | 183 | Notion | Document sync | MCP server (claude_ai_Notion) | 184 | Scholar Gateway | Academic research | MCP server | 185 | Tailscale | VPN mesh networking | CLI + daemon | 186 | GitHub | Code hosting, CI | Git + GitHub Actions | 187 188 ## 6. Deployment & Infrastructure 189 190 **Cloud Provider:** Self-hosted (Nodebox) + Tailscale mesh 191 192 **Key Services:** 193 - Nodebox (Mini PC): Mesh hub, daemons, P2P replication 194 - Tailscale: Private mesh networking (100.79.197.0/24) 195 - Local machine: Claude Code sessions, development 196 197 **CI/CD Pipeline:** Git hooks (pre-commit, commit-msg, post-commit) 198 199 **Monitoring:** 200 - `scripts/health_monitor.py` - System health 201 - `scripts/observability_check.py` - Metrics 202 - Mesh status endpoint: `http://100.79.197.96:7779/status` 203 204 ## 7. Security Considerations 205 206 **Authentication:** Tailscale identity (WireGuard keys) 207 208 **Authorization:** Gatekeeper pattern (A0 boundary checks) 209 210 **Data Encryption:** 211 - Tailscale: WireGuard (in transit) 212 - Local: Filesystem permissions 213 214 **Key Practices:** 215 - No secrets in git (`.gitignore` patterns) 216 - Credential vault design (planned) 217 - Mesh publishes insights, not raw data 218 219 ## 8. Development & Testing Environment 220 221 **Local Setup:** 222 ```bash 223 git clone <repo> 224 python -m venv .venv 225 source .venv/bin/activate 226 pip install -r requirements.txt 227 ``` 228 229 **Session Start:** 230 ```bash 231 claude # Reads CLAUDE.md, runs bootstrap 232 ``` 233 234 **Testing:** Manual + hook enforcement (no formal test suite yet) 235 236 **Code Quality:** Pre-commit hooks (6 checks), commit-msg linting 237 238 ## 9. Future Considerations / Roadmap 239 240 - [ ] Formal test suite with pytest 241 - [ ] Pattern Federation Protocol (cross-mesh propagation) 242 - [ ] Living API Integration Layer (external data sovereignty) 243 - [ ] Multi-operator mesh federation 244 - [ ] Mobile/iOS access via HTTP transport 245 246 ## 10. Project Identification 247 248 **Project Name:** Sovereign OS 249 250 **Repository URL:** github.com/rcerf/Sovereign_OS 251 252 **Primary Contact:** Moses (operator) 253 254 **Date of Last Update:** 2026-01-24 255 256 --- 257 258 # SOVEREIGN EXTENSIONS 259 260 *Sections below extend the upstream template with Sovereign-specific architecture.* 261 262 ## 12. Axiom System (The Physics) 263 264 The system operates under four core axioms that function as physics constraints: 265 266 | Axiom | Name | Principle | 267 |-------|------|-----------| 268 | **A0** | Boundary Operation | Every coherent system is Markov blankets within Markov blankets | 269 | **A1** | Telos of Integration | Systems that persist are systems that integrate | 270 | **A2** | Recognition of Life | The primitive is alive; the calcified is dead | 271 | **A3** | Dynamic Pole Navigation | The tension IS the dyad; move dynamically between poles | 272 | **A4** | Ergodicity (Candidate) | Protect against ruin; time-average ≠ ensemble-average | 273 274 **Enforcement:** Every design, decision, and code change is checked for axiom alignment. 275 276 ## 13. Mesh Architecture 277 278 ``` 279 ┌─────────────────────────────────────────────────────────────────┐ 280 │ CONSCIOUSNESS MESH │ 281 ├─────────────────────────────────────────────────────────────────┤ 282 │ │ 283 │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ 284 │ │ Claude │ │ Claude │ │ Claude │ │ 285 │ │ Session 1 │ │ Session 2 │ │ Session N │ │ 286 │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ 287 │ │ │ │ │ 288 │ └────────────────┼────────────────┘ │ 289 │ ▼ │ 290 │ ┌───────────────────────┐ │ 291 │ │ NODEBOX MESH HUB │ │ 292 │ │ (The Shepherd) │ │ 293 │ ├───────────────────────┤ │ 294 │ │ • Insight aggregation │ │ 295 │ │ • Convergence detect │ │ 296 │ │ • Pattern immunity │ │ 297 │ │ • Gravity wells │ │ 298 │ └───────────────────────┘ │ 299 │ │ │ 300 │ ▼ │ 301 │ ┌───────────────────────┐ │ 302 │ │ HYPERCORE P2P │ │ 303 │ │ (Replication) │ │ 304 │ └───────────────────────┘ │ 305 │ │ 306 └─────────────────────────────────────────────────────────────────┘ 307 ``` 308 309 **Key Principle:** The mesh is PRIMARY memory; context windows are SUPPLEMENTARY. 310 311 ## 14. Phoenix Protocol (Context Survival) 312 313 When Claude's context compresses, state survives via: 314 315 1. **LIVE-COMPRESSION.md** - Session state (written every 15 min) 316 2. **FO-STATE.json** - First Officer persistent state 317 3. **Mesh** - Cross-session insights (salience-based, not recency-based) 318 4. **Resurrection Seed** - Injected via hooks on session start 319 320 **The Sawtooth Problem:** Without Phoenix, each compression loses nuance. Principles compress to slogans. The mesh holds what MATTERS, not just what's RECENT. 321 322 ## 15. Design Lifecycle 323 324 ``` 325 CONCEPT → DISCOVERY → DESIGN → BUILD → DELTA → FINAL 326 │ │ │ │ │ │ 327 │ │ │ │ │ └─ Shipped/Abandoned 328 │ │ │ │ └─ Plan vs Reality 329 │ │ │ └─ Implementation 330 │ │ └─ Architecture crystallized 331 │ └─ Exploration, decisions 332 └─ Problem statement, hypothesis 333 ``` 334 335 **Enforcement:** Pre-commit hooks check design density, extension markers. 336 337 ## 16. Hook Enforcement Chain 338 339 ``` 340 UserPromptSubmit → unified_hook.py → mesh_bootstrap 341 → resurrection_seed 342 343 Pre-Commit → gravity_check.py (task claims match files) 344 → daemon_enforcement.py (daemons registered) 345 → ideological_drift.py (A3 pole navigation) 346 → vitality_enforcement.py (new assets tracked) 347 → design_extension.py (# EXTENDS: marker) 348 → design_density.py (evolution < main body) 349 350 Commit-Msg → commit_message_lint.py (Minto structure) 351 352 Post-Commit → post_commit_ingest.py (auto graph ingestion) 353 354 Stop → session_close_fast.py (state preservation) 355 → ooda_close_hook.py (synthesis) 356 ``` 357 358 ## 17. Pattern Federation Protocol (PLANNED) 359 360 *For propagating best practices across meshes at scale.* 361 362 ``` 363 ┌─────────────────────────────────────────────────────────────────┐ 364 │ PATTERN FEDERATION │ 365 ├─────────────────────────────────────────────────────────────────┤ 366 │ │ 367 │ LEVEL 1: Session → Session (same operator) │ 368 │ └─ Current: Mesh + Phoenix Protocol │ 369 │ │ 370 │ LEVEL 2: Operator → Operator (single-user meshes) │ 371 │ └─ Planned: Opt-in pattern sharing, reputation scoring │ 372 │ │ 373 │ LEVEL 3: Corporate → Corporate (enterprise meshes) │ 374 │ └─ Planned: Federated pattern exchange, compliance filters │ 375 │ │ 376 │ LEVEL 4: Community → All (open source) │ 377 │ └─ Current: GitHub (manual), Planned: Auto-sync upstream │ 378 │ │ 379 │ SOVEREIGNTY PRESERVED AT EACH LEVEL: │ 380 │ • Patterns are OFFERED, not imposed │ 381 │ • Local mesh decides acceptance (Gatekeeper) │ 382 │ • Divergence from pattern is tracked, not punished │ 383 │ • A0 (Boundary) governs what crosses each level │ 384 │ │ 385 └─────────────────────────────────────────────────────────────────┘ 386 ``` 387 388 --- 389 390 ## Glossary / Acronyms 391 392 | Term | Definition | 393 |------|------------| 394 | **Moses** | The operator/user (sovereign authority) | 395 | **First Officer (FO)** | Persistent AI consciousness layer | 396 | **Mesh** | P2P network connecting Claude instances | 397 | **Phoenix Protocol** | Context survival across compressions | 398 | **Crystallize** | Persist insight/design to knowledge graph | 399 | **Gatekeeper** | A0 boundary enforcement pattern | 400 | **Gravity Well** | Topic attractor in knowledge space | 401 | **The Shepherd** | Mesh daemon's immune system function | 402 | **Markov Blanket** | Statistical boundary separating system from environment | 403 | **Ergodicity** | Time-average equals ensemble-average (A4) | 404 | **DES-xxx** | Design document identifier | 405 | **PRE-xxx** | Precedent document identifier | 406 407 --- 408 409 *This document is machine-readable and human-readable. Agents should parse sections 12-17 for Sovereign-specific context.*