WHITE_PAPER_old.md
1 # Abzu: A Sovereign Mesh Network Protocol 2 3 > **Technical White Paper v0.4** — January 2026 4 > *Censorship-resistant communication with statistical invisibility.* 5 > 6 > *"The system should not depend on secrecy, and it should be possible for it to fall into enemy hands without inconvenience."* — Auguste Kerckhoffs, 1883 7 8 --- 9 10 ## Abstract 11 12 Abzu is a multi-protocol mesh networking engine providing **censorship-resistant, privacy-preserving peer-to-peer communication**. Built entirely in Rust, it combines: 13 14 - **Geometric routing** (Yggdrasil-style spanning tree coordinates) 15 - **Content-addressed storage** (BLAKE3 CIDs) 16 - **Stealth transport** (TLS 1.3 masquerade) 17 - **Statistical invisibility** (adaptive cover traffic with post-decrypt discrimination) 18 - **Group messaging** (Circles with epoch-based key rotation) 19 20 This document describes the architecture, cryptographic foundations, threat model, and current implementation status for v0.4. 21 22 --- 23 24 ## Vision and Motivation 25 26 ### Why Abzu Exists 27 28 The internet was designed for wartime resilience but has evolved into a centralized surveillance apparatus. DNS, TLS certificate authorities, BGP, and cloud infrastructure create natural chokepoints enabling both state and corporate censorship. 29 30 Abzu operates **beneath** this infrastructure—using TCP/WebSocket as transport substrates while providing: 31 32 | Property | Implementation | 33 | :--- | :--- | 34 | **No central servers** | Spanning tree self-organization | 35 | **No tracking** | No logs, no metadata collection, no accounts | 36 | **Encrypted tunnels** | ChaCha20-Poly1305 AEAD | 37 | **Censorship resistance** | FakeTLS masquerade + cover traffic | 38 | **Multi-hop routing** | Onion-wrapped frames via LCA path-building | 39 | **Group Messaging** | Circles: Epoch-keyed groups with broadcast | 40 41 ### Design Philosophy 42 43 Abzu follows the **Sovereign OS** principle: communication infrastructure should be something you *own*, not something you *rent* from a corporation. 44 45 ### Kerckhoffs' Principle 46 47 Security derives entirely from: 48 49 1. **Cryptographic primitives** (Ed25519, X25519, ChaCha20-Poly1305, BLAKE3) 50 2. **Protocol design** (this document) 51 3. **Key secrecy** (the *only* secret is your private key) 52 53 An adversary with complete access to this document and source code gains no advantage. 54 55 --- 56 57 ## Threat Model 58 59 ### Adversaries We Defend Against 60 61 | Adversary | Capability | Abzu Defense | 62 | :--- | :--- | :--- | 63 | **ISP/Carrier** | Traffic logging, DNS hijacking, IP blocking | FakeTLS masquerade, geometric routing | 64 | **State Actor** | CALEA compliance, metadata analysis | No central infrastructure, cryptographic addressing | 65 | **Passive Observer** | Traffic pattern analysis | Cover traffic, timing jitter, MTU padding | 66 | **Active Attacker** | MITM injection, connection hijacking | Ed25519 verification, AEAD encryption | 67 | **DPI Systems** | Protocol fingerprinting | TLS 1.3 wire format, no magic bytes | 68 69 ### Honest Limitations 70 71 > [!IMPORTANT] 72 > **Read This Section** 73 74 1. **Infrastructure-level attacks**: If your ISP physically disconnects you, no overlay helps. Abzu runs *over* the internet. 75 76 2. **Traffic analysis at scale**: Global timing correlation remains possible. Abzu adds latency noise but is not a full mix-network. 77 78 3. **Endpoint compromise**: Malware on your device defeats encryption. 79 80 4. **IANA/ARIN dependency**: IP addresses are centrally allocated. BGP-level blocking affects all overlay networks. 81 82 5. **CALEA compliance**: US carriers must enable lawful intercept. Abzu's defense: intercepted traffic is encrypted and appears as TLS noise. 83 84 --- 85 86 ## Architecture Overview 87 88 ```text 89 ┌─────────────────────────────────────────────────────────────────────┐ 90 │ Control Plane │ 91 │ JSON-RPC 2.0 Interface │ 92 └───────────────────────────────┬─────────────────────────────────────┘ 93 │ 94 ┌───────────────────────────────▼─────────────────────────────────────┐ 95 │ abzu-daemon │ 96 │ CLI • Configuration • RPC Server │ 97 └───────────────────────────────┬─────────────────────────────────────┘ 98 │ 99 ┌──────────────────────┼──────────────────────┐ 100 │ │ │ 101 ┌─────▼─────┐ ┌──────▼─────┐ ┌──────▼──────┐ 102 │ abzu-core │ │abzu-router │ │abzu-transport│ 103 │───────────│ │────────────│ │─────────────│ 104 │ Node │ │ RoutingTable│ │ AbzuFrame │ 105 │ Switchboard│ │ TreeCoords │ │ FakeTLS │ 106 │ Store │ │ Path Build │ │ Cover Traffic│ 107 │ Circles │ └────────────┘ └─────────────┘ 108 └───────────┘ 109 ``` 110 111 ### Crate Responsibilities 112 113 | Crate | Role | Key Modules | 114 | :--- | :--- | :--- | 115 | **abzu-core** | Node lifecycle, peer management, frame dispatch, Circles | `node.rs`, `switchboard.rs`, `store.rs` | 116 | **abzu-router** | Pure-logic routing (no I/O), tree coordinates | `table.rs`, `coords.rs`, `path.rs` | 117 | **abzu-transport** | Wire protocol, encryption, DPI evasion | `wire.rs`, `cover.rs`, `shaping.rs`, `fake_tls.rs` | 118 | **abzu-sdk** | High-level client API, FFI bindings | `lib.rs`, `identity.rs` | 119 | **abzu-daemon** | CLI binary, configuration, RPC | `main.rs` | 120 121 --- 122 123 ## Cryptographic Foundations 124 125 ### Identity: Ed25519 126 127 Each node generates or loads an Ed25519 keypair: 128 129 - **Private key**: 32 bytes, never transmitted 130 - **Public key**: 32 bytes, serves as node identifier 131 - **Signature**: 64 bytes, used for authentication 132 133 ### Session Encryption: X25519 + ChaCha20-Poly1305 134 135 Per-peer session keys established via: 136 137 1. **X25519 key exchange**: Ephemeral keypairs per connection 138 2. **HKDF-SHA256**: Derive symmetric key from shared secret 139 3. **ChaCha20-Poly1305**: All traffic encrypted with AEAD 140 141 This provides **Perfect Forward Secrecy**—compromise of long-term keys doesn't expose past traffic. 142 143 ### Group Encryption: ChaCha20-Poly1305 with Epoch Keys 144 145 Circle messages use symmetric encryption: 146 147 ```rust 148 fn derive_circle_key(circle_id: &[u8; 32], epoch: u64, members: &[[u8; 32]]) -> [u8; 32] { 149 let mut hasher = blake3::Hasher::new(); 150 hasher.update(b"abzu-circle-key"); 151 hasher.update(circle_id); 152 hasher.update(&epoch.to_be_bytes()); 153 for key in sorted(members) { 154 hasher.update(&key); 155 } 156 hasher.finalize().into() 157 } 158 ``` 159 160 **Properties:** 161 162 - **Deterministic**: All members derive same key independently 163 - **Epoch-bound**: Key rotates on membership change 164 - **Forward secrecy**: Removed members can't read new messages 165 - **Backward secrecy**: New members can't read old messages 166 167 ### Hashing: BLAKE3 168 169 Content addressing uses BLAKE3: 170 171 - **Output**: 32 bytes 172 - **Speed**: Fastest cryptographic hash available 173 - **Merkle support**: Built-in for chunked content 174 175 ### Address Derivation 176 177 Every Ed25519 public key deterministically maps to an IPv6 address in `0200::/7`: 178 179 ```rust 180 Address::from_public_key(&verifying_key) 181 // Yggdrasil algorithm: count leading 1-bits, embed in address 182 ``` 183 184 --- 185 186 ## Transport Layer 187 188 ### Tiered Security Model 189 190 | Tier | Name | Features | 191 | :--- | :--- | :--- | 192 | **0** | Off | Raw ChaCha20 encryption only | 193 | **1** | Blend | FakeTLS ClientHello + TLS record framing | 194 | **2** | Shadow | + MTU padding (1400 bytes) + timing jitter (50-500ms) | 195 | **3** | Ghost | + Adaptive cover traffic with pattern mimicry | 196 197 ### FakeTLS (`fake_tls.rs`) 198 199 Masquerades as TLS 1.3 to evade DPI: 200 201 1. **ClientHello**: Valid TLS 1.3 handshake with randomized session ID, SNI rotation 202 2. **Record framing**: All data wrapped in TLS Application Data records (`0x17 0x03 0x03`) 203 3. **Post-handshake**: ChaCha20-Poly1305 encrypted Abzu frames 204 205 ### Ghost Mode: Statistical Invisibility 206 207 Ghost mode makes cover traffic **cryptographically indistinguishable** from real traffic: 208 209 ```text 210 ┌────────────────────────────────────────────────────────────────┐ 211 │ On The Wire │ 212 │ • All frames encrypted identically │ 213 │ • No magic bytes, no distinguishing markers │ 214 │ • Cover and real traffic indistinguishable │ 215 └────────────────────────────────────────────────────────────────┘ 216 ▼ 217 ┌────────────────────────────────────────────────────────────────┐ 218 │ After Decryption │ 219 │ • Deserialize → AbzuFrame::Cover identified │ 220 │ • Switchboard silently discards │ 221 │ • No metrics pollution │ 222 └────────────────────────────────────────────────────────────────┘ 223 ``` 224 225 #### Security Hardening (January 2026) 226 227 | Vulnerability | Attack | Resolution | 228 | :--- | :--- | :--- | 229 | Serialization overhead | DPI filters by size offset | Overhead compensation in `AbzuFrame::cover()` | 230 | Echo attack | Probe with unique-sized packet | 16-byte bucket fuzzing in `record_size()` | 231 232 --- 233 234 ## Routing Layer 235 236 ### Tree Coordinates 237 238 Nodes self-organize into a **spanning tree** (Yggdrasil-inspired): 239 240 - **TreeCoords**: Path from root as port sequence `[1, 5, 3]` 241 - **Depth**: Distance from root 242 - **LCA**: Lowest Common Ancestor for path computation 243 244 ### Multi-Hop Path Building 245 246 Routes computed via LCA algorithm: 247 248 ```text 249 Source: [1,2,3] → LCA: [1] → Target: [1,5,6] 250 Path: UP(2) → DOWN(5) → DOWN(6) 251 ``` 252 253 **Stress-tested to 20+ hops** with verified path symmetry. 254 255 ### Onion-Style Frame Wrapping 256 257 Multi-hop packets use nested `Route` frames: 258 259 ```rust 260 AbzuFrame::wrap_onion(payload, target, &[hop1, hop2, hop3]) 261 ``` 262 263 Each intermediate node unwraps one layer and forwards. 264 265 --- 266 267 ## Circles: Group Messaging 268 269 Circles are **distributed state machines** for group communication (<100 members). 270 271 ### State Model 272 273 ```rust 274 pub struct Circle { 275 pub id: [u8; 32], // Random identifier 276 pub name: String, 277 pub epoch: u64, // Monotonic, increments on change 278 pub members: Vec<Member>, // Pubkey + role 279 pub symmetric_key: [u8; 32] // Derived from epoch 280 } 281 282 pub enum MemberRole { Founder, Admin, Member } 283 ``` 284 285 ### Group Encryption 286 287 All Circle messages encrypted with epoch-derived symmetric key: 288 289 ```rust 290 // Encrypt: nonce || ciphertext 291 pub fn encrypt_circle_message(circle_id, plaintext) -> Vec<u8> 292 293 // Decrypt: extract nonce, decrypt with circle key 294 pub fn decrypt_circle_message(circle_id, encrypted) -> Vec<u8> 295 ``` 296 297 ### Gossip Broadcast Protocol 298 299 Circles use **epidemic gossip** for O(log n) message propagation: 300 301 ```rust 302 // Gossip to √n random peers, clamped [2, 8] 303 pub async fn gossip_to_circle(circle_id, msg_id, frame) -> Result<usize> 304 ``` 305 306 **Algorithm:** 307 308 1. Check seen-set (5-min TTL) — skip if duplicate 309 2. Mark message as seen 310 3. Calculate fanout: `min(max(√n, 2), 8)` 311 4. Select random subset of members 312 5. Forward to each target 313 314 **Flow:** 315 316 1. Sender encrypts message with circle's symmetric key 317 2. Sender gossips `CircleMessage` to √n random members 318 3. Each recipient decrypts, stores, then re-gossips to their own √n 319 4. Recipients send `CircleAck` back to sender 320 321 **Repair Mechanism:** 322 323 `GossipHave` frames periodically announce message IDs for lazy-pull repair. 324 325 ### Membership Changes 326 327 When membership changes (`add_circle_member` / `remove_circle_member`): 328 329 1. Epoch increments 330 2. Symmetric key is re-derived 331 3. All members get new key automatically (deterministic derivation) 332 333 **Security guarantee**: Old members can't decrypt new messages; new members can't decrypt old messages. 334 335 ### Trust Policies: Privacy vs Accountability 336 337 Circles support **configurable trust policies** that determine the privacy-accountability trade-off: 338 339 | Policy | Ledger | Invite Tree | Use Case | 340 |--------|--------|-------------|----------| 341 | **Anonymous** (default) | ❌ | ❌ | Activists, whistleblowers, high-risk users | 342 | **Private** | ❌ | ✅ | Friend groups, private clubs | 343 | **Accountable** | ✅ | ✅ | Moderated communities, organizations | 344 345 **Anonymous** circles store only pubkeys—no invitation chains, no vouch history, no social graph. Designed for adversarial contexts where device seizure is a risk. 346 347 **Private** circles maintain invite trees (enabling moderation via `prune_branch`) but don't keep permanent audit logs. 348 349 **Accountable** circles maintain full trust ledgers with invite chains, vouch history, and prune records for complete auditability. 350 351 > [!NOTE] 352 > Default is **Anonymous**—safety-first design. See [CIRCLE_TRUST_POLICIES.md](./CIRCLE_TRUST_POLICIES.md) for full specification. 353 354 ### Wire Protocol (Circle Frames) 355 356 ```rust 357 CircleCreate { id: [u8; 32], name: Vec<u8> } 358 CircleInvite { circle_id: [u8; 32], invitee: [u8; 32] } 359 CircleJoin { circle_id: [u8; 32], member: [u8; 32] } 360 CircleLeave { circle_id: [u8; 32], member: [u8; 32] } 361 CircleMessage { circle_id: [u8; 32], id: u64, sender: [u8; 32], msg: Vec<u8>, timestamp: u64 } 362 CircleAck { circle_id: [u8; 32], msg_id: u64, member_key: [u8; 32] } 363 ``` 364 365 --- 366 367 ## Wire Protocol (Complete) 368 369 ```rust 370 pub enum AbzuFrame { 371 // Core 372 KeepAlive, 373 Cover { noise: Vec<u8> }, 374 375 // Routing 376 Route { target: [u8; 32], next_hop: [u8; 32], payload: Vec<u8> }, 377 378 // Content 379 Chunk { cid: [u8; 32], data: Vec<u8> }, 380 Request { cid: [u8; 32], requester: [u8; 32] }, 381 382 // 1:1 Chat 383 Chat { id: u64, to: [u8; 32], msg: Vec<u8>, timestamp: u64 }, 384 ChatAck { id: u64 }, 385 ReadReceipt { id: u64 }, 386 387 // Session 388 Hello { ephemeral_pub: [u8; 32], timestamp: u64 }, 389 HelloAck { ephemeral_pub: [u8; 32], confirmation: Vec<u8> }, 390 Announce { peer_key: [u8; 32], public_addr: Vec<u8>, nat_type: u8, timestamp: u64 }, 391 392 // Circles 393 CircleCreate { id: [u8; 32], name: Vec<u8> }, 394 CircleInvite { circle_id: [u8; 32], invitee: [u8; 32] }, 395 CircleJoin { circle_id: [u8; 32], member: [u8; 32] }, 396 CircleLeave { circle_id: [u8; 32], member: [u8; 32] }, 397 CircleMessage { circle_id: [u8; 32], id: u64, sender: [u8; 32], msg: Vec<u8>, timestamp: u64 }, 398 CircleAck { circle_id: [u8; 32], msg_id: u64, member_key: [u8; 32] }, 399 400 // Gossip (Epidemic Broadcast) 401 GossipHave { circle_id: [u8; 32], msg_ids: Vec<u64>, epoch: u64 }, 402 } 403 ``` 404 405 **Encoding**: Postcard (varint-compressed, `no_std` compatible) 406 407 --- 408 409 ## SDK API 410 411 ### Client Methods 412 413 | Method | Description | 414 | :--- | :--- | 415 | `create_circle(name)` | Create new Circle, become Founder | 416 | `list_circles()` | List all Circles | 417 | `add_circle_member(id, pubkey)` | Add member (rotates key) | 418 | `remove_circle_member(id, pubkey)` | Remove member (rotates key) | 419 | `send_circle_message(id, content)` | Encrypt + gossip to Circle | 420 | `gossip_to_circle(id, msg_id, frame)` | Gossip to √n random members | 421 | `get_circle_history(id, limit)` | Retrieve decrypted messages | 422 423 Full SDK documentation: [SDK_API_REFERENCE.md](./SDK_API_REFERENCE.md) 424 425 --- 426 427 ## Current Capabilities (v0.4) 428 429 ### Implemented ✓ 430 431 - Node lifecycle (create, run, graceful shutdown) 432 - Peer connections (FakeTLS, WebSocket) 433 - Full wire protocol (all frame types) 434 - Content-addressed storage with network discovery 435 - Encrypted 1:1 chat with delivery/read acknowledgments 436 - **Tiered security** (Blend, Shadow, Ghost modes) 437 - **Multi-hop routing** with onion-wrapped frames 438 - **Ghost mode security hardening** (overhead compensation, echo prevention) 439 - **Path building** with LCA algorithm (tested to 20 hops) 440 - **NAT traversal** (STUN discovery, UDP hole punching) 441 - **Perfect Forward Secrecy** (X25519 + HKDF-SHA256 handshake) 442 - **Universal SDK & Mobile FFI** (iOS/Android bindings via UniFFI) 443 - **Circles** (group messaging with epoch keys, gossip broadcast, encryption) 444 445 ### Test Coverage 446 447 **150+ tests** across the workspace: 448 449 | Crate | Tests | Coverage | 450 | :--- | :--- | :--- | 451 | abzu-core | 53 | Circles, messaging, storage, switchboard | 452 | abzu-router | 20+ | Tree coords, LCA, path building | 453 | abzu-transport | 70+ | Wire format, Ghost mode, cover traffic | 454 455 --- 456 457 ## Roadmap 458 459 > [!NOTE] 460 > Timelines are intention, not commitment. 461 462 ### Near-term 463 464 - [x] Bootstrap nodes for network entry — *Implemented* 465 - [/] Circle invitation protocol — *Frame defined, handler stub* 466 - [ ] DHT layer (`abzu-dht`) — *[Design approved](./DHT_DESIGN_SPECIFICATION.md)* 467 - [ ] WebSocket transport — *In progress* 468 469 ### Medium-term 470 471 - [ ] WebRTC transport (DHT-based signaling) 472 - [ ] UDP/QUIC transport 473 - [ ] Voice/media over Circles 474 475 ### Horizon 476 477 - [ ] LoRa transport (off-grid) 478 - [ ] Mix-network integration (Nym-style) 479 - [ ] Threshold cryptography for admin operations 480 481 --- 482 483 ## Design Principles 484 485 1. **Pure Logic Routing** — Routing layer performs no I/O 486 2. **Transport Agility** — Same logic over TCP, WebSocket, future transports 487 3. **Stealth First** — No magic bytes, no distinguishing headers 488 4. **Content Integrity** — All data verified against hash before use 489 5. **Local First** — Network for discovery/sync, not primary storage 490 6. **Post-Decrypt Discrimination** — Cover traffic only identifiable after decryption 491 7. **Epoch-Based Security** — Membership changes trigger key rotation 492 493 --- 494 495 ## References 496 497 | Project | Contribution | 498 | :--- | :--- | 499 | **Yggdrasil** | Spanning tree coordinates, sovereign IP derivation | 500 | **Iroh** | Content-addressed networking, BLAKE3 CIDs | 501 | **Reticulum** | Transport abstraction, embedded-first design | 502 | **Tor** | Onion routing concepts | 503 | **Nym** | Mix-network principles | 504 | **Signal** | Group messaging concepts | 505 506 --- 507 508 ## License 509 510 **Abzu Community License 1.3** — Adrian Murray, 2026 511 512 The core logic will be open sourced because **this needs to belong to everyone, not just one person**. 513 514 --- 515 516 > *"The best way to predict the future is to build it."* — Alan Kay