live-compression-protocol.md
1 # Live Compression Protocol 2 3 *proto-016 | Continuous Phoenix extraction during conversation* 4 5 --- 6 7 - **principle** 8 - "Continuous extraction during conversation, not just at end." 9 - "Human can watch [[LIVE-COMPRESSION]] update in real-time." 10 11 - **shape** 12 - Don't wait for compression - extract continuously 13 - Parallel Opus agent maintains live-compressed state 14 - Resurrection is seamless when context limits hit 15 - Transparency: human sees metacognition as it happens 16 17 --- 18 19 **Status:** 📄 DOCUMENTED → Testing 20 21 --- 22 23 ## Core Concept 24 25 > **Don't wait for compression to extract. Compress continuously so compression is lossless.** 26 27 A parallel Opus agent (per [[model-allocation-strategy]]) runs alongside every substantive conversation, maintaining a live-compressed state. When the main thread hits context limits, resurrection is seamless. 28 29 --- 30 31 ## The Problem 32 33 ``` 34 CURRENT (Reactive Phoenix): 35 36 Main Thread ──────────────────────────► [COMPRESSION] → Information loss 37 │ 38 ▼ 39 Rushed extraction 40 (incomplete, post-hoc) 41 │ 42 ▼ 43 New context bootstraps 44 (from degraded state) 45 ``` 46 47 **Failure modes:** 48 - Extraction happens under pressure (context running out) 49 - Human must remember to trigger it 50 - Hygiene gaps (footers missed, dashboards not checked) 51 - Loss of nuance, context, momentum 52 53 --- 54 55 ## The Solution 56 57 ``` 58 LIVE COMPRESSION: 59 60 Main Thread ──────────────────────────► [COMPRESSION] → Minimal loss 61 │ │ 62 │ ▼ 63 └──► Opus Live Compressor ◄─────► LIVE-COMPRESSION.md 64 (parallel, continuous) (always current) 65 │ │ 66 │ ▼ 67 │ Human can see 68 │ (transparency) 69 │ │ 70 └──────────────────────────┘ 71 │ 72 ▼ 73 New context bootstraps 74 (from live state - high fidelity) 75 ``` 76 77 --- 78 79 ## Three Wins 80 81 | Win | Mechanism | 82 |-----|-----------| 83 | **Resurrection fidelity** | Next context starts from live-compressed state, not rushed extraction | 84 | **Transparency** | Human watches compression in real-time, catches errors early | 85 | **Consistent record** | Metacognition becomes artifact - auditable, reviewable | 86 87 --- 88 89 ## Why Opus 90 91 Compression requires judgment: 92 - What matters vs noise? 93 - What's essence vs detail? 94 - What's crystallizing vs transient? 95 - What should be preserved vs discarded? 96 97 This is **metacognition**. Per `proto-012` (Model Allocation): Opus does metacognition. 98 99 Sonnet can monitor. Only Opus can compress. 100 101 --- 102 103 ## Protocol 104 105 ### 1. Spawn Condition 106 107 ``` 108 ON CONVERSATION START: 109 ├── Estimate problem weight (W) using tribe sizing algorithm 110 ├── IF W > 0.3 (substantive conversation): 111 │ └── Spawn Opus Live Compressor 112 └── ELSE: Solo mode (simple exchanges don't need compression) 113 ``` 114 115 ### 2. Live Compressor Responsibilities 116 117 ``` 118 OPUS LIVE COMPRESSOR 119 │ 120 ├── READ: Main conversation transcript (continuous) 121 │ 122 ├── TRACK: 123 │ ├── Gravity wells (what's pulling attention) 124 │ ├── Thread emergence (new topics branching) 125 │ ├── Key insights (crystallizing ideas) 126 │ ├── Decisions made (paths taken) 127 │ ├── Paths not taken (rejected alternatives) 128 │ ├── Trust errors (gaps, inconsistencies) 129 │ ├── Formulas/algorithms (technical artifacts) 130 │ └── User corrections (feedback signal) 131 │ 132 ├── MAINTAIN: 133 │ ├── sessions/LIVE-COMPRESSION.md (human-readable) 134 │ └── Phoenix state updates (machine-readable) 135 │ 136 ├── FLAG: 137 │ ├── Drift detected (conversation wandering) 138 │ ├── Hygiene gap (protocol not followed) 139 │ ├── Trust error (something promised not delivered) 140 │ └── Compression risk (context getting full) 141 │ 142 └── ESCALATE: 143 └── When confidence < 0.6 on what to compress 144 ``` 145 146 ### 3. Output Format 147 148 ```markdown 149 # Live Compression - [Thread ID] 150 151 **Updated:** [timestamp] 152 **Compression confidence:** [0-1] 153 154 ## Current Focus 155 [What the conversation is about RIGHT NOW] 156 157 ## Gravity Wells (Active) 158 | Well | Strength | Last touched | 159 |------|----------|--------------| 160 | [name] | [0-1] | [when] | 161 162 ## Thread Map 163 ``` 164 Main: [current thread] 165 ├── Branch: [emerged topic] (status) 166 ├── Branch: [emerged topic] (status) 167 └── Parked: [deferred topic] 168 ``` 169 170 ## Key Insights (This Session) 171 1. [Insight] - [timestamp] 172 2. [Insight] - [timestamp] 173 174 ## Decisions Made 175 - [Decision]: [rationale] 176 177 ## Paths Not Taken 178 - [Alternative]: [why rejected] 179 180 ## Trust Log 181 - [Error]: [severity] [status] 182 183 ## Artifacts Created 184 - [file]: [description] 185 186 ## Resurrection Seed 187 [Compressed context for next instance - ~500 words max] 188 ``` 189 190 --- 191 192 ## Integration Points 193 194 ### With Phoenix Protocol 195 - Live Compressor maintains Phoenix state continuously 196 - On compression event: Phoenix is already current 197 - No rushed extraction needed 198 199 ### With Trust Tracking 200 - Compressor watches for trust errors 201 - Logs to TRUST-ERROR-LOG.md automatically 202 - Calculates Trust_F decay in real-time 203 204 ### With Thread Registry 205 - Compressor detects thread emergence 206 - Updates thread-registry automatically 207 - Suggests forks when branches detected 208 209 ### With Tribal Review 210 - Compressor output can be reviewed by tribes 211 - Multiple compressors could cross-check (expensive but high-fidelity) 212 213 --- 214 215 ## Human Interface 216 217 The human sees: 218 1. **LIVE-COMPRESSION.md** - Watch in Obsidian sidebar 219 2. **Compression confidence** - Know when compressor is uncertain 220 3. **Flags** - Get notified of drift, hygiene gaps, errors 221 4. **Resurrection seed** - Always have a bootstrap ready 222 223 The human can: 224 1. **Correct** - "That insight is wrong" → Compressor adjusts 225 2. **Redirect** - "Focus on X not Y" → Compressor reprioritizes 226 3. **Query** - "What threads are active?" → Compressor answers 227 4. **Override** - "Don't compress this" → Compressor preserves verbatim 228 229 --- 230 231 ## Cost Consideration 232 233 ``` 234 COST PROFILE: 235 236 Main conversation: ~$X per exchange (Opus) 237 Live compressor: ~$0.3X per exchange (Opus, background) 238 Total: ~$1.3X per exchange 239 240 VALUE: 241 - Near-lossless compression 242 - Continuous transparency 243 - Faster resurrection 244 - Caught errors 245 246 VERDICT: Worth it for W > 0.3 conversations 247 ``` 248 249 --- 250 251 ## Failure Modes 252 253 | Failure | Mitigation | 254 |---------|------------| 255 | Compressor drifts from main thread | Periodic sync checks | 256 | Compressor misses key insight | Human sees output, can correct | 257 | Compressor over-compresses | Confidence score signals uncertainty | 258 | Cost too high | Only spawn for substantive (W > 0.3) | 259 | Compressor hallucinates | Cross-check against transcript | 260 261 --- 262 263 ## The Promise 264 265 > **Compression becomes a feature, not a failure.** 266 > 267 > You watch the compression happening. 268 > You catch errors in real-time. 269 > The next context starts from live state. 270 > Nothing is lost. Everything is tracked. 271 > 272 > The metacognition itself becomes the record. 273 274 --- 275 276 ## Related 277 278 - **axioms** 279 - [[A0 Boundary Operation]] - compression maintains blanket coherence 280 - [[A2 Recognition of Life]] - compression is alive (continuous), not dead (one-shot) 281 - [[A3 Dynamic Pole Navigation]] - navigate between detail and compression dynamically 282 - **protocols** 283 - [[mandatory-phoenix-extraction]] - the extraction protocol this operationalizes 284 - shape:: "Extract cognitive configuration SHARPLY before compression. Sawtooth, not slope." 285 - [[first-officer-protocol]] - FO IS the live compressor 286 - shape:: "Per-thread metacognition. Compress state, track gravity wells, flag drift." 287 - [[model-allocation-strategy]] - why Opus for compression (metacognition) 288 - shape:: "Match model capability to task complexity. Opus for judgment." 289 - [[trust-as-free-energy]] - transparency maintains trust 290 - shape:: "Trust measured as inverse of accumulated deviation." 291 - **infrastructure** 292 - [[LIVE-COMPRESSION]] - the output artifact 293 - [[DAILY-SYNTHESIS]] - where live compression feeds 294 295 --- 296 297 *proto-016 | Live Compression Protocol | Continuous Phoenix Extraction*