1  # Integration Over Separation
  2  
  3  *proto-024 | Architectural pattern for adding functionality to existing systems*
  4  
  5  ---
  6  
  7  - **principle**
  8    - "When adding new functionality, prefer integrating as a layer into existing systems rather than creating separate processes."
  9    - "One daemon with layers beats multiple daemons with coordination overhead."
 10  
 11  - **shape**
 12    - New functionality becomes a layer within an existing component
 13    - Shared event streams and signal queues
 14    - Single operational surface (one thing to start/stop/monitor)
 15    - The integration IS the simplification
 16  
 17  ---
 18  
 19  ## The Insight
 20  
 21  When building new functionality that needs to observe or react to system events, there are two architectural choices:
 22  
 23  ```
 24  SEPARATE DAEMON APPROACH:
 25  ┌─────────────┐     ┌─────────────┐
 26  │ First       │     │ Building    │
 27  │ Officer     │────▶│ Steward     │
 28  │ (daemon 1)  │     │ (daemon 2)  │
 29  └─────────────┘     └─────────────┘
 30       │                    │
 31       │ duplicate watchers │
 32       │ coordination msgs  │
 33       └────────┬───────────┘
 34                ▼
 35           complexity
 36  
 37  INTEGRATION APPROACH:
 38  ┌─────────────────────────────────┐
 39  │         First Officer           │
 40  │  ┌───────────────────────────┐  │
 41  │  │   BuildingStewardLayer    │  │
 42  │  │   (observes same events)  │  │
 43  │  └───────────────────────────┘  │
 44  │         single daemon           │
 45  └─────────────────────────────────┘
 46                │
 47                ▼
 48           simplicity
 49  ```
 50  
 51  **The integration approach:**
 52  - No duplicate file watchers
 53  - No inter-process communication
 54  - Unified signal queue
 55  - Single operational surface
 56  - Shared state without serialization
 57  
 58  ---
 59  
 60  ## When to Apply
 61  
 62  **INTEGRATE when:**
 63  
 64  | Condition | Why Integration Wins |
 65  |-----------|---------------------|
 66  | New functionality observes same events | Avoid duplicate watchers |
 67  | New functionality emits to same consumers | Use existing signal path |
 68  | Operational simplicity matters | One thing to run, not two |
 69  | Shared state needed | Avoid serialization overhead |
 70  | Same lifecycle (start/stop together) | Single daemon management |
 71  
 72  **Example:** Building Steward observes all First Officer events to detect pattern references. Instead of running a separate daemon that watches the same files, it's a layer within First Officer that sees every `record_event()` call.
 73  
 74  ---
 75  
 76  ## When NOT to Apply
 77  
 78  **SEPARATE when:**
 79  
 80  | Condition | Why Separation Wins |
 81  |-----------|---------------------|
 82  | Different failure domains | One can crash without killing the other |
 83  | Different scaling needs | Can run multiple instances independently |
 84  | Different teams own them | Organizational boundaries matter |
 85  | Different deployment cycles | Can update independently |
 86  | Fundamentally different concerns | Not actually related functionality |
 87  
 88  **Example:** Mission Control synthesizes across threads and writes daily synthesis. It has a different lifecycle (runs once per synthesis, not continuously) and different failure domain. It's a separate daemon.
 89  
 90  ---
 91  
 92  ## Implementation Pattern
 93  
 94  ```python
 95  class HostComponent:
 96      """The existing component that will host the new layer."""
 97  
 98      def __init__(self):
 99          # ... existing init ...
100  
101          # Add new functionality as a layer
102          self._new_layer = NewFunctionalityLayer()
103  
104      def process_event(self, event):
105          """Existing event processing."""
106          # ... existing logic ...
107  
108          # Let the new layer observe the same event
109          signals = self._new_layer.observe(event)
110          for signal in signals:
111              self._emit_signal(signal)
112  
113  
114  class NewFunctionalityLayer:
115      """The new functionality, designed as a layer not a daemon."""
116  
117      def observe(self, event) -> List[Signal]:
118          """
119          Called by host on every event.
120          Returns signals to emit (or empty list).
121          """
122          signals = []
123  
124          # React to event if relevant
125          if self._is_relevant(event):
126              signals.append(self._create_signal(event))
127  
128          return signals
129  ```
130  
131  **Key design points:**
132  1. Layer has `observe()` method called by host
133  2. Layer returns signals, doesn't emit directly
134  3. Host decides when/how to emit
135  4. Layer has no lifecycle of its own
136  
137  ---
138  
139  ## The Test
140  
141  Before creating a separate daemon, ask:
142  
143  1. **Does it watch the same things?** → Integrate
144  2. **Does it emit to the same consumers?** → Integrate
145  3. **Does it have the same lifecycle?** → Integrate
146  4. **Is operational simplicity important?** → Integrate
147  
148  If all four answers are "yes," integration is the right choice.
149  
150  ---
151  
152  ## Axiom Alignment
153  
154  | Axiom | Alignment |
155  |-------|-----------|
156  | **A0 (Boundary)** | Integration draws ONE boundary around related functionality, not two |
157  | **A1 (Integration)** | Directly embodies "move toward connection, not isolation" |
158  | **A3 (Navigation)** | This IS a navigation decision - sometimes separate is right |
159  | **A4 (Ergodicity)** | Fewer daemons = fewer things that can fail = lower ruin risk |
160  
161  ---
162  
163  ## The Meta-Insight
164  
165  > **A1 (Telos of Integration) applies to architecture, not just content.**
166  >
167  > Satan didn't know he was choosing isolation when he created a separate daemon.
168  > The coordination overhead, the duplicate watchers, the operational complexity—
169  > these are the isolation costs.
170  >
171  > **Integration is the default. Separation requires justification.**
172  
173  ---
174  
175  ## Instances
176  
177  ### Positive Instance: Building Steward
178  - **Context:** Needed pattern lifecycle tracking + architecture guidance
179  - **Decision:** Integrated as `BuildingStewardLayer` within `FirstOfficer`
180  - **Result:** Single daemon, shared event stream, unified signals
181  - **Outcome:** ✓ Simpler operations, no coordination overhead
182  
183  ### Negative Instance (What We Avoided): Separate Building Steward Daemon
184  - **Alternative:** `building_steward_daemon.py` running separately
185  - **Problems it would have caused:**
186    - Duplicate file watchers
187    - Need to serialize events between processes
188    - Two things to start/stop
189    - Race conditions on shared state
190  - **Outcome:** ✗ Avoided this complexity
191  
192  ### Positive Instance: Mission Control (Appropriate Separation)
193  - **Context:** Cross-thread synthesis, different lifecycle
194  - **Decision:** Separate daemon that runs periodically
195  - **Justification:** Different lifecycle (periodic vs. continuous), different failure domain
196  - **Outcome:** ✓ Appropriate separation, each component has single responsibility
197  
198  ---
199  
200  ## Related
201  
202  - [[first-officer-protocol]] - The host component for Building Steward
203  - [[A1-telos-of-integration]] - The axiom this pattern embodies
204  - [[transparent-automation-principle]] - Layer works automatically, CLI for override
205  - [[typed-resonance-principle]] - Another layer integrated into First Officer
206  
207  ---
208  
209  *proto-024 | Integration Over Separation | 2026-01-15*