/ libp2p-efi-debootd-integration-plan.md
libp2p-efi-debootd-integration-plan.md
1 --- 2 Integration Plan: libp2p.efi + debootd 3 4 Architecture Overview 5 6 ┌─────────────────────────────────────────────────────────────────────────┐ 7 │ Development Setup │ 8 ├─────────────────────────────────────────────────────────────────────────┤ 9 │ │ 10 │ ┌─────────────────────┐ ┌─────────────────────────────────┐ │ 11 │ │ QEMU + OVMF │ UDP │ Host OS (your machine) │ │ 12 │ │ │ ◄─────► │ │ │ 13 │ │ ┌───────────────┐ │ libp2p │ ┌───────────────────────────┐ │ │ 14 │ │ │ SWFETCH.EFI │ │ QUIC │ │ SwarmISH Bridge (Go) │ │ │ 15 │ │ │ (libp2p.efi │ │ │ │ │ │ │ 16 │ │ │ + fetch glue)│ │ │ │ - Speaks libp2p to EFI │ │ │ 17 │ │ └───────────────┘ │ │ │ - Fetches from Swarm │ │ │ 18 │ │ │ │ │ │ - Serves kernel/initramfs│ │ │ 19 │ │ ▼ │ │ │ - Caches locally │ │ │ 20 │ │ [Boot artifacts] │ │ └───────────────────────────┘ │ │ 21 │ │ vmlinuz │ │ │ │ │ 22 │ │ initramfs.img │ │ ▼ │ │ 23 │ │ os-image.squashfs │ │ ┌─────────────┐ │ │ 24 │ └─────────────────────┘ │ │ Swarm Node │ │ │ 25 │ │ │ (optional) │ │ │ 26 │ │ └─────────────┘ │ │ 27 └─────────────────────────────────────────────────────────────────────────┘ 28 29 --- 30 Phase 1: SwarmISH Bridge (Go) - Week 1 31 32 Goal: Create a libp2p-speaking bridge that serves content to EFI 33 34 1.1 Create bridge project structure 35 36 debootd/swarmish-bridge/ 37 ├── go.mod 38 ├── main.go # Entry point, libp2p host setup 39 ├── handlers/ 40 │ └── fetch.go # Protocol handler for /deboot/fetch/1.0.0 41 ├── storage/ 42 │ ├── local.go # Serve from local filesystem (dev mode) 43 │ └── swarm.go # Fetch from real Swarm (future) 44 └── manifest/ 45 └── manifest.go # Parse host-configuration-manifest.json 46 47 1.2 Bridge responsibilities 48 49 - Listen on libp2p (QUIC transport, same as test-server) 50 - Implement custom protocol /deboot/fetch/1.0.0 51 - Protocol messages: 52 REQUEST: { "type": "manifest" } → returns host config manifest 53 REQUEST: { "hash": "bzz://..." } → returns raw bytes of artifact 54 RESPONSE: { "ok": true, "data": <bytes> } or { "ok": false, "error": "..." } 55 - Dev mode: serve from debootd/image/ directory (vmlinuz, initramfs.img, os-image.squashfs) 56 - Production mode: fetch from actual Swarm node 57 58 1.3 Implementation starting point 59 60 Extend libp2p/test-server/main.go: 61 // Add fetch protocol handler 62 host.SetStreamHandler("/deboot/fetch/1.0.0", handleFetchRequest) 63 64 func handleFetchRequest(s network.Stream) { 65 // Read request (JSON) 66 // Lookup artifact (local file or Swarm) 67 // Stream response back 68 } 69 70 --- 71 Phase 2: SWFETCH.EFI - Week 1-2 72 73 Goal: Create a fetch-capable EFI module building on libp2p.efi 74 75 2.1 File structure 76 77 libp2p/src/ 78 ├── main.rs # Existing entry point 79 ├── fetch/ # NEW: Fetch protocol implementation 80 │ ├── mod.rs 81 │ ├── protocol.rs # /deboot/fetch/1.0.0 client 82 │ └── manifest.rs # Parse manifest JSON 83 └── boot/ # NEW: Boot handoff logic 84 ├── mod.rs 85 ├── loader.rs # Load kernel/initramfs into memory 86 └── handoff.rs # UEFI boot services exit, jump to kernel 87 88 2.2 SWFETCH.EFI boot sequence 89 90 1. Initialize UEFI SNP networking 91 2. Generate libp2p identity (existing) 92 3. Connect to SwarmISH bridge at hardcoded IP (dev mode) 93 - e.g., 10.0.2.2:4001 (QEMU host) 94 4. Open stream, negotiate /deboot/fetch/1.0.0 95 5. Request manifest → parse → get artifact hashes 96 6. Request vmlinuz → load to memory at 0x1000000 97 7. Request initramfs → load to memory at 0x3000000 98 8. (Optional) Request squashfs → write to disk partition 99 9. Set up Linux boot params (zero page) 100 10. Exit boot services 101 11. Jump to kernel entry point 102 103 2.3 Key additions to libp2p.efi 104 105 // src/fetch/protocol.rs 106 pub struct FetchClient<'a> { 107 quic: &'a mut QuicClient, 108 stream_id: u64, 109 } 110 111 impl FetchClient { 112 pub fn request_manifest(&mut self) -> Result<Manifest, Error> { ... } 113 pub fn fetch_artifact(&mut self, hash: &str, dest: &mut [u8]) -> Result<usize, Error> { ... } 114 } 115 116 // src/boot/loader.rs 117 pub fn load_linux_kernel( 118 boot_services: &BootServices, 119 kernel_data: &[u8], 120 initramfs_data: &[u8], 121 cmdline: &str, 122 ) -> Result<(), Error> { ... } 123 124 --- 125 Phase 3: Integration Testing - Week 2 126 127 3.1 Test harness 128 129 debootd/integration-test/ 130 ├── run-test.sh # Orchestrates full boot test 131 ├── expected-output.txt # Expected serial console output 132 └── verify-boot.sh # Checks if OS booted successfully 133 134 3.2 Test sequence 135 136 #!/bin/bash 137 # run-test.sh 138 139 # 1. Build all artifacts 140 (cd ../image && ./build.sh) 141 (cd ../../libp2p && make build) 142 143 # 2. Start SwarmISH bridge in background 144 (cd ../swarmish-bridge && go run main.go) & 145 BRIDGE_PID=$! 146 147 # 3. Create disk with ESP containing SWFETCH.EFI 148 ./create-test-disk.sh 149 150 # 4. Launch QEMU 151 qemu-system-x86_64 \ 152 -bios /usr/share/OVMF/OVMF_CODE.fd \ 153 -drive file=test-disk.img,format=raw \ 154 -netdev user,id=net0,hostfwd=udp::9999-:9999 \ 155 -device virtio-net-pci,netdev=net0 \ 156 -serial stdio | tee boot.log & 157 QEMU_PID=$! 158 159 # 5. Wait for boot, check for success markers 160 sleep 30 161 grep "debootd ready" boot.log && echo "SUCCESS" || echo "FAILED" 162 163 # 6. Cleanup 164 kill $BRIDGE_PID $QEMU_PID 165 166 --- 167 Phase 4: Debootd Daemon Integration - Week 3 168 169 4.1 Connect debootd to SwarmISH bridge 170 171 The debootd daemon (currently a stub) needs to: 172 1. Start its own libp2p host 173 2. Connect to SwarmISH bridge for updates 174 3. Monitor for ENS changes (simulated initially) 175 176 4.2 State flow 177 178 ┌──────────────────────────────────────────────────────────────────┐ 179 │ Full Boot Flow │ 180 ├──────────────────────────────────────────────────────────────────┤ 181 │ │ 182 │ SWFETCH.EFI │ 183 │ │ │ 184 │ ├─► Connect to bridge │ 185 │ ├─► Fetch manifest │ 186 │ ├─► Fetch kernel + initramfs │ 187 │ ├─► Write boot attestation to ESP │ 188 │ └─► Boot Linux │ 189 │ │ │ 190 │ ▼ │ 191 │ Initramfs │ 192 │ │ │ 193 │ ├─► Read /meta/slots.json │ 194 │ ├─► Mount active OS slot (squashfs) │ 195 │ └─► switch_root to systemd │ 196 │ │ │ 197 │ ▼ │ 198 │ debootd daemon │ 199 │ │ │ 200 │ ├─► Mark boot successful │ 201 │ ├─► Connect to SwarmISH bridge (libp2p) │ 202 │ ├─► Monitor for updates │ 203 │ └─► On update: fetch to inactive slot → soft-reboot │ 204 │ │ 205 └──────────────────────────────────────────────────────────────────┘ 206 207 --- 208 Phase 5: Swap in Real Swarm - Week 4+ 209 210 5.1 SwarmISH bridge enhancement 211 212 Add real Swarm backend: 213 // storage/swarm.go 214 type SwarmStorage struct { 215 beeClient *bee.Client // Connect to local or remote Bee node 216 } 217 218 func (s *SwarmStorage) Fetch(hash string) ([]byte, error) { 219 // Parse bzz:// URL 220 // Call Bee API to retrieve 221 // Return content 222 } 223 224 5.2 Real SWFETCH.EFI (later) 225 226 Once bridge is proven, implement native Swarm in EFI: 227 - Embed Swarm chunking/BMT verification 228 - Connect directly to Swarm peers 229 - No bridge needed 230 231 --- 232 Immediate Next Steps 233 ┌─────────────────────────────────────────────────┬───────┬──────────┐ 234 │ Task │ Owner │ Priority │ 235 ├─────────────────────────────────────────────────┼───────┼──────────┤ 236 │ Create swarmish-bridge/ skeleton │ You │ P0 │ 237 ├─────────────────────────────────────────────────┼───────┼──────────┤ 238 │ Add /deboot/fetch/1.0.0 protocol to bridge │ You │ P0 │ 239 ├─────────────────────────────────────────────────┼───────┼──────────┤ 240 │ Add src/fetch/ module to libp2p.efi │ You │ P0 │ 241 ├─────────────────────────────────────────────────┼───────┼──────────┤ 242 │ Test EFI ↔ Bridge connectivity │ You │ P1 │ 243 ├─────────────────────────────────────────────────┼───────┼──────────┤ 244 │ Add kernel loading to SWFETCH.EFI │ You │ P1 │ 245 ├─────────────────────────────────────────────────┼───────┼──────────┤ 246 │ Create integration test harness │ You │ P2 │ 247 ├─────────────────────────────────────────────────┼───────┼──────────┤ 248 │ Replace debootd-stub with libp2p-enabled daemon │ You │ P2 │ 249 └─────────────────────────────────────────────────┴───────┴──────────┘ 250 --- 251 File Changes Summary 252 253 New files to create: 254 debootd/swarmish-bridge/ 255 ├── go.mod 256 ├── main.go 257 └── handlers/fetch.go 258 259 libp2p/src/fetch/ 260 ├── mod.rs 261 ├── protocol.rs 262 └── manifest.rs 263 264 Files to modify: 265 libp2p/src/main.rs # Add fetch + boot flow 266 libp2p/Makefile # Add swfetch target 267 debootd/vm/run-qemu.sh # Support SWFETCH.EFI boot 268 269 --- 270 Would you like me to start implementing any of these components? I'd suggest starting with the SwarmISH bridge since it unblocks EFI development.