/ README.adoc
README.adoc
1 // SPDX-License-Identifier: AGPL-3.0-or-later 2 // SPDX-FileCopyrightText: 2025 Hyperpolymath Contributors 3 = Bebop-V-FFI 4 :toc: left 5 :toclevels: 3 6 :icons: font 7 :source-highlighter: rouge 8 :version: 0.1.0 9 :status: Active Development 10 11 > High-performance FFI bindings between Bebop binary serialization and the V programming language for Industrial IoT edge computing 12 13 image:https://img.shields.io/badge/RSR-Bronze-CD7F32[RSR Bronze] 14 image:https://img.shields.io/badge/TPCF-Perimeter%203-blue[TPCF Perimeter 3] 15 image:https://img.shields.io/badge/License-AGPL%20OR%20Palimpsest--0.8-green[License] 16 image:https://img.shields.io/badge/IIoT-Edge%20Computing-orange[IIoT Edge] 17 image:https://img.shields.io/badge/Kaldor-Integration-purple[Kaldor Integration] 18 19 == Overview 20 21 `bebop-v-ffi` provides a small, explicit FFI boundary that exposes the 22 https://bebop.sh[Bebop] binary serialization format to the https://vlang.io[V] 23 programming language. 24 25 The project defines a *stable C ABI* (the "FFI contract") and allows multiple 26 implementations behind it (initially Zig; Rust welcome), so that V applications 27 can decode and encode Bebop messages without reimplementing the wire format. 28 29 This library is a core component of the https://github.com/hyperpolymath/kaldor-iiot[Kaldor IIoT] ecosystem, enabling efficient message passing between resource-constrained edge devices (ESP32-C6, RISC-V microcontrollers) and backend services. 30 31 === Why "FFI" (not "bridge")? 32 33 This project is deliberately named `bebop-v-ffi` because the hard part here is not 34 "connecting two projects" — it's defining and maintaining a correct Foreign Function 35 Interface. 36 37 An FFI boundary is where you must be explicit about things that higher-level code 38 usually hides: 39 40 * *ABI stability* - calling conventions, alignment/padding 41 * *Ownership and lifetimes* - who frees what, when 42 * *Error propagation* - how failures cross the boundary 43 * *Message framing* - for networked data 44 45 Getting any of those wrong can produce silent corruption rather than a clean crash. 46 47 So this repo treats the *C ABI as the core deliverable*: a small, boring, stable 48 contract that V can rely on, while different implementations can evolve underneath 49 without changing user code. 50 51 === Current Status 52 53 [cols="2,1,3"] 54 |=== 55 |Component |Status |Notes 56 57 |ABI header 58 |**drafted** 59 |`include/bebop_v_ffi.h` 60 61 |V bindings 62 |**drafted** 63 |`v/bebop_bridge.v` 64 65 |Zig implementation 66 |**placeholder skeleton** 67 |`implementations/zig` 68 69 |Rust implementation 70 |**help wanted** 71 |`implementations/rust/README.md` 72 73 |Example framing 74 |**included** 75 |`v/examples/iiot_server.v`, `v/examples/iiot_client.v` 76 |=== 77 78 === Quickstart (conceptual) 79 80 1. Generate C bindings from schema with `bebopc --lang c` (see `scripts/gen_c_from_schema.sh`) 81 2. Build an implementation (Zig today) 82 3. Use the `bebop_bridge.v` module in `v/` directory from your V app 83 84 === Why Bebop? 85 86 https://bebop.sh[Bebop] is a schema-based binary serialization format optimized for real-time applications: 87 88 * **10-100x faster** than Protocol Buffers, MessagePack, and JSON 89 * **Zero-copy deserialization** - reads directly from wire format 90 * **Tiny code generation** - ~50KB for full runtime (ideal for microcontrollers) 91 * **Schema evolution** - backwards-compatible message updates 92 * **No reflection** - compile-time code generation, no runtime overhead 93 94 [source] 95 ---- 96 Benchmark: 1M messages (100-byte payload) 97 ┌────────────────┬───────────┬─────────────┐ 98 │ Format │ Serialize │ Deserialize │ 99 ├────────────────┼───────────┼─────────────┤ 100 │ Bebop │ 12ms │ 8ms │ 101 │ FlatBuffers │ 45ms │ 15ms │ 102 │ Protocol Bufs │ 180ms │ 220ms │ 103 │ MessagePack │ 350ms │ 410ms │ 104 │ JSON │ 890ms │ 1,200ms │ 105 └────────────────┴───────────┴─────────────┘ 106 ---- 107 108 === Why V? 109 110 https://vlang.io[V] is a systems programming language designed for reliability and performance: 111 112 * **Compiles to native code** - C backend, ~1MB binaries 113 * **No hidden allocations** - predictable memory usage 114 * **No garbage collector pauses** - manual memory management option 115 * **C interop** - seamless FFI with existing C libraries 116 * **Fast compilation** - ~1 second for most projects 117 * **Ideal for embedded** - targets ESP32, ARM, RISC-V 118 119 V fills the gap between Rust's complexity and C's lack of safety, making it perfect for IIoT firmware development. 120 121 === Why This FFI? 122 123 Kaldor IIoT needs to move telemetry data (temperature, humidity, loom status, spinning metrics) between: 124 125 * **Edge devices** (ESP32-C6 running V firmware) 126 * **Gateway nodes** (Raspberry Pi, RISC-V SBCs) 127 * **Backend services** (Deno/TypeScript, Rust WASM) 128 129 Bebop-V-FFI provides the glue layer, enabling: 130 131 [source] 132 ---- 133 ┌─────────────────┐ Bebop Binary ┌─────────────────┐ 134 │ V Firmware │◄────────────────────►│ Deno Backend │ 135 │ (ESP32-C6) │ ~8 bytes/msg │ (TypeScript) │ 136 └────────┬────────┘ └────────┬────────┘ 137 │ │ 138 │ Matter Protocol (Thread mesh) │ WASM Compute 139 │ │ 140 ▼ ▼ 141 ┌─────────────────┐ ┌─────────────────┐ 142 │ Gateway Node │ │ TimescaleDB │ 143 │ (RISC-V SBC) │ │ (Time-series) │ 144 └─────────────────┘ └─────────────────┘ 145 ---- 146 147 == Key Features 148 149 * **Zero-Copy Parsing** - Reads Bebop messages without intermediate allocations 150 * **Code Generation** - V source generated from `.bop` schema files 151 * **Memory Safe** - Bounds checking, no buffer overflows 152 * **Tiny Footprint** - <20KB compiled FFI layer 153 * **Cross-Platform** - Linux, macOS, Windows, embedded targets 154 * **Thread Safe** - No global state, safe for concurrent use 155 * **Offline-First** - Works completely air-gapped 156 * **RSR Compliant** - Rhodium Standard Repository Bronze tier 157 158 == Quick Start 159 160 === Installation 161 162 [source,bash] 163 ---- 164 # Clone repository 165 git clone https://github.com/hyperpolymath/bebop-v-ffi.git 166 cd bebop-v-ffi 167 168 # Build the FFI library 169 v build -prod src/ 170 171 # Run tests 172 v test tests/ 173 174 # Install Bebop compiler (for schema compilation) 175 # See: https://bebop.sh/guide/installation/ 176 ---- 177 178 === Define Schema 179 180 Create a `.bop` schema file: 181 182 [source,bebop] 183 ---- 184 // schemas/telemetry.bop 185 struct SensorReading { 186 uint32 device_id; 187 uint64 timestamp_ms; 188 float32 temperature_c; 189 float32 humidity_pct; 190 uint8 status_flags; 191 } 192 193 message LoomStatus { 194 1 -> uint32 loom_id; 195 2 -> string pattern_name; 196 3 -> uint32 picks_completed; 197 4 -> uint32 picks_total; 198 5 -> float32 efficiency_pct; 199 } 200 ---- 201 202 === Generate V Code 203 204 [source,bash] 205 ---- 206 # Generate V bindings from Bebop schema 207 bebop --generator v --input schemas/telemetry.bop --output src/telemetry.v 208 ---- 209 210 === Use in V 211 212 [source,v] 213 ---- 214 import bebop_ffi 215 import telemetry 216 217 fn main() { 218 // Create a sensor reading 219 reading := telemetry.SensorReading{ 220 device_id: 0x1001 221 timestamp_ms: u64(time.now().unix_milli()) 222 temperature_c: 23.5 223 humidity_pct: 65.2 224 status_flags: 0x01 225 } 226 227 // Serialize to Bebop binary (zero-copy) 228 buffer := bebop_ffi.encode(reading) 229 println('Serialized: ${buffer.len} bytes') 230 231 // Send over network/Matter protocol... 232 233 // Deserialize (zero-copy) 234 decoded := bebop_ffi.decode[telemetry.SensorReading](buffer) 235 println('Temperature: ${decoded.temperature_c}°C') 236 } 237 ---- 238 239 == Architecture 240 241 [source] 242 ---- 243 ┌─────────────────────────────────────────────────────────┐ 244 │ User Application │ 245 │ (V source code) │ 246 └─────────────────────────┬───────────────────────────────┘ 247 │ 248 ▼ 249 ┌─────────────────────────────────────────────────────────┐ 250 │ Generated V Bindings │ 251 │ (from .bop schemas via bebop compiler) │ 252 ├─────────────────────────────────────────────────────────┤ 253 │ struct SensorReading { ... } │ 254 │ fn encode(msg) -> []u8 │ 255 │ fn decode[T](buf) -> T │ 256 └─────────────────────────┬───────────────────────────────┘ 257 │ 258 ▼ 259 ┌─────────────────────────────────────────────────────────┐ 260 │ Bebop-V-FFI Core │ 261 │ (this library - V + C) │ 262 ├─────────────────────────────────────────────────────────┤ 263 │ - Buffer management (arena allocator) │ 264 │ - Endianness handling │ 265 │ - Bounds checking │ 266 │ - Zero-copy view creation │ 267 └─────────────────────────┬───────────────────────────────┘ 268 │ 269 ▼ 270 ┌─────────────────────────────────────────────────────────┐ 271 │ Bebop Runtime │ 272 │ (C library, ~50KB compiled) │ 273 └─────────────────────────────────────────────────────────┘ 274 ---- 275 276 === Design Rationale 277 278 **Why V instead of Rust for firmware?** 279 280 While Rust is superior for complex systems, V offers: 281 282 1. **Faster iteration** - 1s compile times vs 30s+ for Rust 283 2. **Simpler mental model** - No borrow checker complexity on microcontrollers 284 3. **Smaller binaries** - Critical for ESP32's 4MB flash 285 4. **C-like FFI** - Direct integration with ESP-IDF SDK 286 287 **Why FFI instead of pure V?** 288 289 Bebop's core runtime is C-based, battle-tested, and optimized. Wrapping it via FFI gives us: 290 291 1. **Proven serialization** - No bugs in the hot path 292 2. **Future-proofing** - Bebop updates flow through automatically 293 3. **Multi-language parity** - Same wire format across V, Rust, TypeScript 294 295 == Integration with Kaldor IIoT 296 297 Bebop-V-FFI is designed to work seamlessly with the Kaldor IIoT platform: 298 299 [source,v] 300 ---- 301 // firmware/main.v - ESP32-C6 Matter device 302 import matter 303 import bebop_ffi 304 import kaldor.telemetry 305 306 fn on_sensor_tick() { 307 reading := collect_sensor_data() 308 309 // Serialize with Bebop 310 payload := bebop_ffi.encode(reading) 311 312 // Send via Matter protocol (Thread mesh) 313 matter.publish("kaldor/telemetry", payload) 314 } 315 316 fn collect_sensor_data() telemetry.SensorReading { 317 return telemetry.SensorReading{ 318 device_id: matter.get_device_id() 319 timestamp_ms: u64(time.now().unix_milli()) 320 temperature_c: read_temperature_sensor() 321 humidity_pct: read_humidity_sensor() 322 status_flags: get_device_status() 323 } 324 } 325 ---- 326 327 === Kaldor Message Types 328 329 | Message Type | Size | Use Case | 330 |--------------|------|----------| 331 | `SensorReading` | 17 bytes | Environmental telemetry | 332 | `LoomStatus` | ~40 bytes | Weaving progress tracking | 333 | `SpinnerMetrics` | 24 bytes | Spinning node performance | 334 | `AlertEvent` | ~60 bytes | Fault notifications | 335 | `GossipHeartbeat` | 12 bytes | Mesh network health | 336 337 == Documentation 338 339 * *link:ROADMAP.adoc[ROADMAP.adoc]* - Development roadmap and milestones 340 * *link:CLAUDE.md[CLAUDE.md]* - AI assistant and developer guide 341 * *link:SECURITY.md[SECURITY.md]* - Security policy and vulnerability reporting 342 * *link:CONTRIBUTING.md[CONTRIBUTING.md]* - Contribution guidelines 343 * *link:docs/API.adoc[API Reference]* - Full API documentation 344 345 == Standards Compliance 346 347 === RSR Framework: Bronze Tier 348 349 Bebop-V-FFI meets *Rhodium Standard Repository (RSR) Bronze tier* requirements: 350 351 * Type safety (V compile-time guarantees) 352 * Memory safety (bounds checking, no raw pointers in public API) 353 * Offline-first (no network dependencies) 354 * Complete documentation 355 * `.well-known/` directory 356 * Build system (justfile) 357 * CI/CD pipeline 358 359 === TPCF: Perimeter 3 (Community Sandbox) 360 361 This project uses the *Tri-Perimeter Contribution Framework (TPCF)*: 362 363 * *Perimeter 1*: Core maintainers only 364 * *Perimeter 2*: Trusted contributors 365 * *Perimeter 3*: *Community Sandbox* - Open to all 366 367 All contributions are welcome! See link:CONTRIBUTING.md[CONTRIBUTING.md]. 368 369 == Building from Source 370 371 === Prerequisites 372 373 * V compiler 0.4.4+ (`v version`) 374 * C compiler (GCC 11+ or Clang 14+) 375 * Bebop compiler (`bebopc --version`) 376 * `just` command runner 377 378 === Build Commands 379 380 [source,bash] 381 ---- 382 # Build library 383 just build 384 385 # Build with optimizations 386 just build-release 387 388 # Run tests 389 just test 390 391 # Generate V bindings from schemas 392 just codegen 393 394 # Check RSR compliance 395 just validate-rsr 396 397 # Build for ESP32-C6 target 398 just build-esp32 399 ---- 400 401 == Testing 402 403 [source,bash] 404 ---- 405 # Run all tests 406 v test tests/ 407 408 # Run with verbose output 409 v test tests/ -stats 410 411 # Run specific test 412 v test tests/test_encode.v 413 414 # Benchmark serialization 415 v run benchmarks/bench_encode.v 416 ---- 417 418 == Performance 419 420 | Operation | Time (1M msgs) | Memory | 421 |-----------|----------------|--------| 422 | Encode SensorReading | 8ms | 0 allocs | 423 | Decode SensorReading | 5ms | 0 allocs | 424 | Encode LoomStatus | 12ms | 1 alloc (string) | 425 | Decode LoomStatus | 9ms | 1 alloc (string) | 426 427 **Binary sizes:** 428 429 * FFI library: ~18KB (stripped) 430 * Full firmware (with Matter): ~280KB 431 * Bebop runtime: ~50KB 432 433 == Security 434 435 See link:SECURITY.md[SECURITY.md] for: 436 437 * Supported versions 438 * Vulnerability reporting process 439 * Security best practices 440 441 == License 442 443 SPDX-License-Identifier: `AGPL-3.0-or-later OR LicenseRef-Palimpsest-0.8` 444 445 Dual licensed under your choice of: 446 447 * *AGPL-3.0-or-later* - GNU Affero General Public License 448 * *Palimpsest License v0.8* - Community ownership with reversibility 449 450 == Related Projects 451 452 * https://github.com/hyperpolymath/kaldor-iiot[kaldor-iiot] - Parent IIoT platform 453 * https://github.com/hyperpolymath/bunsenite[bunsenite] - Nickel config parser with similar FFI architecture 454 * https://bebop.sh[Bebop] - Binary serialization format 455 * https://vlang.io[V Language] - Systems programming language 456 457 == Roadmap 458 459 See link:ROADMAP.adoc[ROADMAP.adoc] for the full development roadmap including: 460 461 * *Phase 1*: Core FFI implementation and schema codegen 462 * *Phase 2*: Release and packaging (crates.io, vpkg) 463 * *Phase 3*: Next-gen Rust-V-Bebop hybrid implementation 464 465 == Help Wanted: Rust Implementation (plug-compatible) 466 467 I'd love a contributor to implement the same ABI in Rust (`staticlib`/`cdylib`), so V 468 users can choose Zig or Rust underneath. 469 470 *What you'd do:* 471 472 * Implement the functions in `include/bebop_v_ffi.h` 473 * Use `VBytes` (`ptr+len`) for all strings/bytes (*do not* assume NUL-terminated data) 474 * Match the framing + golden test vectors under `test-vectors/` 475 476 *Good first PR:* 477 478 * `bebop_decode_sensor_reading()` for `SensorReading` 479 * `bebop_ctx_new()` / `bebop_ctx_free()` and `bebop_free_reading()` 480 481 See `docs/60-contributing-implementations.adoc` for notes and pitfalls. 482 483 == Community 484 485 * *Issues*: https://github.com/hyperpolymath/bebop-v-ffi/issues 486 * *Discussions*: https://github.com/hyperpolymath/bebop-v-ffi/discussions 487 * *Kaldor Community*: https://github.com/hyperpolymath/kaldor-iiot/discussions 488 489 == Acknowledgments 490 491 * https://bebop.sh[Bebop Team] - For the excellent serialization format 492 * https://vlang.io[V Language Community] - For the pragmatic systems language 493 * https://github.com/hyperpolymath/kaldor-iiot[Kaldor Contributors] - For the IIoT vision 494 * RSR Framework contributors 495 * TPCF community 496 497 --- 498 499 *Version*: {version} + 500 *Status*: {status} + 501 *Last Updated*: 2025-12-24 502 503 _"Microseconds matter when you're weaving the future."_