architecture.md
1 # Architecture 2 3 OpenCLI is built on a **Dual-Engine Architecture** that supports both declarative pipelines and programmatic TypeScript adapters. 4 5 ## High-Level Architecture 6 7 ``` 8 ┌─────────────────────────────────────────────────────┐ 9 │ opencli CLI │ 10 │ (Commander.js entry point) │ 11 ├─────────────────────────────────────────────────────┤ 12 │ Engine Layer │ 13 │ ┌──────────────┐ ┌──────────────┐ ┌────────────┐ │ 14 │ │ Registry │ │ Dynamic │ │ Output │ │ 15 │ │ (commands) │ │ Loader │ │ Formatter │ │ 16 │ └──────────────┘ └──────────────┘ └────────────┘ │ 17 ├─────────────────────────────────────────────────────┤ 18 │ Adapter Layer │ 19 │ ┌─────────────────┐ ┌──────────────────────────┐ │ 20 │ │ Pipeline │ │ TypeScript Adapters │ │ 21 │ │ (declarative) │ │ (browser/desktop/AI) │ │ 22 │ └─────────────────┘ └──────────────────────────┘ │ 23 ├─────────────────────────────────────────────────────┤ 24 │ Connection Layer │ 25 │ ┌─────────────────┐ ┌──────────────────────────┐ │ 26 │ │ Browser Bridge │ │ CDP (Chrome DevTools) │ │ 27 │ │ (Extension+WS) │ │ (Electron apps) │ │ 28 │ └─────────────────┘ └──────────────────────────┘ │ 29 └─────────────────────────────────────────────────────┘ 30 ``` 31 32 ## Core Modules 33 34 ### Registry (`src/registry.ts`) 35 Central command registry. All adapters register their commands via the `cli()` function with metadata: site, name, description, domain, strategy, args, columns. 36 37 ### Discovery (`src/discovery.ts`) 38 CLI discovery and manifest loading. Discovers commands from TypeScript adapter files, parses pipelines, and registers them into the central registry. 39 40 ### Execution (`src/execution.ts`) 41 Command execution: argument validation, lazy loading of adapter modules, and executing the appropriate handler function. 42 43 ### Commander Adapter (`src/commanderAdapter.ts`) 44 Bridges the Registry commands to Commander.js subcommands. Handles positional args, named options, browser session wiring, and output formatting. Isolates all Commander-specific logic so the core is framework-agnostic. 45 46 ### Browser (`src/browser.ts`) 47 Manages connections to Chrome via the Browser Bridge WebSocket daemon. Handles JSON-RPC messaging, tab management, and extension/standalone mode switching. 48 49 ### Pipeline (`src/pipeline/`) 50 The pipeline engine. Processes declarative steps: 51 - **fetch** — HTTP requests with cookie/header strategies 52 - **map** — Data transformation with template expressions 53 - **limit** — Result truncation 54 - **filter** — Conditional filtering 55 - **download** — Media download support 56 57 ### Output (`src/output.ts`) 58 Unified output formatting: `table`, `json`, `yaml`, `md`, `csv`. 59 60 ## Authentication Strategies 61 62 OpenCLI uses a 3-tier authentication strategy: 63 64 | Strategy | How It Works | When to Use | 65 |----------|-------------|-------------| 66 | `public` | Direct HTTP fetch, no auth | Public APIs (HackerNews, BBC) | 67 | `cookie` | Reuse Chrome cookies via Browser Bridge | Logged-in sites (Bilibili, Zhihu) | 68 | `header` | Custom auth headers | API-key based services | 69 | `intercept` | Network request interception | GraphQL/XHR capture (Twitter) | 70 | `ui` | DOM interaction via accessibility snapshot | Desktop apps, write operations | 71 72 ## Directory Structure 73 74 ``` 75 src/ 76 ├── main.ts # Entry point 77 ├── cli.ts # Commander.js CLI setup + built-in commands 78 ├── commanderAdapter.ts # Registry → Commander bridge 79 ├── discovery.ts # CLI discovery, manifest loading 80 ├── execution.ts # Arg validation, command execution 81 ├── registry.ts # Command registry 82 ├── serialization.ts # Command serialization helpers 83 ├── runtime.ts # Browser session & timeout management 84 ├── browser/ # Browser Bridge connection 85 ├── output.ts # Output formatting 86 ├── doctor.ts # Diagnostic tool 87 ├── pipeline/ # Pipeline engine 88 │ ├── runner.ts 89 │ ├── template.ts 90 │ ├── transform.ts 91 │ └── steps/ 92 │ ├── fetch.ts 93 │ ├── map.ts 94 │ ├── limit.ts 95 │ ├── filter.ts 96 │ └── download.ts 97 └── clis/ # Site adapters 98 ├── twitter/ 99 ├── reddit/ 100 ├── bilibili/ 101 ├── cursor/ 102 └── ... 103 ```