Architecture.md
1 # Arti: Architectural notes and outline. 2 3 ## Guidelines 4 5 I'm hoping to have small, optional, separable pieces here. 6 7 I'd like as little code as possible to actually read and write to the 8 network, and as much code as possible to pretend that the network doesn't 9 exist. I hope this will make everything easier to test. 10 11 ## Structure 12 13 Here follows a rough outline of our current crate dependency diagram. (Not all 14 crates and not all dependencies are shown; this diagram is simplified in order to 15 try to give a better understanding of the code structure.) 16 17 As a naming convention, crates that are user-facing start with "arti", crates 18 that are tor-specific start with "tor-", and crates that aren't tor-specific 19 have more general names. 20 21 ### Simplified module diagram 22 23 ```mermaid 24 graph TD 25 26 subgraph Application 27 arti --> tor-socksproto 28 end 29 30 subgraph Async Tor implementation 31 arti-client --> tor-dirmgr 32 arti-client --> tor-circmgr 33 tor-dirmgr --> tor-circmgr 34 tor-dirmgr --> tor-dirclient 35 tor-circmgr --> tor-chanmgr 36 tor-chanmgr --> tor-ptmgr 37 tor-circmgr --> tor-guardmgr 38 end 39 40 arti --> arti-client 41 42 subgraph Core Tor protocol 43 tor-proto --> tor-cell 44 end 45 46 tor-dirmgr & tor-circmgr & tor-guardmgr --> tor-netdir 47 48 subgraph Mid-level Tor functionality 49 tor-netdir --> tor-netdoc 50 tor-netdir --> tor-linkspec 51 tor-linkspec & tor-netdoc --> tor-protover 52 tor-netdoc --> tor-cert 53 tor-consdiff 54 tor-persist 55 end 56 57 tor-circmgr & tor-guardmgr --> tor-persist 58 59 subgraph Generic functionality 60 tor-bytes --> tor-llcrypto 61 tor-checkable --> tor-llcrypto 62 end 63 64 subgraph Rust async runtime 65 tor-rtcompat 66 end 67 68 arti-client & tor-dirmgr & tor-circmgr & tor-chanmgr & tor-guardmgr --> tor-rtcompat 69 70 tor-dirclient --> tor-proto 71 tor-dirmgr --> tor-consdiff 72 73 tor-circmgr & tor-chanmgr & arti-client --> tor-proto 74 tor-cell --> tor-bytes & tor-cert & tor-linkspec 75 tor-consdiff --> tor-llcrypto 76 tor-cert & tor-linkspec --> tor-bytes 77 ``` 78 79 ### List of crates 80 81 The current crates are: 82 83 * [`arti-bench`](../../crates/arti-bench/README.md) -- A simple benchmarking utility for Arti. 84 * [`arti-client`](../../crates/arti-client/README.md) -- High-level functionality for accessing the Tor network as a client. 85 * [`arti-config`](../../crates/arti-config/README.md) -- Removed crate. (Tools for configuration management in Arti) 86 * [`arti`](../../crates/arti/README.md) -- A minimal command line program for connecting to the Tor network 87 * [`arti-testing`](../../crates/arti-testing/README.md) -- Tool for running an Arti client with unusual behavior or limitations. 88 * [`caret`](../../crates/caret/README.md) -- Integers with some named values. 89 * [`fs-mistrust`](../../crates/fs-mistrust/README.md) -- Check whether file permissions are private. 90 * [`retry-error`](../../crates/retry-error/README.md) -- An error attempt to represent multiple failures. 91 * [`safelog`](../../crates/safelog/README.md) -- Mark data as sensitive for logging purposes. 92 * [`tor-basic-utils`](../../crates/tor-basic-utils/README.md) -- Utilities (low-level) for Tor 93 * [`tor-bytes`](../../crates/tor-bytes/README.md) -- Utilities to decode/encode things into bytes. 94 * [`tor-cell`](../../crates/tor-cell/README.md) -- Coding and decoding for the cell types that make up Tor's protocol 95 * [`tor-cert`](../../crates/tor-cert/README.md) -- Implementation for Tor certificates 96 * [`tor-chanmgr`](../../crates/tor-chanmgr/README.md) -- Manage a set of channels on the Tor network. 97 * [`tor-checkable`](../../crates/tor-checkable/README.md) -- Traits for wrapping up signed and/or time-bound objects 98 * [`tor-circmgr`](../../crates/tor-circmgr/README.md) -- circuits through the Tor network on demand. 99 * [`tor-config`](../../crates/tor-config/README.md) -- Tools for configuration management in Arti 100 * [`tor-congestion`](../../crates/tor-congestion/README.md) -- Algorithms for congestion control on the Tor network 101 * [`tor-consdiff`](../../crates/tor-consdiff/README.md) -- Restricted ed diff and patch formats for Tor. 102 * [`tor-dirclient`](../../crates/tor-dirclient/README.md) -- Implements a minimal directory client for Tor. 103 * [`tor-dirmgr`](../../crates/tor-dirmgr/README.md) -- Code to fetch, store, and update Tor directory information. 104 * [`tor-error`](../../crates/tor-error/README.md) -- Support for error handling in Tor and Arti 105 * [`tor-events`](../../crates/tor-events/README.md) -- Tools for generating a stream of structured events, similar to C tor's `ControlPort`. 106 * [`tor-guardmgr`](../../crates/tor-guardmgr/README.md) -- Guard node selection for Tor network clients. 107 * [`tor-linkspec`](../../crates/tor-linkspec/README.md) -- Descriptions of Tor relays, as used to connect to them. 108 * [`tor-llcrypto`](../../crates/tor-llcrypto/README.md) -- Low-level cryptographic implementations for Tor. 109 * [`tor-netdir`](../../crates/tor-netdir/README.md) -- Represents a clients'-eye view of the Tor network. 110 * [`tor-netdoc`](../../crates/tor-netdoc/README.md) -- Parse and represent directory objects used in Tor. 111 * [`tor-persist`](../../crates/tor-persist/README.md) -- Persistent data storage for use with Tor. 112 * [`tor-proto`](../../crates/tor-proto/README.md) -- Implementations for the core Tor protocol 113 * [`tor-protover`](../../crates/tor-protover/README.md) -- Implementation of Tor's "subprotocol versioning" feature. 114 * [`tor-ptmgr`](../../crates/tor-ptmgr/README.md) -- Manage a set of anti-censorship pluggable transports. 115 * [`tor-rtcompat`](../../crates/tor-rtcompat/README.md) -- Compatibility between different async runtimes for Arti. 116 * [`tor-rtmock`](../../crates/tor-rtmock/README.md) -- Support for mocking with `tor-rtcompat` asynchronous runtimes. 117 * [`tor-socksproto`](../../crates/tor-socksproto/README.md) -- Implements SOCKS in the flavors provided by Tor. 118 * [`tor-units`](../../crates/tor-units/README.md) -- Safe wrappers for primitive numeric types. 119 120 ## Design considerations, privacy considerations. 121 122 As we build the APIs for Arti, we've been aiming for 123 simplicity and safety: we want it to be as easy as possible to use 124 `arti-client`, while trying to make certain kinds of privacy or security 125 violation hard to write accidentally. 126 127 The lower-level we get, however, the more safety we lose. If we need to 128 allow a piece of functionality that isn't safe for general purposes, we 129 usually put it at a more low-level crate. 130 131 Privacy isn't just a drop-in feature, however. There are still 132 plenty of ways to accidentally leak information, even if you're 133 anonymizing your connections over Tor. We'll try to document 134 those in a user's guide at some point as Arti becomes more mature. 135 136 ## Object dependencies and handling dependency inversion. 137 138 (Or, "Why so much `Weak<>`?") 139 140 Sometimes we need to add a circular dependency in our object graph. For example, 141 the directory manager (`DirMgr`) needs a circuit manager (`CircMgr`) in order to 142 contact directory services, but the `CircMgr` needs to ask the `DirMgr` for a 143 list of relays on the network, in order to build circuits. 144 145 We handle this situation by having the lower-level type (in this case the 146 `CircMgr`) keep a weak reference to the higher-level object, via a `Weak<dyn 147 Trait>` pointer. Using a `dyn Trait` here allows the lower-level crate to 148 define the API that it wants to use, while not introducing a dependency on the 149 higher-level crate. Using a `Weak` pointer ensures that the lower-level object 150 won't keep the higher level object alive: thus when the last strong reference to the 151 higher-level object is dropped, it can get cleaned up correctly. 152 153 Below is a rough diagram of how our high-level "manager" objects fit together. 154 **Bold** lines indicate a direct, no-abstraction ownership relationship. Thin 155 lines indicate ownership for a single purpose, or via a limited API. Dotted 156 lines indicate a dependency inversion as described above, implemented with a 157 `Weak<dyn Trait>`. 158 159 ```mermaid 160 graph TD 161 162 TorClient ==> DirMgr & CircMgr 163 TorClient --> |C| ChanMgr & GuardMgr 164 DirMgr ==> CircMgr 165 CircMgr ==> GuardMgr & ChanMgr 166 CircMgr -.-> |W| DirMgr 167 GuardMgr -.-> |W| DirMgr 168 ChanMgr --> |As a ChannelFactory| PtMgr 169 ``` 170 171 We also use `Weak<>` references to these manager objects when implementing 172 **background tasks** that need to run on a schedule. We don't want the 173 background tasks to keep the managers alive if there are no other references to 174 the `TorClient`, so we tend to structure them more or less as follows: 175 176 ```rust 177 async fn run_background_task(mgr: Weak<FooMgr>, schedule: ScheduleObject) { 178 while schedule.await { 179 if let Some(mgr) = Weak::upgrade(&mgr) { 180 // Use mgr for background task 181 // ... 182 } else { 183 break; 184 } 185 } 186 ``` 187 188