hssvc-high-level-apis.md
1 2 2023-08-23 3 4 Here are some notes about high level api/ui design for onion services, 5 and what it might look like. 6 7 2023-08-28 8 9 See [!1541](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1541) 10 for comments on this note. 11 12 # Top CLI level: the `arti` CLI tool. 13 14 I'm imagining that the configuration looks something like this: 15 16 ```toml 17 # I'm making this an array of 'onion_service' tables. 18 # 19 # One alternative would be to make it a set of named tables 20 # as in "[onion_service.example_name]". 21 [[onion_service]] 22 23 # This is the only required option. It sets a nickname 24 # for the onion service that's used to identity it in the 25 # interface and key manager. 26 # 27 # Different onion services must have different names. 28 name = "example" 29 30 # If false, this onion service isn't actually enabled. 31 # We're providing this as a convenience to make it easier 32 # to turn services on and off. 33 enabled = true 34 35 # I had thought of calling this is_single_onion_service but I am 36 # frightened to make this too easy to set by accident. Honestly 37 # I would prefer that its name be even scarier. 38 is_non_anonymous = false 39 40 #### 41 # These options are about setting limits on concurrency and 42 # introduction rate. 43 #### 44 45 # This option is a (rate,burst) tuple that we should send to 46 # the introduction point to configure how many introduction 47 # requests it accepts. I'm open to a better way to 48 # specify these. 49 rate_limit_at_intro = [ 100, 500 ] 50 51 # How many streams will we allow to be open at once for this service? 52 max_concurrent_streams = 1000 53 54 # How many streams will we allow to be open at once for a single circuit 55 # on this service? 56 max_concurrent_streams_per_circuit = 10 57 58 # If true, will we require proof-of-work when we're under heavy load. 59 enable_proof_of_work = true 60 61 # Rate/burst for dispatching requests from rendezvous request queue when 62 # pow defense is enabled. I don't know if we want to duplicate this 63 # mechanism, but tor does it. 64 pow_queue_rate = [ 100, 500 ] 65 66 # Disable the compiled hashx backend for proof-of-work. 67 disable_pow_compilation = false 68 69 # Descriptor-based client authentication. If this is set 70 # to any array, even an empty one, then authentication is required. 71 encrypt_descriptor = [ 72 'curve25519:aaaaaaa', # A given public key can be put in the config. 73 'dir:/path/to/dir', # A directory full of keys can be listed. 74 ] 75 # Note that you can also give a singleton, as in: 76 # encrypt_descriptor = 'dir:/path/to/dir". 77 78 # Set the number of introduction points to try to use for the onion service. 79 num_intro_points = 3 80 81 # This option configures port relaying, which is the only option 82 # available at the CLI for actually implementing an onion service. 83 # 84 # The syntax for each port is: 85 # [ SOURCE, DEST ] 86 # 87 # Allowable SOURCE values are: 88 # integer: a port. 89 # 'low-high': a range of ports 90 # '*': matches all ports. 91 # 92 # Allowable destination values are: 93 # 94 # 'tcp:sockaddr' or 'sockaddr' 95 # forward to a given socket address 96 # 'unix:/path' 97 # forward to an AF_LOCAL address. 98 # 'reject' 99 # close the stream. 100 # 'ignore' 101 # ignore the request. (Q: Should this even exist?) 102 # 'destroy': 103 # tear down the circuit. 104 # 105 # The patterns in `proxy_ports` match from first to last; we take 106 # the first one that matches. If no pattern matches, we tear down 107 # the circuit. 108 # 109 proxy_ports = [ 110 [80, '127.0.0.1:9998'], 111 [443, '127.0.0.1:9999'], 112 ['1-1024', 'reject'], 113 ['*', 'ignore'] 114 ] 115 ``` 116 117 118 # Top API level: the `arti-client` API. 119 120 I think we translate the above options into a set of configuration 121 structures something like this. 122 123 (In areas there this isn't a 100% match for the above we should probably 124 reconcile them to minimize needless divergence.) 125 126 ``` 127 pub struct OnionSvcConfig { 128 // note that this is the only option that can't be changed. Maybe 129 // that means that instead of making it part of the OnionSvcConfig 130 // it should be at a higher level? IOW, instead of Vec<OnionSvcConfig>, 131 // we would have HashMap<String,OnionSvcConfig>. 132 133 name: String, 134 enabled: bool, 135 rate_limits: RateLimitConfig, 136 pow: ProofOfWorkConfig, 137 encrypt_descriptor: Option<DescEncryption> 138 139 // Note that this doesn't include proxy configuration 140 // at this level, since that's not part of the onion 141 // service itself. 142 } 143 144 pub struct TokenBucketConfig { 145 max_per_sec: u32, 146 max_burst: u32, 147 } 148 149 pub struct RateLimitConfig { 150 rate_limit_at_intro: Option<TokenBucketConfig>, 151 max_concurrent_streams: u32, // or Option<NonZeroU32> 152 max_concurrent_streams_per_circuit: u32, // or Option<NonZeroU32> 153 } 154 155 pub struct ProofOfWorkCOnfig { 156 enable: bool, 157 queue_rate: Option<TokenBucketConfig>, 158 disable_compilation: bool 159 } 160 161 pub struct DescEncryption { 162 authorized_client: Vec<AuthorizedClient>, 163 } 164 165 pub enum AuthorizedClient { 166 DirectoryOfKeys(PathBuf), 167 Curve25519Key(curve25519::PublicKey), 168 } 169 170 mod proxy { 171 172 pub struct ProxyConfig { 173 rules: Vec<ProxyRule>, 174 } 175 176 pub struct ProxyRule { 177 source: ProxyPattern, 178 target: ProxyTarget, 179 } 180 181 pub enum ProxyPattern { 182 Port(u16), 183 PortRange(u16,u16), 184 All, 185 } 186 187 pub enum ProxyTarget { 188 Tcp(SocketAddr), 189 Unix(PathBuf), 190 RejectStream, 191 DropStream, 192 DestroyCircuit 193 } 194 } 195 ``` 196 197 198 On to the APIs. I'm imagining that these methods goes into `TorClient`: 199 200 ``` 201 pub async fn launch_onion_service(&self, config: OnionSvcConfig) { 202 -> Result<(Arc<OnionSvc>, impl Stream<Item=IncomingStream>)> 203 { 204 ... 205 } 206 // ^ Note that if we move `name` out of the config, we should 207 // make it an argument here. 208 209 pub fn lookup_onion_service(&self, name: &str) -> Option<Arc<OnionSvc>> 210 { 211 ... 212 } 213 ``` 214 215 There's an `OnionSvc` handle in the return value here so that we can 216 manage the service. It's in an `Arc<>` so that it's visible to the RPC 217 system. 218 219 I think that the API for `OnionSvc` looks something like this: 220 221 ``` 222 impl OnionSvc { 223 pub fn reconfigure(&self, new_config: OnionSvcConfig) -> Result<()> {..} 224 pub fn shutdown(&self) -> Result<()> {..} 225 pub fn detach(&self) -> Result<()> {..} 226 } 227 ``` 228 229 To implement proxies, I think we do something like this. I think it goes 230 in a separate crate. 231 232 ``` 233 pub struct OnionSvcProxy { 234 .. 235 } 236 237 impl OnionSvcProxy { 238 pub fn new( 239 config: ProxyConfig, 240 streams: impl Stream<Item=IncomingStream> 241 ) -> Result<Self> 242 { .. } 243 244 pub fn reconfigure(&self, new_config: ProxyConfig) -> Result<()> {..} 245 246 // implementation function. Runs forever. Probably not public. 247 fn handle_requests_loop(&self) -> Result<()> {..} 248 249 // run forever in a new task. 250 pub fn launch<R:Runtime>(&self) -> Result<()> {..} 251 } 252 ``` 253 254 255 # Missing APIs 256 257 For some purposes we'd like to have an "ephemeral" onion service: this 258 amounts to one where we don't store anything on disk and instead 259 the caller takes responsibility for key management and persistence. 260 261 Would this be as simple as providing an alternative API like this? 262 263 ``` 264 pub async fn launch_ephemeral_onion_service( 265 &self, 266 config: OnionSvcConfig 267 persistence: OnionSvcPersist) { 268 -> Result<(Arc<OnionSvc>, impl Stream<Item=IncomingStream>)> 269 { 270 ... 271 } 272 273 pub struct OnionSvcPersist { 274 my_state_handler: Box<dyn tor_persist::StorageHandle<serde_json::Object>>, 275 276 my_key_mgr: Box<dyn AbstractKeyMgr>, 277 } 278 ``` 279