article.svx
1 --- 2 title: A simple functional API client 3 subtitle: Reduce friction in your API layer 4 date: 2022-01-25 5 tags: 6 - architecture 7 - functional-programming 8 author: Cam 9 --- 10 11 > This article outlines the design of the API client I choose when starting a 12 > new project. It is presented as a logical evolution from using the 'built-in', 13 > base level HTTP client that comes with a language of choice, to the client as 14 > I use it today. 15 > 16 > Code examples are included to illustrate the way our API client looks at each 17 > step of the way. The examples are written in fairly specific Javascript, as 18 > that is the language in which I find this pattern most frequently useful, but 19 > the same ideas can be applied in any language. Such translation is left as 20 > an exercise to the reader. 21 > 22 > Anyway, enough of that, on with the story! 23 24 In my experience, pretty much every app these days—mobile apps, web apps, 25 or whatever other kind of app you can imagine—is just a fancy wrapper 26 around one API or another. As such, the core of every app is its API client 27 which, despite the fact that we must use it for every single feature, 28 is often the oldest, smelliest, crustiest bit of code in the project. 29 30 If not paying attention, it's only a matter of time before your app becomes a 31 mess of calls to `fetch` and strings containing URLs hardcoded all over the 32 place. In one file you've interpreted the response one way, and in another file 33 the same request is used for different data, who knows which one is correct. 34 No one has even *considered* trying to figure out what to do with errors, and 35 half the time you don't even check for them. Why spend time on error handling 36 when they're all unexpected anyway. 37 38 ```javascript 39 const response = await fetch(`/users/${id}`, { 40 method: 'POST', 41 headers: { 42 Authorization: `Bearer ${bearer}`, 43 'Content-Type': 'application/json', 44 }, 45 body: JSON.stringify({ username }), 46 }); 47 const { user: updatedUser } = await response.json(); 48 ``` 49 50 It should be clear that having this code scattered around the codebase will not do. 51 There's so much friction involved in just writing a single API call: 52 1. What was the URL? 53 2. What HTTP method should it be? 54 3. Is that parameter supposed to be a query parameter, a URL segment, or in the body? 55 4. Is the body a JSON body or a form? 56 57 Of course, the obvious next step is to just make a function for that so we don't need 58 to remember those things. Unfortunately, we not only need relevant info like the 59 user ID and username, but also unrelated stuff like the `bearer` token. While I assume 60 that was somehow acessible to the application level code you were writing earlier, now 61 that our API call function is moved to a separate file, maybe not? Let's just pass that 62 in for now. 63 64 ```javascript 65 async function updateUsername(id, username, bearer) { 66 const response = await fetch(`/users/${id}`, { 67 method: 'POST', 68 headers: { 69 Authorization: `Bearer ${bearer}`, 70 'Content-Type': 'application/json', 71 }, 72 body: JSON.stringify({ username }), 73 }); 74 const { user } = await response.json(); 75 return user; 76 } 77 78 const updatedUser = await updateUsername(id, username, bearer); 79 ``` 80 81 This certainly will not do. Nobody wants to be thinking about what tokens are 82 required for each API call---that's not something a normal function has to deal 83 with. We want to make these API calls as frictionless as possible. Everyone's first 84 instinct is to try OOP, so let's make a `Client` class and put all the common request 85 stuff in there. Given that we had access to the `bearer` token before, we can just 86 put this client wherever that was and still have access to it at any moment we need it, 87 no problem. 88 89 ```javascript 90 class Client { 91 constructor(bearer) { 92 this.bearer = bearer; 93 } 94 95 async updateUsername(id, username) { 96 const response = await fetch(`/users/${id}`, { 97 method: 'POST', 98 headers: { 99 Authorization: `Bearer ${this.bearer}`, 100 'Content-Type': 'application/json', 101 }, 102 body: JSON.stringify({ username }), 103 }); 104 const { user } = await response.json(); 105 return user; 106 } 107 } 108 109 const client = new Client(bearer); 110 const updatedUser = await client.updateUsername(id, username); 111 ``` 112 113 This looks ok for now, but as the API grows there is no chance that I want to be 114 putting *every single request* into one massive file. Going the class-based 115 approach was the wrong call. This is not a surprise... since when are classes 116 ever the right answer? 117 118 Let's get those functions back into separate files, but try and keep this 119 `client` instance idea. We can represent each API call as an object and 120 pass those to the client to invoke them with the correct parameters. 121 122 ```javascript 123 class Client { 124 constructor(bearer) { 125 this.bearer = bearer; 126 } 127 128 invoke(request) { 129 const headers = new Headers(request.init.headers); 130 headers.set('Authorization', `Bearer ${bearer}`); 131 return fetch(request.url, { ...request.init, headers }); 132 } 133 } 134 135 function updateUsername(id, username) { 136 return { 137 url: `/users/${id}`, 138 init: { 139 headers: { 'Content-Type': 'application/json' }, 140 body: JSON.stringify({ username }), 141 }, 142 }; 143 } 144 145 const client = new Client(bearer); 146 const updatedUser = await client.invoke(updateUsername(id, username)) 147 .then((response) => response.json()) 148 .then((response) => response.user); 149 ``` 150 151 Something here sure looks redundant... Such verbosity for so little gain. We've even 152 gone a step backwards, the common process of extracting data from the result object (and 153 checking for errors if we were to bother to do such a thing) have to be repeated at every 154 call site. The friction is back. 155 156 Not only that, we have this object of `{ url, init }` that every single request function 157 needs to construct. Nobody's going to remember those details, it'll just be copy pasted 158 all over the place, so let's go ahead and make a constructor for that. While we're at it, 159 to make this class less prone to unwanted introspection, let's just make it a regular 160 closure. 161 162 ```javascript 163 function client({ bearer }) { 164 return function invoke(request) { 165 const headers = new Headers(request.init.headers); 166 headers.set('Authorization', `Bearer ${bearer}`); 167 return fetch(request.url, { ...request.init, headers }); 168 } 169 } 170 171 function request(url, init) { 172 return { url, init }; 173 } 174 175 function updateUsername(id, username) { 176 return request(`/users/${id}`, { 177 body: JSON.stringify({ username }), 178 }); 179 } 180 181 const invoke = client({ bearer }); 182 const updatedUser = await invoke(updateUsername(id, username)) 183 .then((response) => response.json()) 184 .then((response) => response.user); 185 ``` 186 187 We're getting closer, that first line `invoke(updateUser(id, username))` is pretty 188 frictionless, but there are still those two `.then()` calls tacked on to the end of every 189 invocation that we need to get rid of. 190 191 We can do that by flipping the client inside out. Instead of having the request object 192 passed to the invoker, let's pass the invoker to a request *function*. That way, the request 193 function can do whatever postprocessing it needs to do. We can do this transformation 194 invisibly to keep the `invoke(request)` structure as it was. 195 196 ```javascript 197 function client({ bearer }) { 198 function invoke(request) { 199 // ... invoke has not changed 200 } 201 202 return (request) => request(invoke); 203 } 204 205 function request(url, init) { 206 return { url, init }; 207 } 208 209 function updateUsername(id, username) { 210 return async (invoke) => { 211 const response = await invoke(request(`/users/${id}`, { 212 body: JSON.stringify({ username }), 213 })); 214 const { user } = await response.json(); 215 return user; 216 }; 217 } 218 219 const invoke = client({ bearer }); 220 const updatedUser = await invoke(updateUsername(id, username)); 221 ``` 222 223 Now it's starting to look pretty good! Making an API call has been boiled down to 224 a single frictionless line. As a bonus, the whole API client fits in a small box 225 on a blog post, with no external dependencies or codegen setup required! 226 227 The request functions are a bit verbose now, but otherwise, we've achieved what we 228 set out to do. This is the final form of the actual API client. It is not, however, 229 the end of the article, because we have only just scratched the surface of how this 230 thing can be used. 231 232 The main thing to notice about this whole thing is that each request object is 233 actually a function, and functions can be composed. 234 235 Let's start by breaking this thing up a little further. 236 237 ```javascript 238 function asyncPipe(...fns) { 239 return async (value) => { 240 for (const fn of fns) { 241 value = await fn(value); 242 } 243 return value; 244 }; 245 } 246 247 // ... client has not changed 248 249 function request(url, init) { 250 return (invoke) => invoke({ url, init }); 251 } 252 253 // Request 254 function updateUsernameRequest(id, username) { 255 return request(`/users/${id}`, { 256 body: JSON.stringify({ username }), 257 }); 258 } 259 260 // Action 261 function updateUsername(id, username) { 262 return asyncPipe( 263 updateUsernameRequest(id, username), 264 (response) => response.json(), 265 (response) => response.user, 266 ); 267 } 268 ``` 269 270 A bit of an aside: one trick I find to writing a good API layer is to actually split it 271 into *multiple* layers, which is what can be seen here. By making a distinction between a 272 *request* and an *action*, as marked above, we finally start decoupling the application 273 and the API hiding behind it. When it comes time to put this into practice, requests 274 should be kept in a separate place from actions. Consider requests as low-level API 275 operations to be composed to create actions, while actions are functions that achieve 276 the business goals of the application. 277 278 In the example above, we've used the `asyncPipe` function (an async adaptation of the 279 common `pipe` or `compose` function found in most functional programming libraries) to 280 compose an action out of a request and some other functions. 281 282 Of course, actions themselves are still just functions under the hood, so they can 283 be composed this way too. Consider a situation where we need to make multiple API calls 284 to do something more complicated, such as setting the user's profile picture. In such 285 a hypothetical situation, this might require first requesting a URL to upload the 286 image to, then uploading the image to that URL, and finally notifying the server that 287 the image has been uploaded and the user's profile can be updated. 288 289 ```javascript 290 function asyncChain(request, continuation) { 291 return async (context) => { 292 const response = await request(context); 293 const next = await continuation(response); 294 return next(context); 295 }; 296 } 297 298 function prepareProfilePictureRequest(id) { 299 return request(`/users/${id}/prepareprofilepicture`, { method: "POST" }); 300 } 301 302 function prepareProfilePicture(id) { 303 return asyncPipe( 304 prepareProfilePictureRequest(id), 305 (response) => response.json(), 306 ); 307 } 308 309 function updateProfilePictureRequest(id, requestId) { 310 return request(`/users/${id}/updateprofilepicture`, { 311 method: "POST", 312 body: JSON.stringify({ requestId }), 313 }); 314 } 315 316 function updateProfilePicture(id, picture) { 317 return asyncChain( 318 prepareProfilePicture(id), 319 async ({ uploadToUrl, requestId }) => { 320 await upload(uploadToUrl, picture); 321 return updateProfilePictureRequest(id, requestId), 322 }, 323 ); 324 } 325 326 await invoke(updateProfilePicture(id, picture)); 327 ``` 328 329 To understand what's happening here, we start to get a little bit more technical... 330 Taking advantage of the fact that functions are monads, and writing `asyncChain` to 331 act on an async-function-monad, we are able to consider requests and actions as 332 values which can be composed in any way that monads can be composed. 333 334 Similarly, `asyncPipe` is the async-function-monad equivalent of the map operator, 335 which is another way by which we compose requests. 336 337 Borrowing the syntax from Haskell, we can take a moment to look at the types of these 338 functions (forgetting that `asyncPipe` was implemented variadically) in hopes of 339 making the similarities more clear. 340 341 ```haskell 342 asyncChain :: Request a -> (a -> Promise (Request b)) -> Request b 343 asyncPipe :: Request a -> (a -> Promise b) -> Request b 344 345 -- Recall some of Haskell's operators 346 (>>=) :: Monad m => m a -> (a -> m b) -> m b 347 (<&>) :: Functor f => f a -> (a -> b) -> f b 348 ``` 349 350 Substituting `Request` for `m` or `f` in the types of `>>=` (bind) and `<&>` (map), 351 respectively, it's pretty clear that `asyncChain` and `asyncPipe` are basically the 352 same, just with an inserted `Promise`. 353 354 > I'm no expert on this subject, so maybe don't quote me on saying this is an 355 > "async monad" or "async functor", but if not it's *awfully close* and works 356 > out okay in practice. If someone feels like doing a more thorough analysis 357 > on this, that might be interesting! 358 359 Now, since these things are basically async-monads, we can bring in my favourite use 360 of generator functions to build the async-generator-to-deterministic-async-monad-transformation, 361 and end up with what is basically async-await syntax but for requests! 362 363 ```javascript 364 // ... with everything else unchanged 365 366 function doRequest(generator) { 367 function next(iter) { 368 return async (input) => { 369 const { value, done } = await iter.next(input); 370 if (done) return () => Promise.resolve(value); 371 return asyncChain(value, next(iter)); 372 }; 373 } 374 375 return async (context) => { 376 const iter = await generator(); 377 const request = await next(iter)(undefined); 378 return request(context); 379 }; 380 } 381 382 function updateProfilePicture(id, picture) { 383 return doRequest(async function* () { 384 const { uploadToUrl, requestId } = yield prepareProfilePicture(id); 385 await upload(uploadToUrl, requestId); 386 return updateProfilePictureRequest(id, requestId); 387 }); 388 } 389 390 await invoke(updateProfilePicture(id, picture)); 391 ``` 392 393 Believe it or not, the two definitions of `updateProfilePicture` are 394 functionally the same. We now have a perfectly readable and convenient way to write 395 our API calls into separate, reusable utility files with no `client` parameters to 396 be cluttering function signatures. 397 398 The tradeoff is that we complicate things somewhat by using wrapped generators, but 399 personally I have no problems with that. If you really don't like it though, you can 400 simply opt-out of going so deep and stick to the function-returning-function version 401 we had earlier and still use the core API client, knowing that it's more powerful 402 than it looks. 403 404 While that's as far as I have taken this particular API client personally, I have no 405 doubt that it can go further. Plus, as a bonus, even with all the helper functions 406 to show off real strengths, it still all fits in a few boxes on a blog post! Next time 407 you start a project, try slipping this pattern in there, and I doubt anyone would 408 question it. At least not until you start doing all the fancy stuff. 409 410 ----- 411 412 P.S.: I previously created [`openfetch`](https://github.com/foxfriends/openfetch) 413 (available on [NPM](https://www.npmjs.com/package/openfetch)) which implements this 414 client and generates a request-level API client from an OpenAPI 3.0 specification. 415 If you happen to be using OpenAPI and are interested in trying this pattern out, 416 that is one way you can get started.