store-retrieve-messages.md
1 --- 2 title: Retrieve Messages Using Store Protocol 3 hide_table_of_contents: true 4 displayed_sidebar: build 5 --- 6 7 This guide provides detailed steps to create a Light Node for retrieving and filtering historical messages using the [Store protocol](/learn/concepts/protocols#store). 8 9 ## Create a light node 10 11 Use the `createLightNode()` function to create a Light Node and interact with the Waku Network: 12 13 ```js 14 import { createLightNode } from "@waku/sdk"; 15 16 // Create and start a Light Node 17 const node = await createLightNode({ defaultBootstrap: true }); 18 await node.start(); 19 ``` 20 21 ## Connect to store peers 22 23 Use the `node.waitForPeers()` method to wait for the node to connect with Store peers: 24 25 ```js 26 import { Protocols } from "@waku/sdk"; 27 28 // Wait for a successful peer connection 29 await node.waitForPeers([Protocols.Store]); 30 ``` 31 32 You can also specify a dedicated Store peer to use for queries when creating the node. This is particularly useful when running your own Store node or when you want to use a specific Store node in the network: 33 34 ```js 35 const node = await createLightNode({ 36 defaultBootstrap: true, 37 store: { 38 peer: "/ip4/1.2.3.4/tcp/1234/p2p/16Uiu2HAm..." // multiaddr or PeerId of your Store node 39 } 40 }); 41 ``` 42 43 If the specified Store peer is not available, the node will fall back to using random Store peers in the network. 44 45 ## Choose a content topic 46 47 [Choose a content topic](/learn/concepts/content-topics) for filtering the messages to retrieve and create a message `decoder`: 48 49 ```js 50 import { createDecoder } from "@waku/sdk"; 51 52 // Choose a content topic 53 const contentTopic = "/store-guide/1/message/proto"; 54 55 // Create a message decoder 56 const decoder = createDecoder(contentTopic); 57 ``` 58 59 ## Retrieve messages 60 61 `@waku/sdk` provides the `queryWithOrderedCallback()` and `queryGenerator()` functions for querying `Store` nodes and retrieving historical or missed messages. The responses from `Store` nodes are paginated and require you to process each page sequentially. 62 63 ### `queryWithOrderedCallback` 64 65 The `store.queryWithOrderedCallback()` function provides a straightforward method for querying `Store` nodes and processing messages in chronological order through a callback function. It accepts these parameters: 66 67 - `decoders`: List of `decoders` that specify the `content topic` to query for and their [message decryption](https://rfc.vac.dev/waku/standards/application/26/payload) methods. 68 - `callback`: The callback function for processing the retrieved messages. 69 - `options` (optional): [Query options](/build/javascript/store-retrieve-messages#store-query-options) to filter the retrieved messages. 70 71 ```js 72 // Create the callback function 73 const callback = (wakuMessage) => { 74 // Render the message/payload in your application 75 console.log(wakuMessage); 76 }; 77 78 // Query the Store peer 79 await node.store.queryWithOrderedCallback([decoder], callback); 80 ``` 81 82 :::info 83 The `queryWithOrderedCallback()` function always returns the most recent messages in a page first. 84 ::: 85 86 ### `queryGenerator` 87 88 The `store.queryGenerator()` function provides more control and flexibility over processing messages retrieved from `Store` nodes through [Async Generators](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/AsyncGenerator). It accepts these parameters: 89 90 - `decoders`: List of `decoders` that specify the `content topic` to query for and their [message decryption](https://rfc.vac.dev/waku/standards/application/26/payload) methods. 91 - `options` (optional): [Query options](/build/javascript/store-retrieve-messages#store-query-options) to filter the retrieved messages. 92 93 ```js 94 // Create the store query 95 const storeQuery = node.store.queryGenerator([decoder]); 96 97 // Process the messages 98 for await (const messagesPromises of storeQuery) { 99 // Fulfil the messages promises 100 const messages = await Promise.all( 101 messagesPromises.map(async (p) => { 102 const msg = await p; 103 // Render the message/payload in your application 104 console.log(msg); 105 }) 106 ); 107 } 108 ``` 109 110 :::info 111 The `queryGenerator()` function always returns the oldest messages in a page first. 112 ::: 113 114 ## Store query options 115 116 ### `pageDirection` 117 118 The `pageDirection` option specifies the direction in which pages are retrieved: 119 120 - `BACKWARD` (default): Most recent page first. 121 - `FORWARD`: Oldest page first. 122 123 ```js 124 import { PageDirection } from "@waku/sdk"; 125 126 // Retrieve recent messages first 127 const queryOptions = { 128 pageDirection: PageDirection.BACKWARD, 129 }; 130 131 // Retrieve oldest messages first 132 const queryOptions = { 133 pageDirection: PageDirection.FORWARD, 134 }; 135 136 // Query the Store peer with options 137 await node.store.queryWithOrderedCallback([decoder], callback, options); 138 const storeQuery = node.store.queryGenerator([decoder, options]); 139 ``` 140 141 ### `cursor` 142 143 The `cursor` option specifies the starting index for retrieving messages. For example, consider a query that retrieves the first page messages and then continues with the next page: 144 145 ```js 146 import { waku } from "@waku/sdk"; 147 148 // Create the callback function 149 const messages = []; 150 const callback = (wakuMessage) => { 151 messages.push(wakuMessage); 152 // Return "true" to stop retrieving pages 153 // Here, it retrieves only the first page 154 return true; 155 }; 156 157 // Retrieve the first page of messages 158 // This retrieves all the messages if "return true" is not present 159 await node.store.queryWithOrderedCallback([decoder], callback); 160 161 // Create the cursor 162 const lastMessage = messages[messages.length - 1]; 163 const cursor = await waku.createCursor(lastMessage); 164 165 // Retrieve the next page of messages 166 // The message at the cursor index is excluded from the result 167 await node.store.queryWithOrderedCallback([decoder], callback, { 168 cursor: cursor, 169 }); 170 console.log(messages); 171 ``` 172 173 :::info 174 If you omit the `cursor` option, the query will start from the beginning or end of the history, depending on the [page direction](#pagedirection). 175 ::: 176 177 ### `timeFilter` 178 179 The `timeFilter` option specifies a time frame to retrieve messages from. For example, consider a query that retrieves messages from the previous week: 180 181 ```js 182 // Get the time frame 183 const endTime = new Date(); 184 const startTime = new Date(); 185 startTime.setDate(endTime.getDate() - 7); 186 187 // Retrieve a week of messages 188 const queryOptions = { 189 timeFilter: { 190 startTime, 191 endTime, 192 }, 193 }; 194 195 // Query the Store peer with options 196 await node.store.queryWithOrderedCallback([decoder], callback, options); 197 const storeQuery = node.store.queryGenerator([decoder, options]); 198 ``` 199 200 :::info 201 The `timeFilter` option significantly reduces message retrieval performance. To optimise it, consider resuming message retrieval using a [cursor](#cursor) that starts from the last seen message. 202 ::: 203 204 :::tip Congratulations! 205 You have successfully retrieved and filtered historical messages on a Light Node using the `Store` protocol. Have a look at the [store-js](https://github.com/waku-org/js-waku-examples/tree/master/examples/store-js) and [store-reactjs-chat](https://github.com/waku-org/js-waku-examples/tree/master/examples/store-reactjs-chat) examples for working demos. 206 :::