1  # Apibara Streaming Protocol (v1alpha2)
 2  
 3  The streaming protocol is used by Apibara nodes and clients to exchange data.
 4  
 5  The goals are:
 6  
 7  - linearize non-linear data into something easy to follow,
 8  - let clients customize the data they want to receive, reducing the bandwidth
 9    required,
10  - provide support for data at all stages of finality (pending, accepted,
11    finalized).
12  
13  ## The protocol
14  
15  The client starts by calling the `StreamData` method in the Stream gRPC service.
16  The client must send a `StreamDataRequest` message to the server to start the
17  stream.
18  
19  The request includes:
20  
21  - `stream_id`: unique id for the stream. All messages generated in response to
22    this request will have the specified stream id, or 0 if not specified.
23  - `starting_cursor`: specifies from where to start the stream. The cursor is
24    stream-specific.
25  - `finality`: specifies the finality required by the client. This parameter
26    changes the behavior of the stream.
27  - `filter`: specifies what type of data the client wants to receive. This is
28    specific to each stream.
29  
30  After the client requests data, the stream will start sending
31  `StreamDataResponse` messages to the client. The messages can have the following
32  content:
33  
34  - `invalidate`: invalidates data previously sent, for example in response to
35    chain reorganizations.
36  - `data`: sends a new batch of data.
37  - `heartbeat`: periodically sent if no other types of messages were produced.
38    Used to confirm that the client and server are still connected.
39  
40  The client can reset the stream by sending a new `StreamDataRequest`. The server
41  will stop sending data for the previous request and will start sending data for
42  the new stream. Notice that because the flow is async, the client may still
43  receive messages from the old stream definition. Use the `stream_id` to uniquely
44  identify streams.
45  
46  ### Data finality
47  
48  Apibara supports streaming data with different finality. The stream behaves
49  differently based on the finality mode specified in the request:
50  
51  - `finalized`: the stream sends `data` messages for finalized data only. The
52    `data` messages contain the data requested in the stream filter. Notice that
53    there cannot be `invalidate` messages in this type of stream.
54  - `accepted`: the stream sends `data` messages for accepted data (including
55    historical finalized data).
56  - `pending`: the stream sends `data` messages for pending data and for accepted
57    data. Notice that the client may receive the same pending data multiple times.
58  
59  ### Cursor
60  
61  Cursors are used to identify a location in a stream. The `end_cursor` in `Data`
62  messages refers to the location of the latest piece of data in the stream. When
63  clients send this cursor as a `starting_cursor` in a request, the server will
64  resume streaming by sending the first message that _follows_ the provided
65  cursor. When no `starting_cursor` is specified, the stream will start from the
66  genesis block. If the data in or before `end_cursor` was invalidated (following
67  a chain reorganization), the server will inform the client of it and will resume
68  the stream from the new stream's head.
69  
70  A `Cursor` is made of two components:
71  
72  - `order_key`: this is the sequence number for messages,
73  - `unique_key`: this is used to discriminate between data generated from
74    different blocks at the same height.