1  # Apibara Node
 2  
 3  A node combines and transforms multiple input streams into a new output stream.
 4  
 5  ## Stream Protocol
 6  
 7  All messages in the Apibara stream protocol are identified by a sequence number.
 8  Clients must check that the messages they receive have increasing sequence
 9  numbers without gaps (except in the case of message invalidation).
10  
11  The following messages are part of the protocol:
12  
13  - `Data(sequence, data)`: contains the message data together with its sequence
14    number.
15  - `Invalidate(sequence)`: informs the client that all messages with sequence
16    number greater than or equal to the specified `sequence` are now invalid. The
17    stream will resume by sending the new messages from the specified `sequence`
18    number.
19  
20  Invalidation is needed because web3 data is not finalized immediately. Chain
21  reorganizations cause blocks that were previously considered canonical to be
22  removed from the canonical chain. By making data invalidation a core part of the
23  protocol, nodes can push information about chain reorganizations downstream.
24  
25  ## Node Implementation
26  
27  A node is responsible for tracking the state of each input stream across
28  restarts. It also manages the sequence number generator so that output messages
29  are correctly sequenced. Finally, the node stores the messages generated by the
30  stream to persistent storage so that they can be replied to clients that connect
31  at a later point in time.
32  
33  The data transformation and aggregation is performed by applications.
34  Applications communicate with the node through gRPC and must be started
35  separately.
36  
37  ## Source Nodes
38  
39  Nodes that ingest data from an outside source and into Apibara are called
40  _source nodes_. Source nodes can, for example, ingest data from a blockchain
41  node and generate a stream of Apibara messages. Blockchain nodes can implement
42  the Apibara stream protocol directly for lower latency.
43  
44  Source nodes are not limited to blockchain nodes: an HTTP server can generate a
45  stream of user actions (e.g. `CommentPosted`, `PostLiked`, `FriendRequestSent`,
46  etc.) to create applications that mix off-chain and on-chain data.