1  ---
  2  title: Apibara Integrations
  3  titleShort: Overview
  4  description: "Build production-grade indexers using Apibara."
  5  priority: 1000
  6  updatedAt: 2023-08-15 10:00
  7  ---
  8  
  9  # Apibara integrations
 10  
 11  From our own and our users experience building indexers, we noticed common
 12  patterns when it comes to integrate onchain data with the services that power
 13  modern applications.
 14  
 15  Apibara provides built-in _integrations_ that simplify integrating onchain data
 16  with common services such as web API, databases, and data analytics tools.
 17  
 18  
 19  ## Integrations concepts
 20  
 21  All integrations follow these three steps:
 22  
 23   - stream data from a DNA stream by using the user-provided **filter**.
 24   - optionally, each batch of data is **transformed** by running a
 25     JavaScript/Typescript script. This step is powered by Deno, a lightweight
 26     and powerful JavaScript runtime.
 27   - finally, data is sent to the downstream integration. This step looks
 28     different based on the type of integration.
 29  
 30  Notice that all integrations support starting from historical data and then
 31  continue with live (real-time) blocks. This means you can use Apibara to build
 32  both your indexer (which requires historical data), but also notification
 33  services that require the most recent real-time data.
 34  
 35  
 36  ## Types of integrations
 37  
 38  Apibara goal is to bring onchain data to any application. At the moment, we
 39  offer three types of integrations:
 40  
 41  
 42  ### Web API
 43  
 44   - **serverless functions**: invoke serverless functions for each batch of
 45     data, both historical and live data. Functions are also invoked when a chain
 46     reorganization happens, so that your application can manage them.
 47   - **webhooks**: invoke a webhook for each batch of data, with exactly the payload
 48   you provide. This integration doesn't invoke the HTTP webhook in case of
 49   chain reorganization.
 50  
 51  
 52  ### Database mirroring
 53  
 54  Apibara can mirror all onchain data you select to a database of your choice.
 55  This is the easiest and fastest way to build an indexer for your application,
 56  all data is synced automatically and you can focus on other important parts
 57  of your application.
 58  
 59  While some details vary between each database implementation, all database
 60  integrations work as follows:
 61  
 62   - the records returned by the _transform step_ are inserted in the database.
 63   This step is required to return an array of objects.
 64   - Apibara adds the cursor that generated each piece of data.
 65   - When a chain reorganization happens, Apibara removes all records that have
 66   been invalidated.
 67  
 68  We provide integrations for the following two databases:
 69  
 70   - **PostgreSQL**: write data to the _table_ specified by the user. Batch data
 71     is converted to PostgreSQL records using the `json_populate_recordset`
 72     function. Apibara requires a `_cursor` column in the table to keep track of
 73     each batch's cursor, so that data can be invalidated in case of chain
 74     reorganizations.
 75   - **MongoDB**: write data to the _collection_ specified by the user. Data is
 76     converted to BSON and then written to the collection. Apibara adds a
 77     `_cursor` column to each record so that data can be invalidated in case of
 78     chain reorganizations.
 79  
 80  If you'd like us to add a specific database, feel free to [open an issue on
 81  GitHub](https://github.com/apibara/dna/issues).
 82  
 83  
 84  ### Dataset generation
 85  
 86  Apibara is the easiest and fastest way to generate all the datasets that your
 87  analytics team needs. Generate exactly the data you need using the _filter_ and
 88  _transform_ steps. After that, the integration will start streaming all
 89  historical data and will keep your dataset updated as the chain moves forward.
 90  
 91  At the moment, datasets can only be generated locally. In the future, we plan
 92  to add the ability to automatically upload the datasets to any S3-compatible
 93  storage.
 94  
 95  Apibara supports generating datasets in the following formats:
 96  
 97   - **Apache Parquet**: generate Parquet files, the schema is automatically
 98     deduced from the first batch of data. Apibara groups multiple blocks of data
 99     into a single file.
100