1  # Database
  2  
  3  Fedimint uses a simple key-value store as its database. In theory any such KV store with the following features can be used:
  4  
  5  * insert, update, delete actions
  6  * transactions
  7  * key prefix search
  8  * optimistic transactions
  9  
 10  We currently use RocksDB on both the client and the server.
 11  
 12  ## Server DB Layout
 13  The database is logically partitioned based on the module instance id of the module. Each module's keyspace is split up based on prefixing. Each prefix within a module identifies a logical entity.
 14  Entity prefixes between modules can overlap because they are logically partitioned by the module instance id. The key value pairs that belong to consensus do not belong to any specific module, so they are
 15  not prepended with a module prefix. Below is the format for prefixes within a module:
 16  
 17  <GLOBAL PREFIX BYTE><2 BYTE MODULE ID><ENTITY PREFIX>
 18  
 19  Example for the Mint Module: 0xFF 0x00 0x00 0x10
 20  In the above example, the module instance id = 0 and the entity it identifies is a NoteNonce, because it uses the 0x10 byte. 0xFF is the global prefix byte that identifies this as module data.
 21  
 22  Example for consensus data: 0x01
 23  In the above example, because the consensus data does not apply to any specific module, the global prefix byte and module instance id prefixes are missing.
 24  
 25  The client uses the same isolation mechanism as `fedimintd` to store data for each module.
 26  
 27  
 28  ## Database Transactions
 29  In Fedimint, all interactions with the database use a database transaction. Database transactions are an abstraction
 30  for accessing the database in an atomic, consistent, and isolated way. Underneath, Fedimint uses RocksDb's optimistic transactions
 31  which means database transactions are allowed to read and write to the database concurrently. If there are two concurrent transactions
 32  that modify the same key, RocksDb's optimistic transactions will detect this "write-write" conflict and cause the transaction that
 33  commits second to fail. 
 34  
 35  Fedimint has defined a number of different structs for implementing the necessary functionality for database transactions. These structs
 36  follow the [adapter pattern](https://en.wikipedia.org/wiki/Adapter_pattern) to wrap and isolate the features. At the bottom is an explanation of
 37  each interface/struct.
 38  
 39  ## Migrations
 40  In order to avoid breaking changes, `fedimintd`, `gatewayd`, and the client must know of the structure of the data written to disk. If a code upgrade
 41  has occurred, it is possible that the new version of the code expects the data written to disk to be structured differently. When this happens, a database
 42  migration must occur to maintain backwards compatibility. Migrations are defined on a per-module basis in the `get_database_migrations` function and applied
 43  using `apply_migrations`.
 44  
 45  Since introducing a database breaking change is easy (just modifying a struct), tests have been introduced to catch DB breaking changes. `just prepare_db_migration_snapshots` will prepare a database backup of dummy data for a module. `test_migrations` will try to read from this database backup. If the
 46  structure of the data has changed and the backup cannot be reading, this test will fail.
 47  
 48  There are some times when making a DB breaking change (not backwards compatible) is intentional. In that case, to fix the migration tests, `just prepare_db_migration_snapshot` needs to be updated
 49  to reflect the new structure of the data. Then, the db/ folder at the root of the repository needs to be deleted. Then `just prepare_db_migration_snapshot` can
 50  be run to re-generate the database backup. `test_migrations` will need to be updated to read the newly added/modified data.
 51  
 52  ### Interfaces
 53  
 54   - `IRawDatabase` and `IRawDatabaseTransaction` - The interfaces raw database crates implement.
 55   - `IDatabase` and `IDatabaseTransaction` - The interfaces including key subscribe & notify functionality, added on top of databases.
 56   - `IDatabaseTransactionOps` and `IDatabaseTransactionCoreOps` - The interfaces of database transaction operations
 57   - `IDatabaseTransactionOpsCoreTyped` - Like `IDatabaseTransactionOps` but with typed keys and values. Implemented generically over everything implements `IDatabaseTransactionOps`.
 58  
 59  ### Structs
 60  #### Public
 61  
 62   - `Database` and `DatabaseTransaction` - Public facing newtypes over `IDatabase` and `IDatabaseTransaction` that also holds `decoders` and minor helper logic.
 63   - `DatabaseTransactionRef` - A logical reference to `DatabaseTransaction` that does not expose the commit operation.
 64  
 65  #### Internal
 66  
 67   - `BaseDatabase` and `BaseDatabaseTransaction` - Adapter implementing `IDatabase` for `IRawDatabase`
 68   - `PrefixDatabase` and `PrefixDatabaseTransaction` - Adapter over `IDatabase` and `IDatabaseTransaction` implementing key prefix handling to provide database partitioning/isolation.
 69  
 70  #### Raw Implementations
 71  
 72   - `MemDatabase` and `MemDatabaseTransaction` - Base implementation of an in-memory database transaction.
 73   - `RocksDbDatabase` and `RocksDbDatabaseTransaction` - Base implementation of a Rocksdb database. Uses optimistic transaction internally.
 74   - `RocksDbReadOnly` and `RocksDbReadOnlyTransaction` - Base implementation of a Rocksdb read-only database. Will panic on writes.
 75  
 76  ```mermaid
 77  classDiagram
 78      DatabaseTransaction ..* IDatabaseTransaction : wraps
 79      DatabaseTransactionRef ..* DatabaseTransaction : wraps
 80      PrefixDatabaseTransaction ..|> IDatabaseTransaction : implements
 81      PrefixDatabaseTransaction ..* IDatabaseTransaction : wraps
 82      BaseDatabaseTransaction ..|> IDatabaseTransaction : implements
 83      BaseDatabaseTransaction ..* IRawDatabaseTransaction : wraps
 84      MemTransaction ..|> IRawDatabaseTransaction : implements
 85      RocksDbTransaction ..|> IRawDatabaseTransaction : implements
 86  
 87      DatabaseTransactionRef ..|> IDatabaseTransactionOpsCore : implements
 88      DatabaseTransaction ..|> IDatabaseTransactionOpsCore : implements
 89      BaseDatabaseTransaction ..|> IDatabaseTransactionOpsCore : implements
 90      PrefixDatabaseTransaction ..|> IDatabaseTransactionOpsCore : implements
 91      MemTransaction ..|> IDatabaseTransactionOpsCore : implements
 92      RocksDbTransaction ..|> IDatabaseTransactionOpsCore : implements
 93  
 94      class IDatabaseTransactionOpsCore {
 95        <<interface>>
 96        + raw_insert_bytes()
 97        + raw_get_bytes()
 98        + raw_remove()
 99      }
100  
101      class IDatabaseTransaction {
102        <<interface>>
103        + commit_tx()
104      }
105  
106      class DatabaseTransaction {
107        - IDatabaseTransaction
108      }
109  
110      class DatabaseTransactionRef {
111        - &DatabaseTransaction
112      }
113  
114      class PrefixDatabaseTransaction {
115        - IDatabaseTransaction
116      }
117  
118      class BaseDatabaseTransaction {
119        - IRawDatabaseTransaction
120      }
121  
122      class RocksDbTransaction {
123      }
124  
125      class MemTransaction {
126      }