README.md
1 **LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.** 2 3 [](https://travis-ci.org/google/leveldb) 4 [](https://ci.appveyor.com/project/pwnall/leveldb) 5 6 Authors: Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com) 7 8 # Features 9 10 * Keys and values are arbitrary byte arrays. 11 * Data is stored sorted by key. 12 * Callers can provide a custom comparison function to override the sort order. 13 * The basic operations are `Put(key,value)`, `Get(key)`, `Delete(key)`. 14 * Multiple changes can be made in one atomic batch. 15 * Users can create a transient snapshot to get a consistent view of data. 16 * Forward and backward iteration is supported over the data. 17 * Data is automatically compressed using the [Snappy compression library](http://google.github.io/snappy/). 18 * External activity (file system operations etc.) is relayed through a virtual interface so users can customize the operating system interactions. 19 20 # Documentation 21 22 [LevelDB library documentation](https://github.com/google/leveldb/blob/master/doc/index.md) is online and bundled with the source code. 23 24 # Limitations 25 26 * This is not a SQL database. It does not have a relational data model, it does not support SQL queries, and it has no support for indexes. 27 * Only a single process (possibly multi-threaded) can access a particular database at a time. 28 * There is no client-server support builtin to the library. An application that needs such support will have to wrap their own server around the library. 29 30 # Building 31 32 This project supports [CMake](https://cmake.org/) out of the box. 33 34 ### Build for POSIX 35 36 Quick start: 37 38 ```bash 39 mkdir -p build && cd build 40 cmake -DCMAKE_BUILD_TYPE=Release .. && cmake --build . 41 ``` 42 43 ### Building for Windows 44 45 First generate the Visual Studio 2017 project/solution files: 46 47 ```cmd 48 mkdir build 49 cd build 50 cmake -G "Visual Studio 15" .. 51 ``` 52 The default will build for x86. For 64-bit run: 53 54 ```cmd 55 cmake -G "Visual Studio 15 Win64" .. 56 ``` 57 58 To compile the Windows solution from the command-line: 59 60 ```cmd 61 devenv /build Debug leveldb.sln 62 ``` 63 64 or open leveldb.sln in Visual Studio and build from within. 65 66 Please see the CMake documentation and `CMakeLists.txt` for more advanced usage. 67 68 # Contributing to the leveldb Project 69 70 The leveldb project welcomes contributions. leveldb's primary goal is to be 71 a reliable and fast key/value store. Changes that are in line with the 72 features/limitations outlined above, and meet the requirements below, 73 will be considered. 74 75 Contribution requirements: 76 77 1. **Tested platforms only**. We _generally_ will only accept changes for 78 platforms that are compiled and tested. This means POSIX (for Linux and 79 macOS) or Windows. Very small changes will sometimes be accepted, but 80 consider that more of an exception than the rule. 81 82 2. **Stable API**. We strive very hard to maintain a stable API. Changes that 83 require changes for projects using leveldb _might_ be rejected without 84 sufficient benefit to the project. 85 86 3. **Tests**: All changes must be accompanied by a new (or changed) test, or 87 a sufficient explanation as to why a new (or changed) test is not required. 88 89 4. **Consistent Style**: This project conforms to the 90 [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html). 91 To ensure your changes are properly formatted please run: 92 93 ``` 94 clang-format -i --style=file <file> 95 ``` 96 97 ## Submitting a Pull Request 98 99 Before any pull request will be accepted the author must first sign a 100 Contributor License Agreement (CLA) at https://cla.developers.google.com/. 101 102 In order to keep the commit timeline linear 103 [squash](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#Squashing-Commits) 104 your changes down to a single commit and [rebase](https://git-scm.com/docs/git-rebase) 105 on google/leveldb/master. This keeps the commit timeline linear and more easily sync'ed 106 with the internal repository at Google. More information at GitHub's 107 [About Git rebase](https://help.github.com/articles/about-git-rebase/) page. 108 109 # Performance 110 111 Here is a performance report (with explanations) from the run of the 112 included db_bench program. The results are somewhat noisy, but should 113 be enough to get a ballpark performance estimate. 114 115 ## Setup 116 117 We use a database with a million entries. Each entry has a 16 byte 118 key, and a 100 byte value. Values used by the benchmark compress to 119 about half their original size. 120 121 LevelDB: version 1.1 122 Date: Sun May 1 12:11:26 2011 123 CPU: 4 x Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz 124 CPUCache: 4096 KB 125 Keys: 16 bytes each 126 Values: 100 bytes each (50 bytes after compression) 127 Entries: 1000000 128 Raw Size: 110.6 MB (estimated) 129 File Size: 62.9 MB (estimated) 130 131 ## Write performance 132 133 The "fill" benchmarks create a brand new database, in either 134 sequential, or random order. The "fillsync" benchmark flushes data 135 from the operating system to the disk after every operation; the other 136 write operations leave the data sitting in the operating system buffer 137 cache for a while. The "overwrite" benchmark does random writes that 138 update existing keys in the database. 139 140 fillseq : 1.765 micros/op; 62.7 MB/s 141 fillsync : 268.409 micros/op; 0.4 MB/s (10000 ops) 142 fillrandom : 2.460 micros/op; 45.0 MB/s 143 overwrite : 2.380 micros/op; 46.5 MB/s 144 145 Each "op" above corresponds to a write of a single key/value pair. 146 I.e., a random write benchmark goes at approximately 400,000 writes per second. 147 148 Each "fillsync" operation costs much less (0.3 millisecond) 149 than a disk seek (typically 10 milliseconds). We suspect that this is 150 because the hard disk itself is buffering the update in its memory and 151 responding before the data has been written to the platter. This may 152 or may not be safe based on whether or not the hard disk has enough 153 power to save its memory in the event of a power failure. 154 155 ## Read performance 156 157 We list the performance of reading sequentially in both the forward 158 and reverse direction, and also the performance of a random lookup. 159 Note that the database created by the benchmark is quite small. 160 Therefore the report characterizes the performance of leveldb when the 161 working set fits in memory. The cost of reading a piece of data that 162 is not present in the operating system buffer cache will be dominated 163 by the one or two disk seeks needed to fetch the data from disk. 164 Write performance will be mostly unaffected by whether or not the 165 working set fits in memory. 166 167 readrandom : 16.677 micros/op; (approximately 60,000 reads per second) 168 readseq : 0.476 micros/op; 232.3 MB/s 169 readreverse : 0.724 micros/op; 152.9 MB/s 170 171 LevelDB compacts its underlying storage data in the background to 172 improve read performance. The results listed above were done 173 immediately after a lot of random writes. The results after 174 compactions (which are usually triggered automatically) are better. 175 176 readrandom : 11.602 micros/op; (approximately 85,000 reads per second) 177 readseq : 0.423 micros/op; 261.8 MB/s 178 readreverse : 0.663 micros/op; 166.9 MB/s 179 180 Some of the high cost of reads comes from repeated decompression of blocks 181 read from disk. If we supply enough cache to the leveldb so it can hold the 182 uncompressed blocks in memory, the read performance improves again: 183 184 readrandom : 9.775 micros/op; (approximately 100,000 reads per second before compaction) 185 readrandom : 5.215 micros/op; (approximately 190,000 reads per second after compaction) 186 187 ## Repository contents 188 189 See [doc/index.md](doc/index.md) for more explanation. See 190 [doc/impl.md](doc/impl.md) for a brief overview of the implementation. 191 192 The public interface is in include/leveldb/*.h. Callers should not include or 193 rely on the details of any other header files in this package. Those 194 internal APIs may be changed without warning. 195 196 Guide to header files: 197 198 * **include/leveldb/db.h**: Main interface to the DB: Start here. 199 200 * **include/leveldb/options.h**: Control over the behavior of an entire database, 201 and also control over the behavior of individual reads and writes. 202 203 * **include/leveldb/comparator.h**: Abstraction for user-specified comparison function. 204 If you want just bytewise comparison of keys, you can use the default 205 comparator, but clients can write their own comparator implementations if they 206 want custom ordering (e.g. to handle different character encodings, etc.). 207 208 * **include/leveldb/iterator.h**: Interface for iterating over data. You can get 209 an iterator from a DB object. 210 211 * **include/leveldb/write_batch.h**: Interface for atomically applying multiple 212 updates to a database. 213 214 * **include/leveldb/slice.h**: A simple module for maintaining a pointer and a 215 length into some other byte array. 216 217 * **include/leveldb/status.h**: Status is returned from many of the public interfaces 218 and is used to report success and various kinds of errors. 219 220 * **include/leveldb/env.h**: 221 Abstraction of the OS environment. A posix implementation of this interface is 222 in util/env_posix.cc. 223 224 * **include/leveldb/table.h, include/leveldb/table_builder.h**: Lower-level modules that most 225 clients probably won't use directly.