fuzzing.md
1 # Fuzzing Bitcoin Core using libFuzzer 2 3 ## Quickstart guide 4 5 To quickly get started fuzzing Bitcoin Core using [libFuzzer](https://llvm.org/docs/LibFuzzer.html): 6 7 ```sh 8 $ git clone https://github.com/bitcoin/bitcoin 9 $ cd bitcoin/ 10 $ cmake --preset=libfuzzer 11 $ cmake --build build_fuzz 12 $ FUZZ=process_message build_fuzz/bin/fuzz 13 # abort fuzzing using ctrl-c 14 ``` 15 16 One can use `--preset=libfuzzer-nosan` to do the same without common sanitizers enabled. 17 See [further](#run-without-sanitizers-for-increased-throughput) for more information. 18 19 There is also a runner script to execute all fuzz targets. Refer to 20 `./build_fuzz/test/fuzz/test_runner.py --help` for more details. 21 22 For source-based coverage reports, see [developer notes](/doc/developer-notes.md#compiling-for-fuzz-coverage). 23 24 macOS users: We recommend fuzzing on Linux, see [macOS notes](#macos-notes) for 25 more information. 26 27 ## Overview of Bitcoin Core fuzzing 28 29 [Google](https://github.com/google/fuzzing/) has a good overview of fuzzing in general, with contributions from key architects of some of the most-used fuzzers. [This paper](https://agroce.github.io/bitcoin_report.pdf) includes an external overview of the status of Bitcoin Core fuzzing, as of summer 2021. [John Regehr](https://blog.regehr.org/archives/1687) provides good advice on writing code that assists fuzzers in finding bugs, which is useful for developers to keep in mind. 30 31 ## Fuzzing harnesses and output 32 33 [`process_message`](/src/test/fuzz/process_message.cpp) is a fuzzing harness for the [`ProcessMessage(...)` function (`net_processing`)](/src/net_processing.cpp). The available fuzzing harnesses are found in [`src/test/fuzz/`](/src/test/fuzz). 34 35 The fuzzer will output `NEW` every time it has created a test input that covers new areas of the code under test. For more information on how to interpret the fuzzer output, see the [libFuzzer documentation](https://llvm.org/docs/LibFuzzer.html). 36 37 If you specify a corpus directory then any new coverage increasing inputs will be saved there: 38 39 ```sh 40 $ mkdir -p process_message-seeded-from-thin-air/ 41 $ FUZZ=process_message build_fuzz/bin/fuzz process_message-seeded-from-thin-air/ 42 INFO: Seed: 840522292 43 INFO: Loaded 1 modules (424174 inline 8-bit counters): 424174 [0x55e121ef9ab8, 0x55e121f613a6), 44 INFO: Loaded 1 PC tables (424174 PCs): 424174 [0x55e121f613a8,0x55e1225da288), 45 INFO: 0 files found in process_message-seeded-from-thin-air/ 46 INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes 47 INFO: A corpus is not provided, starting from an empty corpus 48 #2 INITED cov: 94 ft: 95 corp: 1/1b exec/s: 0 rss: 150Mb 49 #3 NEW cov: 95 ft: 96 corp: 2/3b lim: 4 exec/s: 0 rss: 150Mb L: 2/2 MS: 1 InsertByte- 50 #4 NEW cov: 96 ft: 98 corp: 3/7b lim: 4 exec/s: 0 rss: 150Mb L: 4/4 MS: 1 CrossOver- 51 #21 NEW cov: 96 ft: 100 corp: 4/11b lim: 4 exec/s: 0 rss: 150Mb L: 4/4 MS: 2 ChangeBit-CrossOver- 52 #324 NEW cov: 101 ft: 105 corp: 5/12b lim: 6 exec/s: 0 rss: 150Mb L: 6/6 MS: 5 CrossOver-ChangeBit-CopyPart-ChangeBit-ChangeBinInt- 53 #1239 REDUCE cov: 102 ft: 106 corp: 6/24b lim: 14 exec/s: 0 rss: 150Mb L: 13/13 MS: 5 ChangeBit-CrossOver-EraseBytes-ChangeBit-InsertRepeatedBytes- 54 #1272 REDUCE cov: 102 ft: 106 corp: 6/23b lim: 14 exec/s: 0 rss: 150Mb L: 12/12 MS: 3 ChangeBinInt-ChangeBit-EraseBytes- 55 NEW_FUNC[1/677]: 0x55e11f456690 in std::_Function_base::~_Function_base() /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/std_function.h:255 56 NEW_FUNC[2/677]: 0x55e11f465800 in CDataStream::CDataStream(std::vector<unsigned char, std::allocator<unsigned char> > const&, int, int) src/./streams.h:248 57 #2125 REDUCE cov: 4820 ft: 4867 corp: 7/29b lim: 21 exec/s: 0 rss: 155Mb L: 6/12 MS: 2 CopyPart-CMP- DE: "block"- 58 NEW_FUNC[1/9]: 0x55e11f64d790 in std::_Rb_tree<uint256, std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > >, std::_Select1st<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > >, std::less<uint256>, std::allocator<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > > >::~_Rb_tree() /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/stl_tree.h:972 59 NEW_FUNC[2/9]: 0x55e11f64d870 in std::_Rb_tree<uint256, std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > >, std::_Select1st<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > >, std::less<uint256>, std::allocator<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > > >::_M_erase(std::_Rb_tree_node<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > >*) /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/stl_tree.h:1875 60 #2228 NEW cov: 4898 ft: 4971 corp: 8/35b lim: 21 exec/s: 0 rss: 156Mb L: 6/12 MS: 3 EraseBytes-CopyPart-PersAutoDict- DE: "block"- 61 NEW_FUNC[1/5]: 0x55e11f46df70 in std::enable_if<__and_<std::allocator_traits<zero_after_free_allocator<char> >::__construct_helper<char, unsigned char const&>::type>::value, void>::type std::allocator_traits<zero_after_free_allocator<char> >::_S_construct<char, unsigned char const&>(zero_after_free_allocator<char>&, char*, unsigned char const&) /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/alloc_traits.h:243 62 NEW_FUNC[2/5]: 0x55e11f477390 in std::vector<unsigned char, std::allocator<unsigned char> >::data() /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/stl_vector.h:1056 63 #2456 NEW cov: 4933 ft: 5042 corp: 9/55b lim: 21 exec/s: 0 rss: 160Mb L: 20/20 MS: 3 ChangeByte-InsertRepeatedBytes-PersAutoDict- DE: "block"- 64 #2467 NEW cov: 4933 ft: 5043 corp: 10/76b lim: 21 exec/s: 0 rss: 161Mb L: 21/21 MS: 1 InsertByte- 65 #4215 NEW cov: 4941 ft: 5129 corp: 17/205b lim: 29 exec/s: 4215 rss: 350Mb L: 29/29 MS: 5 InsertByte-ChangeBit-CopyPart-InsertRepeatedBytes-CrossOver- 66 #4567 REDUCE cov: 4941 ft: 5129 corp: 17/204b lim: 29 exec/s: 4567 rss: 404Mb L: 24/29 MS: 2 ChangeByte-EraseBytes- 67 #6642 NEW cov: 4941 ft: 5138 corp: 18/244b lim: 43 exec/s: 2214 rss: 450Mb L: 43/43 MS: 3 CopyPart-CMP-CrossOver- DE: "verack"- 68 # abort fuzzing using ctrl-c 69 $ ls process_message-seeded-from-thin-air/ 70 349ac589fc66a09abc0b72bb4ae445a7a19e2cd8 4df479f1f421f2ea64b383cd4919a272604087a7 71 a640312c98dcc55d6744730c33e41c5168c55f09 b135de16e4709558c0797c15f86046d31c5d86d7 72 c000f7b41b05139de8b63f4cbf7d1ad4c6e2aa7f fc52cc00ec1eb1c08470e69f809ae4993fa70082 73 $ cat --show-nonprinting process_message-seeded-from-thin-air/349ac589fc66a09abc0b72bb4ae445a7a19e2cd8 74 block^@M-^?M-^?M-^?M-^?M-^?nM-^?M-^? 75 ``` 76 77 In this case the fuzzer managed to create a `block` message which when passed to `ProcessMessage(...)` increased coverage. 78 79 It is possible to specify `bitcoind` arguments to the `fuzz` executable. 80 Depending on the test, they may be ignored or consumed and alter the behavior 81 of the test. Just make sure to use double-dash to distinguish them from the 82 fuzzer's own arguments: 83 84 ```sh 85 $ FUZZ=address_deserialize build_fuzz/bin/fuzz -runs=1 fuzz_corpora/address_deserialize --checkaddrman=5 --printtoconsole=1 86 ``` 87 88 ## Fuzzing corpora 89 90 The project's collection of seed corpora is found in the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo. 91 92 To fuzz `process_message` using the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) seed corpus: 93 94 ```sh 95 $ git clone --depth=1 https://github.com/bitcoin-core/qa-assets 96 $ FUZZ=process_message build_fuzz/bin/fuzz qa-assets/fuzz_corpora/process_message/ 97 INFO: Seed: 1346407872 98 INFO: Loaded 1 modules (424174 inline 8-bit counters): 424174 [0x55d8a9004ab8, 0x55d8a906c3a6), 99 INFO: Loaded 1 PC tables (424174 PCs): 424174 [0x55d8a906c3a8,0x55d8a96e5288), 100 INFO: 991 files found in qa-assets/fuzz_corpora/process_message/ 101 INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes 102 INFO: seed corpus: files: 991 min: 1b max: 1858b total: 288291b rss: 150Mb 103 #993 INITED cov: 7063 ft: 8236 corp: 25/3821b exec/s: 0 rss: 181Mb 104 … 105 ``` 106 107 ## Using the MemorySanitizer (MSan) 108 109 MSan [requires](https://clang.llvm.org/docs/MemorySanitizer.html#handling-external-code) 110 that all linked code be instrumented. The exact steps to achieve this may vary 111 but involve compiling `clang` from source, using the built `clang` to compile 112 an instrumentalized libc++, then using it to build [Bitcoin Core dependencies 113 from source](../depends/README.md) and finally the Bitcoin Core fuzz binary 114 itself. One can use the MSan CI job as an example for how to perform these 115 steps. 116 117 Valgrind is an alternative to MSan that does not require building a custom libc++. 118 119 ## Run without sanitizers for increased throughput 120 121 Fuzzing on a harness compiled with `-DSANITIZERS=address,fuzzer,undefined` is 122 good for finding bugs. However, the very slow execution even under libFuzzer 123 will limit the ability to find new coverage. A good approach is to perform 124 occasional long runs without the additional bug-detectors 125 (`--preset=libfuzzer-nosan`) and then merge new inputs into a corpus as described in 126 the qa-assets repo 127 (https://github.com/bitcoin-core/qa-assets/blob/main/.github/PULL_REQUEST_TEMPLATE.md). 128 Patience is useful; even with improved throughput, libFuzzer may need days and 129 10s of millions of executions to reach deep/hard targets. 130 131 ## Reproduce a fuzzer crash reported by the CI 132 133 - `cd` into the `qa-assets` directory and update it with `git pull qa-assets` 134 - locate the crash case described in the CI output, e.g. `Test unit written to 135 ./crash-1bc91feec9fc00b107d97dc225a9f2cdaa078eb6` 136 - make sure to compile with all sanitizers, if they are needed (fuzzing runs 137 more slowly with sanitizers enabled, but a crash should be reproducible very 138 quickly from a crash case) 139 - run the fuzzer with the case number appended to the seed corpus path: 140 `FUZZ=process_message build_fuzz/bin/fuzz 141 qa-assets/fuzz_corpora/process_message/1bc91feec9fc00b107d97dc225a9f2cdaa078eb6` 142 - If the file does not exist, make sure you are checking out the exact same commit id 143 for the qa-assets repo. If the file was found while running the fuzz engine in the CI, 144 you should be able to reproduce the crash locally with the same (or a similar input) 145 within a few minutes. Alternatively, you can use the base64 encoded file from the CI log, 146 if it exists. e.g. 147 `echo "Nb6Fc/97AACAAAD/ewAAgAAAAIAAAACAAAAAoA==" | 148 base64 --decode > qa-assets/fuzz_corpora/process_message/1bc91feec9fc00b107d97dc225a9f2cdaa078eb6` 149 150 ## Submit improved coverage 151 152 If you find coverage increasing inputs when fuzzing you are highly encouraged to submit them for inclusion in the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo. 153 154 Every single pull request submitted against the Bitcoin Core repo is automatically tested against all inputs in the [`bitcoin-core/qa-assets`](https://github.com/bitcoin-core/qa-assets) repo. Contributing new coverage increasing inputs is an easy way to help make Bitcoin Core more robust. 155 156 ## Building and debugging fuzz tests 157 158 There are 3 ways fuzz tests can be built: 159 160 1. With `-DBUILD_FOR_FUZZING=ON` which forces on fuzz determinism (skipping 161 proof of work checks, disabling random number seeding, disabling clock time) 162 and causes `Assume()` checks to abort on failure. 163 164 This is the normal way to run fuzz tests and generate new inputs. Because 165 determinism is hardcoded on in this build, only the fuzz binary can be built 166 and all other binaries are disabled. 167 168 2. With `-DBUILD_FUZZ_BINARY=ON -DCMAKE_BUILD_TYPE=Debug` which causes 169 `Assume()` checks to abort on failure, and enables fuzz determinism, but 170 makes it optional. 171 172 Determinism is turned on in the fuzz binary by default, but can be turned off 173 by setting the `FUZZ_NONDETERMINISM` environment variable to any value, which 174 may be useful for running fuzz tests with code that deterministic execution 175 would otherwise skip. 176 177 Since `BUILD_FUZZ_BINARY`, unlike `BUILD_FOR_FUZZING`, does not hardcode on 178 determinism, this allows non-fuzz binaries to coexist in the same build, 179 making it possible to reproduce fuzz test failures in a normal build. 180 181 3. With `-DBUILD_FUZZ_BINARY=ON -DCMAKE_BUILD_TYPE=Release`. In this build, the 182 fuzz binary will build but refuse to run, because in release builds 183 determinism is forced off and `Assume()` checks do not abort, so running the 184 tests would not be useful. This build is only useful for ensuring fuzz tests 185 compile and link. 186 187 ## macOS notes 188 189 Support for fuzzing on macOS is not officially maintained by this project. If 190 you are running into issues on macOS, we recommend fuzzing on Linux instead for 191 best results. On macOS this can be done within Docker or a virtual machine. 192 193 Reproducing and debugging fuzz testcases on macOS is supported, by building the 194 fuzz binary without support for any specific fuzzing engine. 195 196 # Fuzzing Bitcoin Core using afl++ 197 198 ## Quickstart guide 199 200 To quickly get started fuzzing Bitcoin Core using [afl++](https://github.com/AFLplusplus/AFLplusplus): 201 202 ```sh 203 $ git clone https://github.com/bitcoin/bitcoin 204 $ cd bitcoin/ 205 $ git clone https://github.com/AFLplusplus/AFLplusplus 206 $ make -C AFLplusplus/ source-only 207 # If afl-clang-lto is not available, see 208 # https://github.com/AFLplusplus/AFLplusplus/blob/stable/docs/fuzzing_in_depth.md#a-selecting-the-best-afl-compiler-for-instrumenting-the-target 209 $ cmake -B build_fuzz \ 210 -DCMAKE_C_COMPILER="$(pwd)/AFLplusplus/afl-clang-lto" \ 211 -DCMAKE_CXX_COMPILER="$(pwd)/AFLplusplus/afl-clang-lto++" \ 212 -DBUILD_FOR_FUZZING=ON 213 $ cmake --build build_fuzz 214 $ mkdir -p inputs/ outputs/ 215 $ echo A > inputs/thin-air-input 216 $ FUZZ=bech32 ./AFLplusplus/afl-fuzz -i inputs/ -o outputs/ -- build_fuzz/bin/fuzz 217 # You may have to change a few kernel parameters to test optimally - afl-fuzz 218 # will print an error and suggestion if so. 219 ``` 220 221 Read the [afl++ documentation](https://github.com/AFLplusplus/AFLplusplus) for more information. 222 223 # Fuzzing Bitcoin Core using Honggfuzz 224 225 ## Quickstart guide 226 227 To quickly get started fuzzing Bitcoin Core using [Honggfuzz](https://github.com/google/honggfuzz): 228 229 ```sh 230 $ git clone https://github.com/bitcoin/bitcoin 231 $ cd bitcoin/ 232 $ git clone https://github.com/google/honggfuzz 233 $ cd honggfuzz/ 234 $ make 235 $ cd .. 236 $ cmake -B build_fuzz \ 237 -DCMAKE_C_COMPILER="$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang" \ 238 -DCMAKE_CXX_COMPILER="$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang++" \ 239 -DBUILD_FOR_FUZZING=ON \ 240 -DSANITIZERS=address,undefined 241 $ cmake --build build_fuzz 242 $ mkdir -p inputs/ 243 $ FUZZ=process_message ./honggfuzz/honggfuzz -i inputs/ -- build_fuzz/bin/fuzz 244 ``` 245 246 Read the [Honggfuzz documentation](https://github.com/google/honggfuzz/blob/master/docs/USAGE.md) for more information. 247 248 # OSS-Fuzz 249 250 Bitcoin Core participates in Google's [OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/bitcoin-core) 251 program, which includes a dashboard of [publicly disclosed vulnerabilities](https://issues.oss-fuzz.com/issues?q=bitcoin-core%20status:open). 252 253 Bitcoin Core follows its [security disclosure policy](https://bitcoincore.org/en/security-advisories/), 254 which may differ from Google's standard 255 [90-day disclosure window](https://google.github.io/oss-fuzz/getting-started/bug-disclosure-guidelines/) 256 . 257 258 OSS-Fuzz also produces [a fuzzing coverage report](https://oss-fuzz.com/coverage-report/job/libfuzzer_asan_bitcoin-core/latest).