release-notes-0.14.0.md
1 # Release Notes 2 3 ## Networking & Tor 4 5 ### Connectivity mode 6 7 A new flag has been added to enable a hybrid tor connectivity mode, where tor 8 is only used for onion address connections, and clearnet for everything else. 9 This new behavior can be added using the `tor.skip-proxy-for-clearnet-targets` 10 flag. 11 12 ### Onion service 13 14 The Onion service created upon lnd startup is [now deleted during lnd shutdown 15 using `DEL_ONION`](https://github.com/lightningnetwork/lnd/pull/5794). 16 17 ### Tor connection 18 19 A new health check, tor connection, [is added to lnd's liveness monitor upon 20 startup](https://github.com/lightningnetwork/lnd/pull/5794). This check will 21 ensure the liveness of the connection between the Tor daemon and lnd's tor 22 controller. To enable it, please use the following flags, 23 ``` 24 healthcheck.torconnection.attempts=xxx 25 healthcheck.torconnection.timeout=xxx 26 healthcheck.torconnection.backoff=xxx 27 healthcheck.torconnection.internal=xxx 28 ``` 29 Read more about the usage of the flags in the `sample-lnd.conf`. 30 31 ## LN Peer-to-Peer Network 32 33 ### Bitcoin Blockheaders in Ping Messages 34 35 [In this release, we implement a long discussed mechanism to use the Lightning 36 Network as a redundant block header 37 source](https://github.com/lightningnetwork/lnd/pull/5621). By sending our 38 latest block header with each ping message, we give peers another source 39 (outside of the Bitcoin P2P network) they can use to spot check their chain 40 state. Peers can also use this information to detect if they've been eclipsed 41 from the traditional Bitcoin P2P network itself. 42 43 As is, we only send this data in Ping messages (which are periodically sent), 44 in the future we could also move to send them as the partial payload for our 45 pong messages, and also randomize the payload size requested as well. 46 47 The `ListPeers` RPC call will now also include a hex encoded version of the 48 last ping message the peer has sent to us. 49 50 ## Backend Enhancements & Optimizations 51 52 ### Full remote database support 53 54 `lnd` now stores [all its data in the same remote/external 55 database](https://github.com/lightningnetwork/lnd/pull/5484) such as `etcd` 56 instead of only the channel state and wallet data. This makes `lnd` fully 57 stateless and therefore makes switching over to a new leader instance almost 58 instantaneous. Read the [guide on leader 59 election](https://github.com/lightningnetwork/lnd/blob/master/docs/leader_election.md) 60 for more information. 61 62 ### Postgres database support 63 64 This release adds [support for Postgres as a database 65 backend](https://github.com/lightningnetwork/lnd/pull/5366) to lnd. Postgres 66 has several advantages over the default bbolt backend: 67 * Better handling of large data sets. 68 * On-the-fly database compaction (auto vacuum). 69 * Database replication. 70 * Inspect data while lnd is running (bbolt opens the database exclusively). 71 * Usage of industry-standard tools to manage the stored data, get performance 72 metrics, etc. 73 74 Furthermore, the SQL platform opens up possibilities to improve lnd's 75 performance in the future. Bbolt's single-writer model is a severe performance 76 bottleneck, whereas Postgres offers a variety of locking models. Additionally, 77 structured tables reduce the need for custom serialization/deserialization code 78 in `lnd`, saving developer time and limiting the potential for bugs. 79 80 Instructions for enabling Postgres can be found in 81 [docs/postgres.md](../postgres.md). 82 83 ### In-memory path finding 84 85 Finding a path through the channel graph for sending a payment doesn't involve 86 any database queries anymore. The [channel graph is now kept fully 87 in-memory](https://github.com/lightningnetwork/lnd/pull/5642) for up a massive 88 performance boost when calling `QueryRoutes` or any of the `SendPayment` 89 variants. Keeping the full graph in memory naturally comes with increased RAM 90 usage. Users running `lnd` on low-memory systems are advised to run with the 91 `routing.strictgraphpruning=true` configuration option that more aggressively 92 removes zombie channels from the graph, reducing the number of channels that 93 need to be kept in memory. 94 95 There is a [fallback option](https://github.com/lightningnetwork/lnd/pull/5840) 96 `db.no-graph-cache=true` that can be used when running a Bolt (`bbolt`) based 97 database backend. Using the database for path finding is considerably slower 98 than using the in-memory graph cache but uses less RAM. The fallback option is 99 not available for `etcd` or Postgres database backends because of the way they 100 handle long-running database transactions that are required for the path finding 101 operations. 102 103 ## Protocol Extensions 104 105 ### Explicit Channel Negotiation 106 107 [A new protocol extension has been added known as explicit channel 108 negotiation](https://github.com/lightningnetwork/lnd/pull/5669). This allows a 109 channel initiator to signal their desired channel type to use with the remote 110 peer. If the remote peer supports said channel type and agrees, the previous 111 implicit negotiation based on the shared set of feature bits is bypassed, and 112 the proposed channel type is used. [Feature bits 44/45 are used to 113 signal](https://github.com/lightningnetwork/lnd/pull/5874) this new feature. 114 115 116 ### Script Enforced Channel Leases 117 118 [A new channel commitment type that builds upon the recently introduced anchors 119 commitment format has been introduced to back channel leases resulting from 120 Lightning Pool](https://github.com/lightningnetwork/lnd/pull/5709). This new 121 channel commitment type features an additional spend requirement on any 122 commitment and HTLC outputs that pay directly to the channel initiator. The 123 channel initiator must now wait for the channel lease maturity, expressed as an 124 absolute height of the chain, to be met before being able to sweep their funds, 125 on top of the usual CSV delay requirement. See the linked pull request for more 126 details. 127 128 ### Re-Usable Static AMP Invoices 129 130 [AMP invoices are now fully re-usable, meaning it's possible for an `lnd` node 131 to use a static AMP invoice multiple times](https://github.com/lightningnetwork/lnd/pull/5803). 132 An AMP invoice can be created by adding the `--amp` flag to `lncli addinvoice`. 133 From there repeated payments can be made to the invoice using `lncli 134 payinvoice`. On the receiver side, notifications will still come in as normal, 135 but notifying only the _new_ "sub-invoice" paid each time. 136 137 A new field has been added to the main `Invoice` proto that allows callers to 138 easily scan to see the current state of all repeated payments to a given 139 `payment_addr`. 140 141 A new `LookupInvoiceV2` RPC has been added to the `invoicerpcserver` which 142 allows callers to look up an AMP invoice by set_id, opting to only return 143 relevant HTLCs, or to look up an AMP invoice by its `payment_addr`, but omit 144 all HTLC information. The first option is useful when a caller wants to get 145 information specific to a repeated payment, omitting the thousands of possible 146 _other_ payment attempts. The second option is useful when a caller wants to 147 obtain the _base_ invoice information, and then use the first option to extract 148 the custom records (or other details) of the prior payment attempts. 149 150 ## RPC Server 151 152 * [Return payment address and add index from 153 addholdinvoice call](https://github.com/lightningnetwork/lnd/pull/5533). 154 155 * [The versions of several gRPC related libraries were bumped and the main 156 `rpc.proto` was renamed to 157 `lightning.proto`](https://github.com/lightningnetwork/lnd/pull/5473) to fix 158 a warning related to protobuf file name collisions. 159 160 * [Stub code for interacting with `lnrpc` from a WASM context through JSON 161 messages was added](https://github.com/lightningnetwork/lnd/pull/5601). 162 163 * The updatechanpolicy call now [detects invalid and pending channels, and 164 returns a policy update failure report](https://github.com/lightningnetwork/lnd/pull/5405). 165 166 * LND now [reports to systemd](https://github.com/lightningnetwork/lnd/pull/5536) 167 that RPC is ready (port bound, certificate generated, macaroons created, 168 in case of `wallet-unlock-password-file` wallet unlocked). This can be used to 169 avoid misleading error messages from dependent services if they use `After` 170 systemd option. 171 172 * [Delete a specific payment, or its failed HTLCs](https://github.com/lightningnetwork/lnd/pull/5660). 173 174 * A new state, [`WalletState_SERVER_ACTIVE`](https://github.com/lightningnetwork/lnd/pull/5637), 175 is added to the state server. This state indicates whether the `lnd` server 176 and all its subservers have been fully started or not. 177 178 * [Adds an option to the BakeMacaroon rpc "allow-external-permissions,"](https://github.com/lightningnetwork/lnd/pull/5304) which makes it possible to bake a macaroon with external permissions. That way, the baked macaroons can be used for services beyond LND. Also adds a new CheckMacaroonPermissions rpc that checks that the macaroon permissions and other restrictions are being followed. It can also check permissions not native to LND. 179 180 * [A new RPC middleware 181 interceptor](https://github.com/lightningnetwork/lnd/pull/5101) was added that 182 allows external tools to hook into `lnd`'s RPC server and intercept any 183 requests made with custom macaroons (and also the responses to those 184 requests). 185 186 * [Adds NOT_FOUND status code for LookupInvoice](https://github.com/lightningnetwork/lnd/pull/5768) 187 188 * The `FundingPsbtFinalize` step is a safety measure that assures the final 189 signed funding transaction has the same TXID as was registered during 190 the funding flow and was used for the commitment transactions. 191 This step is cumbersome to use if the whole funding process is completed 192 external to lnd. [We allow the finalize step to be 193 skipped](https://github.com/lightningnetwork/lnd/pull/5363) for such cases. 194 The API user/script will need to make sure things are verified (and possibly 195 cleaned up) properly. An example script was added to the [PSBT 196 documentation](../psbt.md) to show the simplified process. 197 198 ### Batched channel funding 199 200 [Multiple channels can now be opened in a single 201 transaction](https://github.com/lightningnetwork/lnd/pull/5356) in a safer and 202 more straightforward way by using the `BatchOpenChannel` RPC or the command line 203 version of that RPC called `lncli batchopenchannel`. More information can be 204 found in the [PSBT 205 documentation](../psbt.md#use-the-batchopenchannel-rpc-for-safe-batch-channel-funding). 206 207 ## Wallet 208 209 * It is now possible to fund a psbt [without specifying any 210 outputs](https://github.com/lightningnetwork/lnd/pull/5442). This option is 211 useful for CPFP bumping of unconfirmed outputs or general utxo consolidation. 212 * The internal wallet can now also be created or restored by using an [extended 213 master root key (`xprv`) instead of an 214 `aezeed`](https://github.com/lightningnetwork/lnd/pull/4717) only. This allows 215 wallet integrators to use existing seed mechanism that might already be in 216 place. **It is still not supported to use the same seed/root key on multiple 217 `lnd` instances simultaneously** though. 218 219 * [Publish transaction is now reachable through 220 lncli](https://github.com/lightningnetwork/lnd/pull/5460). 221 222 * Prior to this release, when running on `simnet` or `regtest`, `lnd` would 223 skip the check on wallet synchronization during its startup. In doing so, the 224 integration test can bypass the rule set by `bitcoind`, which considers the 225 node is out of sync when the last block is older than 2 hours([more 226 discussion](https://github.com/lightningnetwork/lnd/pull/4685#discussion_r503080709)). 227 This synchronization check is put back now as we want to make the integration 228 test more robust in catching real world situations. This also means it might 229 take longer to start an `lnd` node when running in `simnet` or `regtest`, 230 something developers need to watch out from this release. 231 232 ### Remote signing 233 234 It is now possible to delegate any operation that needs access to private keys 235 to a [remote signer that serves signing requests over 236 RPC](https://github.com/lightningnetwork/lnd/pull/5689). More information can be 237 found [in the new remote signing document](../remote-signing.md). 238 239 ## Security 240 241 * The release signature verification script [was overhauled to fix some possible 242 attack vectors and user 243 errors](https://github.com/lightningnetwork/lnd/pull/5053). The public keys 244 used to verify the signatures against are no longer downloaded form Keybase 245 but instead are kept in the `lnd` git repository. This allows for a more 246 transparent way of keeping track of changes to the signing keys. 247 248 ### Admin macaroon permissions 249 250 The default file permissions of admin.macaroon were [changed from 0600 to 251 0640](https://github.com/lightningnetwork/lnd/pull/5534). This makes it easier 252 to allow other users to manage LND. This is safe on common Unix systems 253 because they always create a new group for each user. 254 255 If you use a strange system or changed group membership of the group running LND 256 you may want to check your system to see if it introduces additional risk for 257 you. 258 259 ## Custom peer messages 260 261 Lightning nodes have a connection to each of their peers for exchanging 262 messages. In regular operation, these messages coordinate processes such as 263 channel opening and payment forwarding. 264 265 The lightning spec however also defines a custom range (>= 32768) for 266 experimental and application-specific peer messages. 267 268 With this release, [custom peer message 269 exchange](https://github.com/lightningnetwork/lnd/pull/5346) is added to open up 270 a range of new possibilities. Custom peer messages allow the lightning protocol 271 with its transport mechanisms (including tor) and public key authentication to 272 be leveraged for application-level communication. Note that peers exchange these 273 messages directly. There is no routing/path finding involved. 274 275 # Safety 276 277 * Locally force closed channels are now [kept in the channel.backup file until 278 their time lock has fully matured](https://github.com/lightningnetwork/lnd/pull/5528). 279 280 * [Cooperative closes optimistically shutdown the associated `link` before closing the channel.](https://github.com/lightningnetwork/lnd/pull/5618) 281 282 ## Build System 283 284 * [The illumos operating system has been dropped from the set of release binaries. We now also build with postgres and etcd in the main release binaries](https://github.com/lightningnetwork/lnd/pull/5985). 285 286 * [A new pre-submit check has been 287 added](https://github.com/lightningnetwork/lnd/pull/5520) to ensure that all 288 PRs ([aside from merge 289 commits](https://github.com/lightningnetwork/lnd/pull/5543)) add an entry in 290 the release notes folder that at least links to PR being added. 291 292 * [A new build target itest-race](https://github.com/lightningnetwork/lnd/pull/5542) 293 to help uncover undetected data races with our itests. 294 295 * [The itest error whitelist check was removed to reduce the number of failed 296 Travis builds](https://github.com/lightningnetwork/lnd/pull/5588). 297 298 * [A flake in the Neutrino integration tests with anchor sweeps was 299 addressed](https://github.com/lightningnetwork/lnd/pull/5509). 300 301 * [The `lnwire` fuzz tests have been fixed and now run without crashing.](https://github.com/lightningnetwork/lnd/pull/5395) 302 303 * [A flake in the race unit 304 tests](https://github.com/lightningnetwork/lnd/pull/5659) was addressed that 305 lead to failed tests sometimes when the CPU of the GitHub CI runner was 306 strained too much. 307 308 * [Reduce the number of parallel itest runs to 2 on 309 ARM](https://github.com/lightningnetwork/lnd/pull/5731). 310 311 * [Fix Travis itest parallelism](https://github.com/lightningnetwork/lnd/pull/5734) 312 313 * [All CI, containers, and automated release artifact building now all use Go 314 1.17.1](https://github.com/lightningnetwork/lnd/pull/5650). All build tags have 315 been updated accordingly to comply with the new Go 1.17.1 requirements. 316 317 * [All integration tests (except the ARM itests) were moved from Travis CI to 318 GitHub Actions](https://github.com/lightningnetwork/lnd/pull/5811). 319 320 * [The LndMobile iOS build has been updated to work 321 with newer gomobile versions](https://github.com/lightningnetwork/lnd/pull/5842) 322 that output in the `xcframework` packaging format. 323 Applications that use the iOS build will have to be updated to include 324 an `xcframework` instead of a `framework`. 325 326 * [Upgrade the sub packages to 1.16](https://github.com/lightningnetwork/lnd/pull/5813) 327 328 * [CI has been upgraded to build against bitcoind 22.0](https://github.com/lightningnetwork/lnd/pull/5928) 329 330 * [Update to the latest neutrino version](https://github.com/lightningnetwork/lnd/pull/5933) 331 332 333 ## Documentation 334 335 * [Outdated warning about unsupported pruning was replaced with clarification that LND **does** 336 support pruning](https://github.com/lightningnetwork/lnd/pull/5553). 337 338 * [Clarified 'ErrReservedValueInvalidated' error string](https://github.com/lightningnetwork/lnd/pull/5577) 339 to explain that the error is triggered by a transaction that would deplete 340 funds already reserved for potential future anchor channel closings (via 341 CPFP) and that more information (e.g., specific sat amounts) can be found 342 in the debug logs. 343 344 * [Updated C# grpc docs to use Grpc.Net.Client](https://github.com/lightningnetwork/lnd/pull/5766). 345 The Grpc.Core NuGet package is in maintenance mode. Grpc.Net.Client is now the 346 [recommended](https://github.com/grpc/grpc-dotnet#grpc-for-net-is-now-the-recommended-implementation) 347 implementation. 348 349 ## Misc 350 351 * The direct use of certain syscalls in packages such as `bbolt` or `lnd`'s own 352 `healthcheck` package made it impossible to import `lnd` code as a library 353 into projects that are compiled to WASM binaries. [That problem was fixed by 354 guarding those syscalls with build tags](https://github.com/lightningnetwork/lnd/pull/5526). 355 356 * The only way to retrieve hophints for a given node was to create an invoice 357 with the `addInvoice` rpc interface. However, now the function has been 358 [exposed in the go package `invoicesrpc`](https://github.com/lightningnetwork/lnd/pull/5697). 359 360 * The `DeleteAllPayments` and `DeletePayment` RPC methods can now be called from 361 the command line with the [new 362 `lncli deletepayments`](https://github.com/lightningnetwork/lnd/pull/5699) 363 command. 364 365 * [Add more verbose error printed to 366 console](https://github.com/lightningnetwork/lnd/pull/5802) when `lnd` fails 367 loading the user specified config. 368 369 * [Make it possible to add more than one RPC Listener when calling lnd.Main](https://github.com/lightningnetwork/lnd/pull/5777). And 370 add MacChan field for passing back lnd's admin macaroon back to the program 371 calling lnd, when needed. 372 373 * [The `--amp-reuse` CLI flag has been removed as the latest flavor of AMP now natively supports static invoices](https://github.com/lightningnetwork/lnd/pull/5991) 374 375 * Using `go get` to install go executables is now deprecated. Migrate to `go install` our lnrpc proto dockerfile [Migrate `go get` to `go install`](https://github.com/lightningnetwork/lnd/pull/5879) 376 377 * [The premature update map has been revamped using an LRU cache](https://github.com/lightningnetwork/lnd/pull/5902) 378 379 ## Code Health 380 381 ### Code cleanup, refactor, typo fixes 382 383 * [Fix logging typo to log channel point on cooperative closes](https://github.com/lightningnetwork/lnd/pull/5881) 384 385 * [Refactor the interaction between the `htlcswitch` and `peer` packages for cleaner separation.](https://github.com/lightningnetwork/lnd/pull/5603) 386 387 * [Moved the original breach handling and timelock UTXO handling into the contract court package](https://github.com/lightningnetwork/lnd/pull/5745) 388 389 * [Unused error check 390 removed](https://github.com/lightningnetwork/lnd/pull/5537). 391 392 * [Shorten Pull Request check list by referring to the CI checks that are 393 in place](https://github.com/lightningnetwork/lnd/pull/5545). 394 395 * [Added minor fixes to contribution guidelines](https://github.com/lightningnetwork/lnd/pull/5503). 396 397 * [Fixed typo in `dest_custom_records` description comment](https://github.com/lightningnetwork/lnd/pull/5541). 398 399 * [Fixed payment test error message.](https://github.com/lightningnetwork/lnd/pull/5559) 400 401 * [Bumped version of `github.com/miekg/dns` library to fix a Dependabot 402 alert](https://github.com/lightningnetwork/lnd/pull/5576). 403 404 * [Fixed timeout flakes in async payment benchmark tests](https://github.com/lightningnetwork/lnd/pull/5579). 405 406 * [State, subscribechannelevents, subscribepeerevents, subscribeinvoices, subscribetransactions, 407 subscribechannelgraph and subscribechannelbackups no longer logs certain errors](https://github.com/lightningnetwork/lnd/pull/5695). 408 409 * [Flake fix in async bidirectional payment test](https://github.com/lightningnetwork/lnd/pull/5607). 410 411 * [Fixed context timeout when closing channels in tests](https://github.com/lightningnetwork/lnd/pull/5616). 412 413 * [Fixed transaction not found in mempool flake in commitment deadline itest](https://github.com/lightningnetwork/lnd/pull/5615). 414 415 * [Fixed a missing import and git tag in the healthcheck package](https://github.com/lightningnetwork/lnd/pull/5582). 416 417 * [Fixed a data race in payment unit test](https://github.com/lightningnetwork/lnd/pull/5573). 418 419 * [Missing dots in cmd interface](https://github.com/lightningnetwork/lnd/pull/5535). 420 421 * [Link channel point logging](https://github.com/lightningnetwork/lnd/pull/5508). 422 423 * [Canceling the chain notifier no longer logs certain errors](https://github.com/lightningnetwork/lnd/pull/5676). 424 425 * [Fixed context leak in integration tests, and properly handled context 426 timeout](https://github.com/lightningnetwork/lnd/pull/5646). 427 428 * [Removed nested db tx](https://github.com/lightningnetwork/lnd/pull/5643). 429 430 * [Fixed wallet recovery itests on Travis ARM](https://github.com/lightningnetwork/lnd/pull/5688). 431 432 * [Integration tests save embedded etcd logs to help debugging flakes](https://github.com/lightningnetwork/lnd/pull/5702). 433 434 * [Replace reference to JWT library with CVE](https://github.com/lightningnetwork/lnd/pull/5737) 435 436 * [Replace reference to XZ library with CVE](https://github.com/lightningnetwork/lnd/pull/5789) 437 438 * [Replace reference to mongo library with CVE](https://github.com/lightningnetwork/lnd/pull/5761) 439 440 * [Fixed restore backup file test flake with bitcoind](https://github.com/lightningnetwork/lnd/pull/5637). 441 442 * [Timing fix in AMP itest](https://github.com/lightningnetwork/lnd/pull/5725). 443 444 * [Upgraded miekg/dns to improve the security posture](https://github.com/lightningnetwork/lnd/pull/5738). 445 446 * [server.go: dedupe pubkey output in debug/log msgs](https://github.com/lightningnetwork/lnd/pull/5722). 447 448 * [Fixed flakes caused by graph topology subcription](https://github.com/lightningnetwork/lnd/pull/5611). 449 450 * [Order of the start/stop on subsystems are changed to promote better safety](https://github.com/lightningnetwork/lnd/pull/1783). 451 452 * [Fixed flake that occurred when testing the new optimistic link shutdown.](https://github.com/lightningnetwork/lnd/pull/5808) 453 454 * [Respect minimum relay fee during commitment fee updates](https://github.com/lightningnetwork/lnd/pull/5483). 455 456 * [Replace reference to protobuf library with OSV](https://github.com/lightningnetwork/lnd/pull/5759) 457 458 * [Only upload itest logs on failure, fix more 459 flakes](https://github.com/lightningnetwork/lnd/pull/5833). 460 461 * [The interfaces for signing messages and the code for initializing a wallet 462 was refactored as a preparation for supporting remote 463 signing](https://github.com/lightningnetwork/lnd/pull/5708). 464 465 * [Include htlc amount in bandwidth hints](https://github.com/lightningnetwork/lnd/pull/5512). 466 467 * [Fix REST/WebSocket API itest that lead to overall test 468 timeout](https://github.com/lightningnetwork/lnd/pull/5845). 469 470 * [Fix missing label on sweep transactions](https://github.com/lightningnetwork/lnd/pull/5895). 471 472 * [Fixed flake that occurred with the switch dust forwarding test under the data race unit build.](https://github.com/lightningnetwork/lnd/pull/5828) 473 474 * [Run channeldb tests on postgres](https://github.com/lightningnetwork/lnd/pull/5550) 475 476 * [Fixed two flakes in itests that were caused by things progressing too 477 fast](https://github.com/lightningnetwork/lnd/pull/5905). 478 479 * [Fixes two issues around configuration parsing and error 480 logging](https://github.com/lightningnetwork/lnd/pull/5948). 481 482 ## Database 483 484 * [Ensure single writer for legacy 485 code](https://github.com/lightningnetwork/lnd/pull/5547) when using etcd 486 backend. 487 * When starting/restarting, `lnd` will [clean forwarding packages, payment 488 circuits and keystones](https://github.com/lightningnetwork/lnd/pull/4364) 489 for closed channels, which will potentially free up disk space for long 490 running nodes that have lots of closed channels. 491 492 * [Optimized payment sequence generation](https://github.com/lightningnetwork/lnd/pull/5514/) 493 to make LNDs payment throughput (and latency) with better when using etcd. 494 495 * [More robust commit queue design](https://github.com/lightningnetwork/lnd/pull/5513) 496 to make it less likely that we retry etcd transactions and make the commit 497 queue more scalable. 498 499 * [Flatten the payment-htlcs-bucket](https://github.com/lightningnetwork/lnd/pull/5635) 500 in order to make it possible to prefetch all htlc attempts of a payment in one 501 DB operation. Migration may fail for extremely large DBs with many payments 502 (10+ million). Be careful and backup your `channel.db` if you have that many 503 payments. Deleting all failed payments beforehand makes migration safer and 504 faster too. 505 506 * [Prefetch payments on hot paths](https://github.com/lightningnetwork/lnd/pull/5640) 507 to reduce roundtrips to the remote DB backend. 508 509 ## Performance improvements 510 511 * [Update MC store in blocks](https://github.com/lightningnetwork/lnd/pull/5515) 512 to make payment throughput better when using etcd. 513 514 * [The `lnwire` package now uses a write buffer pool](https://github.com/lightningnetwork/lnd/pull/4884) 515 when encoding/decoding messages. Such that most of the heap escapes are fixed, 516 resulting in less memory being used when running `lnd`. 517 518 * [`lnd` will now no longer (in a steady state) need to open a new database 519 transaction each time a private key needs to be derived for signing or ECDH 520 operations](https://github.com/lightningnetwork/lnd/pull/5629). This results 521 in a massive performance improvement across several routine operations at the 522 523 * [When decrypting incoming encrypted brontide messages on the wire, we'll now 524 properly re-use the buffer that was allocated for the ciphertext to store the 525 plaintext](https://github.com/lightningnetwork/lnd/pull/5622). When combined 526 with the buffer pool, this ensures that we no longer need to allocate a new 527 buffer each time we decrypt an incoming message, as we 528 recycle these buffers in the peer. 529 530 * [The `DescribeGraph` and `GetNetworkInfo` calls have been 531 optimized](https://github.com/lightningnetwork/lnd/pull/5873) by caching the 532 response periodically, or using the new channel graph cache directly. This 533 should significantly cut down on the garbage these two calls generate. 534 535 ## Log system 536 537 * [Save compressed log files from logrorate during 538 itest](https://github.com/lightningnetwork/lnd/pull/5354). 539 540 ## Mission control 541 542 * [Interpretation of channel disabled errors was changed to only penalize t 543 he destination node to consider mobile wallets with hub 544 nodes.](https://github.com/lightningnetwork/lnd/pull/5598) 545 546 ## Bug Fixes 547 548 * A bug has been fixed that would cause `lnd` to [try to bootstrap using the 549 current DNS seeds when in SigNet 550 mode](https://github.com/lightningnetwork/lnd/pull/5564). 551 552 * [A validation check for sane `CltvLimit` and `FinalCltvDelta` has been added 553 for `REST`-initiated 554 payments](https://github.com/lightningnetwork/lnd/pull/5591). 555 556 * [A bug has been fixed with Neutrino's `RegisterConfirmationsNtfn` and 557 `RegisterSpendNtfn` calls that would cause notifications to be 558 missed](https://github.com/lightningnetwork/lnd/pull/5453). 559 560 * [A bug has been fixed when registering for spend notifications in the 561 `txnotifier`. A re-org notification would previously not be dispatched in 562 certain scenarios](https://github.com/lightningnetwork/lnd/pull/5465). 563 564 * [Catches up on blocks in the 565 router](https://github.com/lightningnetwork/lnd/pull/5315) in order to fix an 566 "out of order" error that [crops up](https://github.com/lightningnetwork/lnd/pull/5748). 567 568 * [Fix healthcheck might be running after the max number of attempts are 569 reached](https://github.com/lightningnetwork/lnd/pull/5686). 570 571 * [Fix crash with empty AMP or MPP record in 572 invoice](https://github.com/lightningnetwork/lnd/pull/5743). 573 574 * [Config setting sync-freelist was ignored in certain 575 cases](https://github.com/lightningnetwork/lnd/pull/5527). 576 577 * The underlying gRPC connection of a WebSocket is now [properly closed when the 578 WebSocket end of a connection is 579 closed](https://github.com/lightningnetwork/lnd/pull/5683). A bug with the 580 write deadline that caused connections to suddenly break was also fixed in the 581 same PR. 582 583 * [A bug has been fixed in 584 Neutrino](https://github.com/lightninglabs/neutrino/pull/226) that would 585 result in transactions being rebroadcast even after they had been confirmed. 586 [Lnd is updated to use the version of Neutrino containing this 587 fix](https://github.com/lightningnetwork/lnd/pull/5807). 588 589 * A bug has been fixed that would result in nodes not [reconnecting to their 590 persistent outbound peers if the peer's IP 591 address changed](https://github.com/lightningnetwork/lnd/pull/5538). 592 593 * [Use the change output index when validating the reserved wallet balance for 594 SendCoins/SendMany calls](https://github.com/lightningnetwork/lnd/pull/5665) 595 596 * A [bug](https://github.com/lightningnetwork/lnd/pull/5834) has been fixed where 597 certain channels couldn't be passed to `lncli getchaninfo` due to their 8-byte 598 compact ID being too large for an int64. 599 600 * [Dedup stored peer addresses before creating connection requests to prevent 601 redundant connection requests](https://github.com/lightningnetwork/lnd/pull/5839) 602 603 * A [`concurrent map writes` crash was 604 fixed](https://github.com/lightningnetwork/lnd/pull/5893) in the 605 [`btcwallet` dependency](https://github.com/btcsuite/btcwallet/pull/773). 606 607 * [A bug has been fixed that would at times cause intercepted HTLCs to be 608 re-notified](https://github.com/lightningnetwork/lnd/pull/5901), which could 609 lead to higher-level HTLC mismanagement issues. 610 611 * [Do not error log when an invoice that has been canceled and GC'd is expired]( 612 https://github.com/lightningnetwork/lnd/pull/5913) 613 614 * [Don't print bucket names with special characters when compacting]( 615 https://github.com/lightningnetwork/lnd/pull/5878) 616 617 * [Fix pathfinding crash when inbound policy is unknown]( 618 https://github.com/lightningnetwork/lnd/pull/5922) 619 620 * [Stagger connection attempts to multi-address peers to ensure that the peer 621 doesn't close the first successful connection in favour of the next if 622 the first one was successful]( 623 https://github.com/lightningnetwork/lnd/pull/5925) 624 625 * [Fixed an issue with external listeners and the `--noseedbackup` development 626 flag](https://github.com/lightningnetwork/lnd/pull/5930). 627 628 * [Fixed a bug that caused the RPC middleware request ID not to be the same 629 for intercept messages belonging to the same intercepted gRPC request/response 630 pair](https://github.com/lightningnetwork/lnd/pull/5950). 631 632 * [Fix deadlock when using the graph cache]( 633 https://github.com/lightningnetwork/lnd/pull/5941) 634 635 * [Fixes a bug that would cause pruned nodes to stall out](https://github.com/lightningnetwork/lnd/pull/5970) 636 637 * [Add Postgres connection limit](https://github.com/lightningnetwork/lnd/pull/5992) 638 639 ## Documentation 640 641 The [code contribution guidelines have been updated to mention the new 642 requirements surrounding updating the release notes for each new 643 change](https://github.com/lightningnetwork/lnd/pull/5613). 644 645 # Contributors (Alphabetical Order) 646 * Abubakar Nur Khalil 647 * Adrian-Stefan Mares 648 * Alex Bosworth 649 * Alyssa Hertig 650 * András Bánki-Horváth 651 * Bjarne Magnussen 652 * Carla Kirk-Cohen 653 * Carsten Otto 654 * Conner Fromknecht 655 * Elle Mouton 656 * ErikEk 657 * Eugene Siegel 658 * Hampus Sjöberg 659 * Harsha Goli 660 * Jesse de Wit 661 * Johan T. Halseth 662 * Johnny Holton 663 * Joost Jager 664 * Jordi Montes 665 * Juan Pablo Civile 666 * Kishin Kato 667 * Leonhard Weese 668 * Martin Habovštiak 669 * Michael Rhee 670 * Naveen Srinivasan 671 * Olaoluwa Osuntokun 672 * Oliver Gugger 673 * Priyansh Rastogi 674 * Roei Erez 675 * Simon Males 676 * Stevie Zollo 677 * Torkel Rogstad 678 * Wilmer Paulino 679 * Yong Yu 680 * Zero-1729 681 * benthecarman 682 * de6df1re 683 * github2k20 684 * mateuszmp 685 * nathanael 686 * offerm 687 * positiveblue 688 * xanoni