7cbe78eeee
In this commit, we might a very small change to the way writing messages
works in the peer, which should have large implications w.r.t reducing
memory usage amongst chatty nodes.
When profiling the heap on one of my nodes earlier, I noticed this
fragment:
```
Showing top 20 nodes out of 68
flat flat% sum% cum cum%
0 0% 0% 75.53MB 54.61% main.(*peer).writeHandler
75.53MB 54.61% 54.61% 75.53MB 54.61% main.(*peer).writeMessage
```
Which points to an inefficiency with the way we handle allocations when
writing new messages, drilling down further we see:
```
(pprof) list writeMessage
Total: 138.31MB
ROUTINE ======================== main.(*peer).writeMessage in /root/go/src/github.com/lightningnetwork/lnd/peer.go
75.53MB 75.53MB (flat, cum) 54.61% of Total
. . 1104: p.logWireMessage(msg, false)
. . 1105:
. . 1106: // As the Lightning wire protocol is fully message oriented, we only
. . 1107: // allows one wire message per outer encapsulated crypto message. So
. . 1108: // we'll create a temporary buffer to write the message directly to.
75.53MB 75.53MB 1109: var msgPayload [lnwire.MaxMessagePayload]byte
. . 1110: b := bytes.NewBuffer(msgPayload[0:0:len(msgPayload)])
. . 1111:
. . 1112: // With the temp buffer created and sliced properly (length zero, full
. . 1113: // capacity), we'll now encode the message directly into this buffer.
. . 1114: n, err := lnwire.WriteMessage(b, msg, 0)
(pprof) list writeHandler
Total: 138.31MB
ROUTINE ======================== main.(*peer).writeHandler in /root/go/src/github.com/lightningnetwork/lnd/peer.go
0 75.53MB (flat, cum) 54.61% of Total
. . 1148:
. . 1149: // Write out the message to the socket, closing the
. . 1150: // 'sentChan' if it's non-nil, The 'sentChan' allows
. . 1151: // callers to optionally synchronize sends with the
. . 1152: // writeHandler.
. 75.53MB 1153: err := p.writeMessage(outMsg.msg)
. . 1154: if outMsg.errChan != nil {
. . 1155: outMsg.errChan <- err
. . 1156: }
. . 1157:
. . 1158: if err != nil {
```
Ah hah! We create a _new_ buffer each time we want to write a message
out. This is unnecessary and _very_ wasteful (as seen by the profile).
The fix is simple: re-use a buffer unique to each peer when writing out
messages. Since we know what the max message size is, we just allocate
one of these 65KB buffers for each peer, and keep it around until the
peer is removed.
|
||
|---|---|---|
| .github | ||
| aezeed | ||
| autopilot | ||
| brontide | ||
| chainntnfs | ||
| channeldb | ||
| cmd/lncli | ||
| contractcourt | ||
| contrib | ||
| discovery | ||
| docker | ||
| docs | ||
| htlcswitch | ||
| keychain | ||
| lnrpc | ||
| lntest | ||
| lnwallet | ||
| lnwire | ||
| macaroons | ||
| make | ||
| multimutex | ||
| routing | ||
| shachain | ||
| torsvc | ||
| walletunlocker | ||
| zpay32 | ||
| .gitignore | ||
| .travis.yml | ||
| breacharbiter_test.go | ||
| breacharbiter.go | ||
| chainparams.go | ||
| chainregistry.go | ||
| chancloser.go | ||
| config.go | ||
| doc.go | ||
| fundingmanager_test.go | ||
| fundingmanager.go | ||
| Gopkg.lock | ||
| Gopkg.toml | ||
| invoiceregistry.go | ||
| LICENSE | ||
| lnd_test.go | ||
| lnd.go | ||
| log.go | ||
| logo.png | ||
| Makefile | ||
| mock.go | ||
| nodesigner.go | ||
| nursery_store_test.go | ||
| nursery_store.go | ||
| peer_test.go | ||
| peer.go | ||
| pilot.go | ||
| README.md | ||
| release.sh | ||
| rpcserver.go | ||
| sample-lnd.conf | ||
| server_test.go | ||
| server.go | ||
| signal.go | ||
| test_utils.go | ||
| utxonursery_test.go | ||
| utxonursery.go | ||
| version.go | ||
| witness_beacon.go | ||
Lightning Network Daemon
The Lightning Network Daemon (lnd) - is a complete implementation of a
Lightning Network node and currently deployed on
testnet3 - the Bitcoin Test Network. lnd has several pluggable back-end
chain services including btcd (a
full-node), bitcoind, and
neutrino (a new experimental light client). The project's codebase uses the
btcsuite set of Bitcoin libraries, and also
exports a large set of isolated re-usable Lightning Network related libraries
within it. In the current state lnd is capable of:
- Creating channels.
- Closing channels.
- Completely managing all channel states (including the exceptional ones!).
- Maintaining a fully authenticated+validated channel graph.
- Performing path finding within the network, passively forwarding incoming payments.
- Sending outgoing onion-encrypted payments through the network.
- Updating advertised fee schedules.
- Automatic channel management (
autopilot).
Lightning Network Specification Compliance
lnd fully conforms to the Lightning Network specification
(BOLTs). BOLT stands for:
Basic of Lightning Technologies. The specifications are currently being drafted
by several groups of implementers based around the world including the
developers of lnd. The set of specification documents as well as our
implementation of the specification are still a work-in-progress. With that
said, the current status of lnd's BOLT compliance is:
- BOLT 1: Base Protocol
- BOLT 2: Peer Protocol for Channel Management
- BOLT 3: Bitcoin Transaction and Script Formats
- BOLT 4: Onion Routing Protocol
- BOLT 5: Recommendations for On-chain Transaction Handling
- BOLT 7: P2P Node and Channel Discovery
- BOLT 8: Encrypted and Authenticated Transport
- BOLT 9: Assigned Feature Flags
- BOLT 10: DNS Bootstrap and Assisted Node Location
- BOLT 11: Invoice Protocol for Lightning Payments
Developer Resources
The daemon has been designed to be as developer friendly as possible in order
to facilitate application development on top of lnd. Two primary RPC
interfaces are exported: an HTTP REST API, and a gRPC
service. The exported API's are not yet stable, so be warned: they may change
drastically in the near future.
An automatically generated set of documentation for the RPC APIs can be found at api.lightning.community. A set of developer resources including talks, articles, and example applications can be found at: dev.lightning.community.
Finally, we also have an active
Slack
where protocol developers, application developers, testers and users gather to
discuss various aspects of lnd and also Lightning in general.
Installation
In order to build from source, please see the installation instructions.
IRC
- irc.freenode.net
- channel #lnd
- webchat