Commit Graph

2529 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
f4e7c36c80
peer: eliminate excessive queueHandler spinning w/ blocked writeHandler
This commit fixes an existing bug within the iteration between the
queueHandler and the writeHandler. Under certain scenarios, if the
writeHandler was blocked for a non negligible period of time, then the
queueHandler would enter a very tight spinning loop. This was due to
the fact that the break statement in the inner select loop of the
queueHandler wouldn’t actually break the inner for loop, instead it
would cause the execution logic to re-enter that same select loop,
causing a very tight spin.

In this commit, we fix the issue by adding to things: we now label the
inner select loop so we can break out of it if we detect that the
writeHandler has blocked. Secondly, we introduce a new channel between
the queueHandler and the writeHandler to signal the queueHandler that
the writeHandler has finished processing the last message.
2017-10-15 15:19:57 -07:00
Olaoluwa Osuntokun
cbdf139696
peer: add an idle timer to the readHandler
In this commit, we add an idle timer to the readHandler itself. This
will serve to slowly prune away inactive TCP connections as a result of
remote peer being blocked either upon reading or writing to the socket.
Our ping timer interval is 1 minute, so an idle timer interval of 5
minutes seem reasonable.
2017-10-15 15:14:35 -07:00
Olaoluwa Osuntokun
7a2ce62346
config+lnd: add new option --cpuprofile for writing cpuprofile to file
In this commit we add a new option to lnd, cpuprofile. With this option
we add additional telemetry hooks into and, allowing users to generate
CPU profiling files to measure hot spots within the daemon.
2017-10-14 17:08:34 -07:00
Olaoluwa Osuntokun
b7704e2de3
server: make sendToPeer async
This commit removes another case of unnecessary blockage, by modifying
the sendToPeer method to be fully asynchronous. From the PoV of the
callers that utilize this method currently, there’s no reason to block
until the completion of this method. Additionally, as the graph grows
larger without more intelligent the number of messages sent during
initial dump will start to be prohibitive to waiting for full
completion before proceeding.
2017-10-14 17:07:08 -07:00
Olaoluwa Osuntokun
6db90ef09a
server: BroadcastMessage will no longer block until completion
In this commit, we make the BroadcastMessage method on the server more
asynchronous by abandoning the two wait groups that it used for
synchronization. It has been observed that a circular waiting loop
between the AuthenticatedGossiper and a peer’s readHandler can cause
the system to dead lock.

By removing this unnecessary synchronization, we avoid the deadlock
case and allow the gossiper itself to no longer block in this scenario.
2017-10-14 16:04:06 -07:00
Conner Fromknecht
7aa64d58da server: improves readibility of peer connection logic 2017-10-13 16:12:31 -07:00
Conner Fromknecht
96ff63d219 server: fixes race condition during unexpected peer disconnect 2017-10-13 16:12:31 -07:00
Olaoluwa Osuntokun
f20cb89982
build: update lock file for glide 2017-10-10 22:37:17 -07:00
Olaoluwa Osuntokun
3deb94442f
build: update glide to point to latest sphinx impl 2017-10-10 22:30:50 -07:00
Olaoluwa Osuntokun
458e576af7
build: update glide to latest neutrino w/ re-org fixes 2017-10-10 22:19:31 -07:00
Olaoluwa Osuntokun
2eaa9fd683
htlcswitch: fix linter error 2017-10-10 22:19:29 -07:00
Olaoluwa Osuntokun
51b072c4b5
routing: return proper error if encounter non ForwardingError in SendPayment 2017-10-10 22:19:28 -07:00
Olaoluwa Osuntokun
03d8399d0a
build: update glide files to point to latest neutrino+btcwallet 2017-10-10 22:19:28 -07:00
Olaoluwa Osuntokun
646f79f566
routing: perform path finding inside a single DB transaction
This commit modifies the path finding logic such that all path finding
is done inside a _single_ database transaction. With this change, we
ensure that we don’t end up possibly creating hundreds of database
transactions slowing down the path finding and payment sending process
all together.
2017-10-10 22:19:27 -07:00
Olaoluwa Osuntokun
5eb3406b19
channeldb: add new Database method to ChannelGraph 2017-10-10 22:19:26 -07:00
Olaoluwa Osuntokun
eb7b5b342e
routing: add a basic test to exercise route pruning in response to errors 2017-10-10 22:19:25 -07:00
Olaoluwa Osuntokun
ce7179a468
routing: add basic route pruning in response to HTLC onion errors
This commit adds basic route pruning in response to HTLC onion errors.
With this new change, the router will now prune routes in response to
HTLC errors, which will reduce the time to payment success, and also
avoid a bunch of unnecessary network traffic.

We now respond to two errors lnwire.FailTemporaryChannelFailure and
lnwire.FailUnknownNextPeer. In response to the first error, we’ll prune
all routes that contain the channel which was unable to be routed over.
In response to the second error we’ll prune all routes that contain the
node which couldn’t be found.
2017-10-10 22:19:25 -07:00
Olaoluwa Osuntokun
8d7f3943bb
routing: add two new methods to filter routes based on node/channel 2017-10-10 22:19:24 -07:00
Olaoluwa Osuntokun
f6ac31281b
routing: also include the source node in the nextHopMap index
In this commit we modify the newRoute function to also add the source
node to the nextHopMap index. With this addition the indexes will now
allow the router to react based on failures that occur during the
_first_ hop, meaning the channel directly attached to the source node.
2017-10-10 22:19:23 -07:00
Olaoluwa Osuntokun
70e114fa6f
routing: add additional indexes to the Route struct to allow for querying
This commit adds three new indexes to the Route struct. These indexes
allow a caller to check if a channel is in the route, check if a node
is in the route, query the next node after a target node, and query the
next channel after a target node. The combination of these new indexes
will allow the ChannelRouter to prune away routes from the available
set in response to any received errors.
2017-10-10 22:19:22 -07:00
Olaoluwa Osuntokun
7ba6b7fa09
routing: add a String() method to the vertex type 2017-10-10 22:19:22 -07:00
Olaoluwa Osuntokun
63838f5764
peer+server: use new sphinx package and htlcswitch package type names 2017-10-10 22:19:21 -07:00
Olaoluwa Osuntokun
a7d846ac40
lnwire: add godoc comments for lnwire.FailureMessage 2017-10-10 22:19:20 -07:00
Olaoluwa Osuntokun
93c008bb46
htlcswitch: update tests to expect new ForwardingError type 2017-10-10 22:19:19 -07:00
Olaoluwa Osuntokun
12ae63101d
Htlcswitch: rename Deobfuscator and Obfuscator interfaces
This commit renames the Deobfuscator interface to ErrorDecrypter and
the Obfuscator interface to ErrorEncrypter. With this rename, the
purpose of these two interfaces are a bit clearer.

Additionally, DecryptError (which was formerly Deobfuscate) now
directly returns an ForwardingError type instead of the
lnwire.FailureMessage.
2017-10-10 22:19:19 -07:00
Olaoluwa Osuntokun
30a46291f8
htlcswitch: introduce new ForwardingError type
This commit introduces a new type to the package: ForwardingError. It
wraps an existing lnwire.FailureMessage interface, and also includes
the _source_ of the error message. By including the source of the
message, the router can now prune the set of available routes down in
order to reduce the number of subsequent failures based on the source
of the error and the type of the error itself.
2017-10-10 22:19:18 -07:00
Olaoluwa Osuntokun
40dfa5c8fc
glide: update lightning-onion to latest commit hash 2017-10-10 22:19:17 -07:00
Olaoluwa Osuntokun
e5c303375b
server: in peerBootstrapper check for enough conns before attempting backoff
In this commit we modify the main loop within the peerBootstrapper
slightly to check for a sufficient amount of connections, _before_
checking to see if we need to back off the main loop. With this, we
avoid unnecessarily backing off unless an actual error occurs.
2017-10-10 22:19:13 -07:00
Olaoluwa Osuntokun
4aeaf5aaa9
docs/go-fuzz: rename wirefuzz.md to README.md 2017-10-05 18:30:26 -07:00
nsa
1e62865cf9 docs: fuzzing with go-fuzz tutorial
This commit adds a tutorial on fuzzing with the go-fuzz library
into the docs folder. It includes an introduction to fuzzing,
setup and installation steps to run go-fuzz with lnd, tips to
generate a valid corpus for use with go-fuzz, and finally it
includes a small explanation of the test harness that was used
to find bugs in lnd.
2017-10-05 18:28:59 -07:00
Olaoluwa Osuntokun
3368b238c4 Merge pull request #361 from cfromknecht/baby-outputs
Adds second layer HTLC output types to UtxoNursery
2017-10-05 18:27:05 -07:00
Conner Fromknecht
d8688b7d67
utxonursery: adds compile-time iface check for baby and kid outputs 2017-10-05 18:13:48 -07:00
Jim Posen
7807150e73 lnwallet: Implement BOLT-03 key derivation test vectors. 2017-10-05 16:54:38 -07:00
Jim Posen
ee86901acb shachain: Add two missing test cases from the BOLT spec. 2017-10-05 16:54:38 -07:00
Olaoluwa Osuntokun
490e0d0f46
discovery: broadcast OG chan ann in retransmitStaleChannels 2017-10-04 20:46:12 -07:00
Olaoluwa Osuntokun
0d28bb14cb
discovery: also return latest ChannelAnnouncement from updateChannel
This commit adds an additional return value to the updateChannel
method. We also now return the original ChannelAnnouncement, this can
be useful as it let’s the caller ensure that the channel announcement
will be broadcast along side the new channel update.
2017-10-04 20:46:10 -07:00
Olaoluwa Osuntokun
4be2e95a78
discovery: fix boltdb deadlock bug in ann retrans, retrans at start
This commit does two things. First we fix a deadlock bug within boltdb
that could arise when the channel router attempted to open a
transaction, while the retransmission loop was attempting to send a
message to the channel router (circular wait). Second, we’ve refactored
out the retransmission into its own function. This allows us to kick
off retransmission as soon as the AuthenticatedGossiper is created.
2017-10-04 20:46:10 -07:00
Olaoluwa Osuntokun
3b7855e449
routing: implement 2-week zombie channel pruning
This commit implements 2-week zombie channel pruning. This means that
every GraphPruneInterval (currently set to one hour), we’ll scan the
channel graph, marking any channels which haven’t had *both* edges
updated in 2 weeks as a “zombie”. During the second pass, all “zombie”
channel are removed from the channel graph all together.

Adding this functionality means we’ll ensure that we maintain a
“healthy” network view, which will cut down on the number of failed
HTLC routing attempts, and also reflect an active portion of the graph.
2017-10-04 20:46:09 -07:00
Olaoluwa Osuntokun
e81689057a
discovery+routing: remove DeleteEdge from ChannelGraphSource interface
This commit removes the recently added DeleteEdge method from the
ChannelGraphSource interface as it’s no longer needed.
2017-10-04 20:46:08 -07:00
Olaoluwa Osuntokun
81cd954cfc
discovery: don't prune *our* channels during retransmission tick
This commit modifies the recently modified logic for self-channel
retransmission to exclude pruning *our* channels which haven’t been
updated since the broadcastInterval. Instead, we only re-broadcast
channels of ours that haven’t been updated in 24 hours.
2017-10-04 20:46:08 -07:00
Olaoluwa Osuntokun
ebd2dfbfd9
htlcswitch: use Tracef rather than Debugf for log examining HTLC's 2017-10-04 20:46:04 -07:00
Brandon
cce75bbb14 discovery: add retransmit delay to gosipper_test 2017-10-04 20:42:27 -07:00
Brandon
e1ea0aafe8 discovery: adding DeleteEdge to gossiper_test 2017-10-04 20:42:17 -07:00
Brandon
3907ae65c2 routing+discovery: implement 2-week network view pruning 2017-10-04 20:40:21 -07:00
Olaoluwa Osuntokun
32508344db
server: use Debugf instead of Debug for logging in peerBootstrapper 2017-10-04 16:58:57 -07:00
Olaoluwa Osuntokun
c64811a5f3
brontide: set read deadlines on socket during initial handshake
This commit fixes a lingering issue within lnd, which can cause a
server to freeze up, and not handle any incoming connections properly,
or cause clients to freeze and not return in a timely manner from a
failed connection attempt.

To avoid this, each time we need to read from the socket during the
initial brontide handshake, we add a 15 second read deadline. If we
don’t successfully read from the buffer during that time frame, then
the Read method will return a timeout error.

With this in place, we ensure that the main listener goroutine will
never be blocked waiting on a remote party to write ActOne.
2017-10-04 14:58:20 -07:00
Olaoluwa Osuntokun
c8226b1393
server: fix logging statement in peerBootstrapper
This commit fixes an incorrect logging statement within the
peerBootstrapper goroutine. We we’re using a Debug method previously
when we should’ve been using Debugf in order to properly pass the
logging statement through.
2017-10-04 14:52:54 -07:00
Conner Fromknecht
0dcb620194
utxonursery_test: properly print error msg during init failure 2017-10-03 19:40:09 -07:00
Conner Fromknecht
d2e5494ef3
utxonursery: improves documentation grammar and thoroughness 2017-10-03 19:40:09 -07:00
Conner Fromknecht
921f0b1f43
utxonursery_test: adds serialization tests for baby outputs 2017-10-03 19:40:09 -07:00