Commit Graph

97 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
0a4de859a2
discovery+routing: reduce number of active validation barrier jobs
In order to reduce high CPU utilization during the initial network view
sync, we slash down the total number of active in-flight jobs that can
be launched.
2018-01-28 14:55:32 -08:00
Johan T. Halseth
b07f242dc2
routing: use multimutex.Mutex instead of package internal mutex 2018-01-24 10:26:40 +01:00
Olaoluwa Osuntokun
eba6ba0760
routing: properly prune edges that return a FailPermanentChannelFailure
In this commit, we now account for a case where a node sends us a
FailPermanentChannelFailure during a payment attempt. Before this
commit, we wouldn’t properly prune the edge to avoid re-using it. We
remedy this by properly attempting to prune the edge if possible.
Future changes well send a FailPermanentChannelFailure in the case that
we ned to go on-chain for an outgoing HTLC, and cancel back the
incoming HTLC.
2018-01-22 19:19:34 -08:00
Johan T. Halseth
0b8e7ff836
routing: embed FeeSchema in ChannelPolicy
This commit embeds the FeeSchema within the new
struct ChannelPolicy, which also contains the
TimeLockDelta for a channel.
2018-01-12 22:56:31 +01:00
Olaoluwa Osuntokun
d70e4bb0a0
routing: account for case where final destination send TemporaryChannelFailure
In this commit, we fix an existing bug that could cause lnd to crash if
we sent a payment, and the *destination* sent a temp channel failure
error message. When handling such a message, we’ll look in the nextHop
map to see which channel was *after* the node that sent the payment.
However, if the destination sends this error, then there’ll be no entry
in this map.

To address this case, we now add a prevHop map. If we attempt to lookup
a node in the nextHop map, and they don’t have an entry, then we’ll
consult the prevHop map.

We also update the set of tests to ensure that we’re properly setting
both the prevHop map and the nextHop map.
2018-01-10 15:15:55 -08:00
Johan T. Halseth
0571918d3d
routing: process only single ChannelEdgePolicy per channelID concurrently
This commit adds synchronization around the processing
of multiple ChannelEdgePolicy updates for the same
channel ID at the same time.

This fixes a bug that could cause the database access
HasChannelEdge to be out of date when the goroutine
came to the point where it was calling UpdateEdgePolicy.
This happened because a second goroutine would have
called UpdateEdgePolicy in the meantime.

This bug was quite benign, as if this happened at
runtime, we would eventually get the ChannelEdgePolicy
we had lost again, either from a peer sending it to
us, or if we would fail a payment since we were using
outdated information. However, it would cause some of
the tests to flake, since losing routing information
made payments we expected to go through fail if this
happened.

This is fixed by introducing a new mutex type, that
when locking and unlocking takes an additional
(id uint64) parameter, keeping an internal map
tracking what ID's are currently locked and the
count of goroutines waiting for the mutex. This
ensure we can still process updates concurrently,
only avoiding updates with the same channel ID from
being run concurrently.
2018-01-09 13:09:29 +01:00
Olaoluwa Osuntokun
98f63cdce1
routing: add new paymentSession companion struct to missionControl
In this commit, we modify the pruning semantics of the missionControl
struct. Before this commit, on each payment attempt, we would fetch a
new graph pruned view each time. This served to instantly propagate any
detected failures to all outstanding payment attempts. However, this
meant that we could at times get stuck in a retry loop if sends take a
few second, then we may prune an edge, try another, then the original
edge is now unpruned.

To remedy this, we now introduce the concept of a paymentSession. The
session will start out as a snapshot of the latest graph prune view.
Any payment failures are now reported directly to the paymentSession
rather than missionControl. The rationale for this is that
edges/vertexes pruned as result of failures will never decay for a
local payment session, only for the global prune view. With this in
place, we ensure that our set of prune view only grows for a session.

Fixes #536.
2018-01-08 19:50:24 -08:00
Matt Drollette
adf0d98194 multi: fix several typos in godoc comments 2017-12-17 18:40:05 -08:00
Olaoluwa Osuntokun
455568279b
routing: ensure that new blocks connected monotonically increase in height
This commit fixes an existing bug within the ChannelRouter. Prior to
this commit, if the chain view skipped blocks or for some reason we had
a gap in blocks delivered, then we would simply accept them. This had
the potential to cause us to miss on-chain channel closure events. To
remedy this, we won’t process any blocks whose heights aren’t
*strictly* increasing.

A longer term fix would be to have the ChainView take a block height,
and re-dispatch any notifications from that height to the current
height.
2017-12-05 17:46:21 -08:00
Olaoluwa Osuntokun
ffd6b65ec1
routing: ensure access to r.bestHeight is thread-safe 2017-12-05 17:46:20 -08:00
Olaoluwa Osuntokun
73ccb76853
routing: update ChannelUpdate handling to recognize lnwire.ChanUpdateFlag 2017-11-30 22:24:35 -08:00
Olaoluwa Osuntokun
7757721a92
routing: don't dispatch notification if error is non nil after processing ann 2017-11-29 17:53:09 -08:00
Olaoluwa Osuntokun
5bc9f07d12
routing: validate incoming announcements in parallel 2017-11-29 16:44:14 -08:00
Olaoluwa Osuntokun
fcd5e4aa41
routing: when updating chain view filter on restart, use best height as height hint
In this commit, we modify the high value passed into UpdateFilter upon
restart. Before this commit, we would pass in the prune height, which
would cause a full rescan within the FilteredChainView if the best
height as > than the prune height. This was redundant as we would
shortly carry out a manual rescan in the method below. To fix this, we
now pass in the bestHeight, this isn’t an issue as the
syncGraphWithChain method will manually scan up to that best height.
2017-11-29 16:26:39 -08:00
Olaoluwa Osuntokun
f189e2395a
routing: if the graph has never been pruned, prune with current height
In this commit we ensure that if this is the first time that the
ChannelRouter is starting, then we set the pruned height+hash to the
current best height. Otherwise, it’s possible that we attempt to update
the filter with a 0 prune height, which will restart a historical
rescan unnecessarily.
2017-11-15 18:19:39 -08:00
Olaoluwa Osuntokun
2b052cc889
routing: only update the filter if we have a non-zero chain view
In this commit we ensure that we only update the filter, if we have a
non-zero chain view. Otherwise, a mini rescan may be kicked off
unnecessarily  if we don’t yet know of any channels yet in the greater
graph.
2017-11-15 18:17:51 -08:00
Laura Cressman
39d38da732 discovery: create deDupedAnnouncements struct in gosspier.go
For Part 1 of Issue #275. Create isolated private struct in
networkHandler goroutine that will de-duplicate
announcements added to the batch. The struct contains maps
for each of channel announcements, channel updates, and
node announcements to keep track of unique announcements.

The struct has a Reset method to reset stored announcements, an
AddMsg(lnwire.Message) method to add a new message to the current
batch, and a Batch method to return the set of de-duplicated
announcements.

Also fix a few minor typos.
2017-11-15 16:50:19 -08:00
Conner Fromknecht
153794adb0
routing/router: sets bestHeight before filtering chain
This commit alters the behavior of the router's logic on
 startup, ensuring that the chain view is filtered using
 the router's latest prune height. Before, the chain was
 filtered using the bestHeight variable, which was
 uninitialized, benignly forcing a rescan from genesis.
 In tracking down this, we realized that we should
 actually be using the prune height, as this is
 representative of the channel view loaded from disk.
 The best height/hash are now only used during
 startup to determine if we are out of sync.
2017-11-11 15:53:38 -08:00
Olaoluwa Osuntokun
9294358b5b
routing: populate the chain filter before calling syncGraphWithChain
In this commit we fix an existing bug within the ChannelRouter. Before
this commit, we would sync our graph prune state, *then* update the
cain filter. This is incorrect as the blocks we manually pruned may
have included channel closing transactions. As a result, we would miss
the pruning of a set of channels, and assume that they were still
active.

In this commit, we fix this by reversing the order: we first update the
chain filter and THEN sync the channel graph.
2017-11-06 16:10:24 -08:00
Johan T. Halseth
256db86b02
routing: make channel router aware of stale blocks
This commit make the channel router handle the case where
it wakes up on a stale branch, and also the case where a
reorg happens while it is active.
2017-11-03 00:05:32 +01:00
Olaoluwa Osuntokun
176fde4ec0
routing: modify applyChannelUpdate to not error out if update is stale
In this commit we fix a slight bug within the existing SendPayment loop
which would cause the wrong error to be returned to users. Prior to
this commit, if we received an update identical to what we were already
aware of, then that error would be returned rather than the
ForwardingError that encapsulated this update.

In this commit with remedy this by properly returning the exact error.

Partially fixes #391.
2017-10-24 18:23:04 -07:00
Olaoluwa Osuntokun
25614f67f8
routing: restore in memory selfNode within the ChannelRouter
In this commit we restore the in memory ChannelRouter as we’ll no
dynamically set the ChannelRouter’s pointer within he spec path finding
test example.
2017-10-22 18:36:59 -07:00
Olaoluwa Osuntokun
76ec41103e
routing: extend missionControl.RequestRoute to take final CLTV delta 2017-10-22 18:36:58 -07:00
Olaoluwa Osuntokun
aee1619488
routing: add new FinalCLTVDelta attribute to SendPayment
In this commit, we’ll now optionally allow the user to pass in the CLTV
delta value specified by the recipient a payment. If the value isn’t
specified, then we’ll use the current global default for the payment.
2017-10-22 18:36:51 -07:00
Olaoluwa Osuntokun
c8f45e3a04
routing: add new param to FindRoutes for CLTV expiry for final hop
In this commit, we modify the FindRoutes method to pass in the CLTV
expiry for the final hop. If the value isn’t passed in, then we’ll use
the current global default value in place.
2017-10-22 18:36:50 -07:00
Olaoluwa Osuntokun
65482faa79
routing: remove the selfNode attribute from memory
In this commit, we’ve removed the selfNode attribute from memory, as
the set of new tests we’ll write, will depend on us being able to
switch the source node dynamically from the database itself.
2017-10-22 18:36:48 -07:00
Olaoluwa Osuntokun
b4273d1eaa
routing: abstract path finding behind mission control
In this commit, from the PoV of the SendPayment method we now delegate
all path finding+verification to missionControl. This change doesn’t
materially affect anything, it simply expands the abstraction to make
way for future features that more heavily utilize mission control.
2017-10-17 19:44:17 -07:00
Olaoluwa Osuntokun
8ef829ed80
routing: modify SendPayment loop to be lazy, iterative, and use missionControl
In this commit we modify the SendPayment loop to optimize for
time-to-first-payment-success-or-failure. The prior logic would first
attempt to find at least 100 routes to the destination, then
iteratively prune them away as errors were encountered. In this commit,
we modify this approach to instead take a lazy approach: we first find
the current “best” path, attempt to send to that, and if an error
occurs we prune a section of the graph by reporting to missionControl,
then continue.

With this new approach, if the first known path has sufficient
capacity, and is available, then the payment speed is greatly improved
from the PoV of users. Additionally, we avoid the excessive computation
of crawling most of the graph in the k-shortest paths loop. With the
decay on missionControl, all routes will now feed information into the
central knowledge hung, allowing all payments to iteratively find out
the inactive portions of the payment graph.
2017-10-16 19:05:47 -07:00
Olaoluwa Osuntokun
276f2e467b
routing: end path finding on an additional set of critical-ish errors 2017-10-16 18:58:35 -07:00
Olaoluwa Osuntokun
b29a73a0dd
routing: don't prune our own channels during zombie channel collection
This commit is a precautionary commit which ensure that we don’t
attempt to prune our _own_ channels during zombie channel collection.
2017-10-16 18:14:01 -07:00
Olaoluwa Osuntokun
51b072c4b5
routing: return proper error if encounter non ForwardingError in SendPayment 2017-10-10 22:19:28 -07:00
Olaoluwa Osuntokun
646f79f566
routing: perform path finding inside a single DB transaction
This commit modifies the path finding logic such that all path finding
is done inside a _single_ database transaction. With this change, we
ensure that we don’t end up possibly creating hundreds of database
transactions slowing down the path finding and payment sending process
all together.
2017-10-10 22:19:27 -07:00
Olaoluwa Osuntokun
ce7179a468
routing: add basic route pruning in response to HTLC onion errors
This commit adds basic route pruning in response to HTLC onion errors.
With this new change, the router will now prune routes in response to
HTLC errors, which will reduce the time to payment success, and also
avoid a bunch of unnecessary network traffic.

We now respond to two errors lnwire.FailTemporaryChannelFailure and
lnwire.FailUnknownNextPeer. In response to the first error, we’ll prune
all routes that contain the channel which was unable to be routed over.
In response to the second error we’ll prune all routes that contain the
node which couldn’t be found.
2017-10-10 22:19:25 -07:00
Olaoluwa Osuntokun
8d7f3943bb
routing: add two new methods to filter routes based on node/channel 2017-10-10 22:19:24 -07:00
Olaoluwa Osuntokun
f6ac31281b
routing: also include the source node in the nextHopMap index
In this commit we modify the newRoute function to also add the source
node to the nextHopMap index. With this addition the indexes will now
allow the router to react based on failures that occur during the
_first_ hop, meaning the channel directly attached to the source node.
2017-10-10 22:19:23 -07:00
Olaoluwa Osuntokun
3b7855e449
routing: implement 2-week zombie channel pruning
This commit implements 2-week zombie channel pruning. This means that
every GraphPruneInterval (currently set to one hour), we’ll scan the
channel graph, marking any channels which haven’t had *both* edges
updated in 2 weeks as a “zombie”. During the second pass, all “zombie”
channel are removed from the channel graph all together.

Adding this functionality means we’ll ensure that we maintain a
“healthy” network view, which will cut down on the number of failed
HTLC routing attempts, and also reflect an active portion of the graph.
2017-10-04 20:46:09 -07:00
Olaoluwa Osuntokun
e81689057a
discovery+routing: remove DeleteEdge from ChannelGraphSource interface
This commit removes the recently added DeleteEdge method from the
ChannelGraphSource interface as it’s no longer needed.
2017-10-04 20:46:08 -07:00
Brandon
3907ae65c2 routing+discovery: implement 2-week network view pruning 2017-10-04 20:40:21 -07:00
Laura Cressman
156772d04a channeldb: use binary.Read/Write with bools in channel.go
Use binary.Read/Write in functions to serialize and deserialize
channel close summary and HTLC boolean data, as well as in
methods to put and fetch channel funding info. Remove lnd
implementations of readBool and writeBool as they are no
longer needed. Also fix a few minor typos.
2017-10-02 23:13:47 -07:00
Laura Cressman
29687f49eb routing: replace sort.Sort with sort.Slice in router.go
Use sort.Slice in FindRoutes function in routing/router.go, as part
of the move to use new language features. Remove sortableRoutes type
wrapper for slice of Routes since it is no longer needed to sort routes.
2017-10-02 23:13:47 -07:00
Olaoluwa Osuntokun
3ba70fe6ec
routing: add preliminary version of more intelligent payment error handling
This commit modifies the way we currently interpret errors when sending
payments via the SendToSwitch method. We split the errors into two
broad sections: critical errors which cause us to abandon the payment
dispatch all together, and errors which are transient meaning we should
continue trying to remainder of the returned routes.

Note that we haven’t yet properly implemented all the necessary
measures such as filtering edges that are detected as being temporarily
inactive, etc.

This change should correct erroneous behavior such as continuing to try
all available routes in the face of an invalid payment hash error and
the like.
2017-10-02 22:14:13 -07:00
Olaoluwa Osuntokun
486b464e1c
routing: move path caching into FindRoutes
This commit modifies the way we do path caching. Rather than only
caching within SendPayment, we now cache routes within FindRoutes. This
is more natural as SendPayment eventually calls FindRoute. As a result
of this commit, queries to FindRoute are now properly cached, speeding
up applications which are focused on graph visualization or querying
rather than sending payments.
2017-10-02 22:14:13 -07:00
John Griffith
1057a1a7c3 routing: handle onion errors in ChannelRouter 2017-10-02 22:13:05 -07:00
Olaoluwa Osuntokun
0e626ce42c
routing: add a select+quit case when receiving error to ensure graceful shutdown 2017-09-25 20:55:09 -07:00
Olaoluwa Osuntokun
b07e7fb7cc
routing: hop-payload for last hop should be the absolute timeout, not delta
This commit fixes an oversight in the path finding code when converting
a path into a route. Currently, for the last hop, we’d emplace the
expiry delta of the last hop within the per-hop payload. This was left
over from a prior version of the specification.

To fix this, we’ll now emplace the _absolute_ final HTLC expiry with
the payload, such that, the final hop that verify that the HTLC has not
been tampered with in flight.
2017-09-12 21:27:47 +02:00
Olaoluwa Osuntokun
f5d221012d
routing: update ChannelGraphSource due to latest API changes 2017-08-22 00:53:36 -07:00
Olaoluwa Osuntokun
bc4ad34190
routing: don't re-validate a channel update's edge if it already exists
By avoiding re-validating the channel edge, we avoid wasted network
bandwidth and queries.
2017-08-22 00:53:33 -07:00
Olaoluwa Osuntokun
6467fdd829
routing: update path finding and notifications to use mSAT 2017-08-22 00:53:12 -07:00
Olaoluwa Osuntokun
67f17b319a
routing: invalidate routing cache on each new block
This commit makes the routing cache invalidation a bit more aggressive.
We now invalidate the cache on each new block as the routes in the
cache are based on the current block height. Using the cached items may
cause our routes to fail due to them having time locks which have
already expired.
2017-08-02 21:07:06 -07:00
Olaoluwa Osuntokun
f61d977176
routing: obtain current height when creating a new route 2017-08-02 21:02:24 -07:00