Commit Graph

154 Commits

Author SHA1 Message Date
Wilmer Paulino
4598df054e
routing: modify findPath to take into account additional edges
In this commit, we modify our path finding algorithm to take an
additional set of edges that are currently not known to us that are
used to temporarily extend our graph with during a payment session.
These edges should assist the sender of a payment in successfully
constructing a path to the destination.

These edges should usually represent private channels, as they are not
publicly advertised to the network for routing.
2018-04-20 04:01:32 -04:00
Wilmer Paulino
1e5659e5f4
routing: introduce routing hints to lightning payments
In this commit, we add a new field of routing hints to payments over the
Lightning Network. These routing hints can later be used within the path
finding algorithm in order to craft a path that will reach the
destination succesfully.
2018-04-20 04:01:29 -04:00
Dimitris Tsapakidis
4009f7f874 multi: fix typos in comments 2018-04-17 19:03:27 -07:00
Olaoluwa Osuntokun
1365392d2b
routing: don't traverse links that advertise a timelock delta of zero 2018-03-23 17:55:14 -07:00
Olaoluwa Osuntokun
800eea931f
build+multi: switch from bolt to bbolt
In this commit, we switch from boltbd/bolt to coreos/bbolt as the
former is no longer being actively maintained.
2018-03-10 19:01:13 -08:00
Olaoluwa Osuntokun
528aa67df7
routing: weight findPath more heavily towards fees during edge relaxation
In this commit, we modify the edgeWeight function that’s used within
the findPath method to weight fees more heavily than the time lock
value at an edge. We do this in order to greedily prefer lower fees
during path finding. This is a simple stop gap in place of more complex
weighting parameters that will be investigated later.

We also modify the edge distance to use an int64 rather than a float.
Finally an additional test has been added in order to excessive this
new change. Before the commit, the test was failing as we preferred the
route with lower total time lock.
2018-02-12 16:27:38 -08:00
Olaoluwa Osuntokun
33ba60ce66
routing: modify findPaths to take an upper limit on the number of routes
In this commit, we modify the findPaths method to take the max number
of routes to return. With this change, FindRoutes can eventually itself
also take a max number of routes in order to make the function useable
again.
2018-02-12 16:18:49 -08:00
Olaoluwa Osuntokun
6751cd8b9f
routing: update package to account for recent channeldb API changes 2018-02-06 20:14:32 -08:00
Olaoluwa Osuntokun
d70e4bb0a0
routing: account for case where final destination send TemporaryChannelFailure
In this commit, we fix an existing bug that could cause lnd to crash if
we sent a payment, and the *destination* sent a temp channel failure
error message. When handling such a message, we’ll look in the nextHop
map to see which channel was *after* the node that sent the payment.
However, if the destination sends this error, then there’ll be no entry
in this map.

To address this case, we now add a prevHop map. If we attempt to lookup
a node in the nextHop map, and they don’t have an entry, then we’ll
consult the prevHop map.

We also update the set of tests to ensure that we’re properly setting
both the prevHop map and the nextHop map.
2018-01-10 15:15:55 -08:00
Matt Drollette
adf0d98194 multi: fix several typos in godoc comments 2017-12-17 18:40:05 -08:00
Olaoluwa Osuntokun
7a3116f8a9
routing: ensure we set the TotalFees value on routes properly
Before this commit, we wouldn’t properly set the TotalFees attribute.
As a result, our sorting algorithm at the end to select candidate
routes would simply maintain the time-lock order rather than also sort
by total fees. This commit fixes this issue and also allows the test
added in the prior commit to pass.
2017-12-06 18:45:52 -08:00
Olaoluwa Osuntokun
a78ff8ba04
routing: during path finding skip edges that are marked disabled
In this commit, we implement adherence of the disabled bit within a
ChannelUpdate during path finding. If a channel is marked as disabled,
then we won’t attempt to route through it. A test has been added to
exercise this new check.
2017-11-30 22:31:27 -08:00
Olaoluwa Osuntokun
b61670fe23
routing: during path finding skip edge if amount < edge.MinHTLC
In this commit, we update path finding to skip an edge if the amount
we’re trying to route through it is below the MinHTLC (in mSAT) value
for that node. We also add a new test to exercise this behavior. In
order for out test to work properly, we’ve modified the JSON to make
the edge to Goku have a higher min HTLC value.
2017-11-30 22:29:45 -08:00
Laura Cressman
39d38da732 discovery: create deDupedAnnouncements struct in gosspier.go
For Part 1 of Issue #275. Create isolated private struct in
networkHandler goroutine that will de-duplicate
announcements added to the batch. The struct contains maps
for each of channel announcements, channel updates, and
node announcements to keep track of unique announcements.

The struct has a Reset method to reset stored announcements, an
AddMsg(lnwire.Message) method to add a new message to the current
batch, and a Batch method to return the set of de-duplicated
announcements.

Also fix a few minor typos.
2017-11-15 16:50:19 -08:00
Olaoluwa Osuntokun
a7fe7ae941
routing: correct miscalculation multi-hop onion payload fees
In this commit we fix an existing miscalculation in the fees that we
prescribe within the onion payloads for multi-hop routes. Before this
commit, if a route had more than 3 hops, then we would erroneously give
the second to last hop zero fees.

In this commit we correct this behavior, and also re-write the fee
calculation code fragment within newRoute for readability and clarity.
There are now only two cases: this is the last hop, and this is any
other hop. In the case of the last hop, simply send the exact amount
with no additional fee. In the case of an intermediate hop, we use the
_prior_ (closer to the destination) hop to calculate the amount of fees
we need, which allows us to compute the incoming flow. Using that
incoming flow, we then can compute the amount that the hop should
forward out.

Partially fixes #391.
2017-10-24 18:27:35 -07:00
Olaoluwa Osuntokun
4b82e2ec43
routing: correct fee calculation when converting from path to route
In this commit, we correct the fee calculation when converting from a
path to route. Previously we would apply the “no fee” case at the
_first_ hop, rather than the last hop. As a result, we needed to swap
the edges during path finding, otherwise, if the incoming and outgoing
edges had different fee rates, then we would create an invalid onion
payload.

In this commit we now properly switch fee calculation into three cases:
  * a single hop route, so there’s no fee
  * we’re at the first hop in a multi hop route, and we apply the fee
for the _next_ hop
  * we’re at an intermediate hop and the fee calculation proceeds as
normal
2017-10-22 18:36:50 -07:00
Olaoluwa Osuntokun
8df69a83a7
routing: revert the swapping of in/out edges during path finding
In this commit we revert a commit which was added in the past as way to
allow the path -> route conversion code to remain the same, while
properly respecting the necessary time locks and fees. In an upcoming
change, this swap is no longer necessary as we’ll always use: the time
lock of the outgoing node and the fee of the incoming node.
2017-10-22 18:36:49 -07:00
Olaoluwa Osuntokun
23b6d84493
routing: modify newPath to take the preferred final CLTV delta of last hop
In this commit, rather than reading the final CLTV delta from the
channel graph itself (which would require _both_ edges to be advertised
in order to route over), we now instead have moved to allowing the
receiving node to choose their own final CLTV delta.
2017-10-22 18:36:48 -07:00
Olaoluwa Osuntokun
ae6bde2d77
routing: avoid internal bolt db deadlock by reusing transaction in findPath
This commit fixes a bug that could lead to a deadlock inside bolt db
itself. In a recent commit we allowed a db transaction to be passed
directly into findPath, however, the initial call to graph.ForEachNode
instead passed a _nil_ transaction causing the method itself to create
a _new_ transaction, leading to a deadlock.

We fix this issue by instead re-using the transaction pointer.
2017-10-16 18:48:27 -07:00
Olaoluwa Osuntokun
646f79f566
routing: perform path finding inside a single DB transaction
This commit modifies the path finding logic such that all path finding
is done inside a _single_ database transaction. With this change, we
ensure that we don’t end up possibly creating hundreds of database
transactions slowing down the path finding and payment sending process
all together.
2017-10-10 22:19:27 -07:00
Olaoluwa Osuntokun
f6ac31281b
routing: also include the source node in the nextHopMap index
In this commit we modify the newRoute function to also add the source
node to the nextHopMap index. With this addition the indexes will now
allow the router to react based on failures that occur during the
_first_ hop, meaning the channel directly attached to the source node.
2017-10-10 22:19:23 -07:00
Olaoluwa Osuntokun
70e114fa6f
routing: add additional indexes to the Route struct to allow for querying
This commit adds three new indexes to the Route struct. These indexes
allow a caller to check if a channel is in the route, check if a node
is in the route, query the next node after a target node, and query the
next channel after a target node. The combination of these new indexes
will allow the ChannelRouter to prune away routes from the available
set in response to any received errors.
2017-10-10 22:19:22 -07:00
Olaoluwa Osuntokun
7ba6b7fa09
routing: add a String() method to the vertex type 2017-10-10 22:19:22 -07:00
Laura Cressman
29687f49eb routing: replace sort.Sort with sort.Slice in router.go
Use sort.Slice in FindRoutes function in routing/router.go, as part
of the move to use new language features. Remove sortableRoutes type
wrapper for slice of Routes since it is no longer needed to sort routes.
2017-10-02 23:13:47 -07:00
Olaoluwa Osuntokun
b07e7fb7cc
routing: hop-payload for last hop should be the absolute timeout, not delta
This commit fixes an oversight in the path finding code when converting
a path into a route. Currently, for the last hop, we’d emplace the
expiry delta of the last hop within the per-hop payload. This was left
over from a prior version of the specification.

To fix this, we’ll now emplace the _absolute_ final HTLC expiry with
the payload, such that, the final hop that verify that the HTLC has not
been tampered with in flight.
2017-09-12 21:27:47 +02:00
Olaoluwa Osuntokun
321cc28cd8
routing: in findPath skip edge if incoming edge isn't advertised 2017-08-22 00:54:15 -07:00
Olaoluwa Osuntokun
5301da790c
routing: fix path finding, bug use the proper policy during path finding
This commit fixes an lingering bug within the path finding logic of the
router. Previously we used the edge policy directly attached to the
outgoing channel of the node we were traversing to calculate the fees
and time lock information. This is incorrect, as we instead should be
using the policy of the *connecting* node as we’ll need to pay for
transit as they dictate.

To remedy this, we now grab the incoming+outgoing edges and use those
accordingly when building the initial path.
2017-08-22 00:53:15 -07:00
Olaoluwa Osuntokun
6467fdd829
routing: update path finding and notifications to use mSAT 2017-08-22 00:53:12 -07:00
Olaoluwa Osuntokun
5ef077e5c8
routing: cap number of yen's algorithm iterations at 100
This commit makes a precautionary change in order to ensure that the
upper bound on the number of iteration’s within our version of Yen’s
algorithm is fixed.
2017-08-15 19:56:41 -07:00
Olaoluwa Osuntokun
d331ddd2f4
routing: when creating a route, base time locks off current height
This commit implements some missing functionality, namely before all
time locks were calculated off of a base height of 0 essentially.
That’s incorrect as all time locks within HTLC’s would then be already
expired. We remedy this requesting the latest height when creating a
route to ensure that our time locks are set properly.
2017-08-02 21:01:54 -07:00
Olaoluwa Osuntokun
4d8bb21d9d
routing: add ToHopPayloads method to routing.Route
This commit adds a new method to the routing.Route struct:
ToHopPayloads. This function will converts a complete route into the
series of per-hop payloads that is to be encoded within each HTLC using
an opaque Sphinx packet.

We can now use this function when creating the sphinx packet to
properly encoded the hop payload for each hop in the route.
2017-06-16 22:37:47 +02:00
Olaoluwa Osuntokun
62cd6ee046
routing: populate the OutgoingTimeLock field within route hops
This commit inches towards fully validation+adherance of the per-hop
payloads within an HTLC’s route by properly calculating the outgoing
time lock value for each hop according to the current draft
specification.
2017-06-16 22:33:59 +02:00
Olaoluwa Osuntokun
a0c2278a69
routing: add TODO in newRoute for more sanity checks 2017-04-16 15:24:17 -07:00
Olaoluwa Osuntokun
79807022a5
routing: update graph traversal to use latest API 2017-04-14 13:15:00 -07:00
Olaoluwa Osuntokun
5442e42cc1
routing: fix slice mutation bug that could result in an infinite loop
This commit fixes a pretty nasty unnoticed bug within the main
k-shortest paths algorithm loop. After a new candidate path is found,
the rootPath (the path up to the pivot node) and the spurPath (the
_new_ path after the pivot node) are to be combined into a new candiate
shortest path. The prior logic simply appended the spurPath onto the
end of the rootPath to create a slice. However, if the case that the
currnet rootPath is really a sub-path in a larger slice, then this will
mutate the underlying slice.

This bug would manifest when doing path finding and cause an infinite
loop as the slice kept growing with new spurPaths, causing the loop to
never terminate. We remedy this bug by properly create a new backing
slice, and adding the elements to them rather than incorrectly mutating
an underlying slice.
2017-04-13 14:48:17 -07:00
Olaoluwa Osuntokun
a4e26eaa4a
routing: fix bug in path finding when len(rootPath) > len(shortestPath)
This commit fixes a bug within the k-shortest paths routine which could
result in a daemon panic when traversing a graph with particular
characteristics. Before referencing the path to create a sub-slice, we
we’re properly asserting that the length of the path was at least as
long as the current rootPath in question. We fix this by simply
ensuring the length of the slice is adequate before proceeding with the
operation.
2017-04-13 14:44:59 -07:00
Olaoluwa Osuntokun
2d10d83f07
routing: assert that paths have same length in isSamePath 2017-04-13 14:42:35 -07:00
Andrey Samokhvalov
b4ac7071ff discovery+routing: split 'routing' package on 'routing' and 'discovery'
In this commit the routing package was divided on two separete one,
this was done because 'routing' package start take too much responsibily
on themself, so with following commit:

Routing pacakge:
Enitites:
* channeldb.ChannelEdge
* channeldb.ChannelPolicy
* channeldb.NodeLightning

Responsibilities:
* send topology notification
* find payment paths
* send payment
* apply topology changes to the graph
* prune graph
* validate that funding point exist and corresponds to given one
* to be the source of topology data

Discovery package:
Entities:
* lnwire.AnnounceSignature
* lnwire.ChannelAnnouncement
* lnwire.NodeAnnouncement
* lnwire.ChannelUpdateAnnouncement

Responsibilities:
* validate announcement signatures
* sync topology with newly connected peers
* handle the premature annoucement
* redirect topology changes to the router susbsystem
* broadcast announcement to the rest of the network
* exchange channel announcement proofs

Before that moment all that was in the 'routing' which is quite big for
one subsystem.

split
2017-03-29 19:49:05 -07:00
Olaoluwa Osuntokun
69b257154f
routing: fix build on go 1.7.5 by using sort.Interface instead of sort.Lice
This commit fixes the issue of broken builds in versions other than go
1.7.5 by sorting according to the sort.Interface interface rather than
the newly available sort.Slice function.
2017-03-21 12:21:00 -07:00
Olaoluwa Osuntokun
aaa04bb2e5
routing: add findPaths function whcih implements Yen's Algorithm
With this commit we make our routing more robust by looking for the
k-shortest paths rather than a single shortest path and using that
unconditionally. In a nut shell Yen’s algorithm does the following:
   * Find the shortest path from the source to the destination
   * From k=1…K, walk the kth shortest path and find possible
divergence from each node in the path

Our version of Yen’s implemented is slightly modified, rather than
actually increasing the edge weights or remove vertexes from the graph,
we instead use two black-lists: one for edges and the other for
vertexes. Our modified version of Djikstra’s algorithm is then made
aware of these black lists in order to more efficiently implement the
path iteration via spur and root node.
2017-03-21 12:20:37 -07:00
Olaoluwa Osuntokun
56d27f2d58
routing: rename findRoute to findPath, return path in forwards order
This commit modifies the findRoute method by first calling it findPath,
but also making the following modifications.

First, two new black-listing maps are now passed in. These two maps
contain vertexes but also edges to ignore while performing path
finding. These maps will be used in order to ensure that we don’t
duplicate paths or back-track when executing our KSP algorithm.

Next, we now ensure that the path returned from the findPath function
is ordered properly in the direction of source to target. Such a change
is required for our KSP algorithm to function properly.
2017-03-21 12:20:35 -07:00
Olaoluwa Osuntokun
fd18c2d036
routing: newRoute now expects the path in forwards order 2017-03-21 12:20:33 -07:00
Olaoluwa Osuntokun
e0ef63e4e0
routing: findRoute now returns a slice of selected hops in reverse order
This commit modifies the findRoute function to decouple the
validation+creation of a route, from the path finding algorithm itself.
When we say “route”, we mean the full payment route complete with
time-lock and fee information. When we say “path” we simple mean an
ordered set of channel edges from one node to another target node.

With this commit we can now perform path finding independent of route
creation which will be needed in the up coming refactor to implement a
new modified k-shortest paths algorithm.
2017-03-21 12:20:28 -07:00
Olaoluwa Osuntokun
c6c56173a8
routing: modify findRoute to accept starting node as a paramter
This commit slightly modified findRoute to accept the node which should
be used as the starting point in our path finding algorithm. With this
change, as we move to a k-shortest paths algorithm this modification
will be needed as all of our path finding attempts won’t always
originate from a the same starting point.
2017-03-21 12:20:25 -07:00
Olaoluwa Osuntokun
b6199f27da
routing: replace linear vertex scan with distanceHeap usage
In this commit we now utilize the node distance heap that was added in
a prior commit into our core path finding logic. With this new data
structure, we no longer linearly scan the distance of all vertexes from
the source node when deciding which one to greedily explore.

Instead, we now start with the source added to our distance heap, then
new vertexes are progressively added to our heap as their edges are
explored. With this change, we move the computational complexity of our
path finding algorithm closer to the theoretical limit.
2017-03-21 12:20:23 -07:00
Olaoluwa Osuntokun
20aba8060f
routing: add sufficient link capacity to our relaxation condition
This commit modifies our modified version of Dijkstra's to include
sufficient link capacity within the algorithm’s relaxation condition.
With this change, we’ll now avoid exploring the graph in the direction
of a link that has insufficient capacity, allowing us to terminate path
finding more quickly.
2017-03-21 12:20:21 -07:00
Olaoluwa Osuntokun
606b23df43
routing: introduce a heap to keep track of closest nodes during pathfinding
This commit introduces a new heap struct that will be used to keep
track of the next closest node to the source during path finding within
our modified Dijkstra's algorithm.
2017-03-21 12:20:18 -07:00
Olaoluwa Osuntokun
7bdf02bc9e
routing: modify path finding routines use new EdgeInfo/EdgePolicy
This commit modifies the path finding routines to properly use the new
channel edge related API exposed by the database. Additionally, a new
type `ChannelHop` has been introduced which couples an edges routing
policy with the capacity and origin chain of the channel.
2017-03-08 14:25:05 -08:00
Christopher Jämthagen
4200d8a7e0 routing: check route length after edges for route is collected
This fixes the bug reproduced in #114. The prevHop map in newRoute may
include many more edges than what is used to produce the final route, and
thus the check prior to building the route could result with incorrect
errors being reported. We move this check to after the number of edges
to be used for the route is deduced.
2017-02-02 00:24:16 -08:00
Trevin Hofmann
40c7bac3aa multi: fix a variety of typos throughout the repo 2017-01-17 17:02:56 -08:00
Olaoluwa Osuntokun
0c7fcb1755
routing: fix nil pointer panic when node has no outgoing channels 2017-01-17 13:07:05 -08:00
Olaoluwa Osuntokun
ec0c7c5989
routing: fix panic in inner loop of path finding
This commit seems to fix a sporadic error within the integration tests
which would at times cause a panic when a payment as initiated.

This issue was with the way were deleting from the middle of the slice
of unvisited nodes within the graph. Assigning the last element to the
middle would at times cause a panic the last element may be nil. To fix
this, we now manually copy every item over by one, preserving the order
of the slice, and possibly fixing the panic once and for all.
2017-01-07 21:22:23 -08:00
Olaoluwa Osuntokun
e327ffe954
routing: rewrite package to conform to BOLT07 and factor in fees+timelocks
This commit overhauls the routing package significantly to simplify the
code, conform to the rest of the coding style within the package, and
observe the new authenticated gossiping scheme outlined in BOLT07.

As a major step towards a more realistic path finding algorithm, fees
are properly calculated and observed during path finding. If a path has
sufficient capacity _before_ fees are applied, but afterwards the
finalized route would exceed the capacity of a single link, the route
is marked as invalid.

Currently a naive weighting algorithm is used which only factors in the
time-lock delta at each hop, thereby optimizing for the lowest time
lock. Fee calculation also isn’t finalized since we aren’t yet using
milli-satoshi throughout the daemon. The final TODO item within the PR
is to properly perform a multi-path search and rank the results based
on a summation heuristic rather than just return the first (out of
many) route found.

On the server side, once nodes are initially connected to the daemon,
our routing table will be synced with the peer’s using a naive “just
send everything scheme” to hold us over until I spec out some a
efficient graph reconciliation protocol. Additionally, the routing
table is now pruned by the channel router itself once new blocks arrive
rather than depending on peers to tell us when a channel flaps or is
closed.

Finally, the validation of peer announcements aren’t yet fully
implemented as they’ll be implemented within the pending discovery
package that was blocking on the completion of this package. Most off
the routing message processing will be moved out of this package and
into the discovery package where full validation will be carried out.
2016-12-27 16:44:22 -08:00
Olaoluwa Osuntokun
ddc1eb4c8a
lnwallet: correct comment for logging object 2016-12-27 16:43:07 -08:00