This commit adds caching to our route finding. Caching is done on a
tuple-basis mapping a (dest, amt) pair to a previously calculated set
of shortest paths. The cache invalidated on two occasions: when a block
closes a set of transactions, or we received a new channel update or
channel announcement message.
With this change, payments are now snappier from the PoV of an
application developer as we no longer need to do a series of disk-seeks
before we dispatch each payment.
This commit adds payment route failure fallback to SendPayment. By
this, we mean that we now take all the possible routes found during
path finding and try them in series. Either a route fails and we move
onto the next one, or the route is successful and we terminate early.
With this commit, sending payments using lnd is now much more robust as
if there exists an eligible route with sufficient capacity, it will be
utilized.
This commit modifies the existing FindRoute method on the ChannelRouter
to now use the KSP implementation added in a prior commit.
This new method FindRoutes, is able to find all the possible paths
between a source and destination. The method takes all paths reported
by findPaths, and attempt to turn each of them into a route. A route
differs from a path in that is has complete time-lock and fee
information. Some paths may not be able to be turned into routes as
once fees are accounted for the have an insufficient flow. We then take
the routes, sort them by total fee (with time-lock being a
time-breaker), then return them in sorted order.
With this commit we make our routing more robust by looking for the
k-shortest paths rather than a single shortest path and using that
unconditionally. In a nut shell Yen’s algorithm does the following:
* Find the shortest path from the source to the destination
* From k=1…K, walk the kth shortest path and find possible
divergence from each node in the path
Our version of Yen’s implemented is slightly modified, rather than
actually increasing the edge weights or remove vertexes from the graph,
we instead use two black-lists: one for edges and the other for
vertexes. Our modified version of Djikstra’s algorithm is then made
aware of these black lists in order to more efficiently implement the
path iteration via spur and root node.
This commit modifies the findRoute method by first calling it findPath,
but also making the following modifications.
First, two new black-listing maps are now passed in. These two maps
contain vertexes but also edges to ignore while performing path
finding. These maps will be used in order to ensure that we don’t
duplicate paths or back-track when executing our KSP algorithm.
Next, we now ensure that the path returned from the findPath function
is ordered properly in the direction of source to target. Such a change
is required for our KSP algorithm to function properly.
This commit adds a new heap structure to heap.go which will be used for
storing candidate paths during the iterations of the k-shortest paths
algorithm.
This commit modifies the findRoute function to decouple the
validation+creation of a route, from the path finding algorithm itself.
When we say “route”, we mean the full payment route complete with
time-lock and fee information. When we say “path” we simple mean an
ordered set of channel edges from one node to another target node.
With this commit we can now perform path finding independent of route
creation which will be needed in the up coming refactor to implement a
new modified k-shortest paths algorithm.
This commit slightly modified findRoute to accept the node which should
be used as the starting point in our path finding algorithm. With this
change, as we move to a k-shortest paths algorithm this modification
will be needed as all of our path finding attempts won’t always
originate from a the same starting point.
In this commit we now utilize the node distance heap that was added in
a prior commit into our core path finding logic. With this new data
structure, we no longer linearly scan the distance of all vertexes from
the source node when deciding which one to greedily explore.
Instead, we now start with the source added to our distance heap, then
new vertexes are progressively added to our heap as their edges are
explored. With this change, we move the computational complexity of our
path finding algorithm closer to the theoretical limit.
This commit modifies our modified version of Dijkstra's to include
sufficient link capacity within the algorithm’s relaxation condition.
With this change, we’ll now avoid exploring the graph in the direction
of a link that has insufficient capacity, allowing us to terminate path
finding more quickly.
This commit introduces a new heap struct that will be used to keep
track of the next closest node to the source during path finding within
our modified Dijkstra's algorithm.
This commit fixes a bug which was originally introduced when the
topology notifications were added to the channel router. The issue was
that a pointer to the loop-scope range variable was being passed into
the goroutine which dispatches the notification rather than the value
itself. It seems that the memory location is re-used between range
iterations causing the same client to receive _all_ the notifications.
This bug is fixed by passing a copy of the client struct rather than a
pointer to the range variable.
In the process, we also add some additional debug logging messages, and
remove the Curve parameter from any public keys involved in a
notification so the pretty print properly.
This commit modifies the `ChannelEdgeUpdate` struct to include the
channel point itself within the notifications. Such a change improves
the notificaiton experience for callers as it allows them to filter out
update notifications based on a familiar object within the codebase: a
channel point.
This commit adds some new functionality to the channel router: the
ability to dispatch notification to registered clients upon either a
channel being closed, a new node appearing, or an exiting client being
updated or opened for the first time.
With this change, the integration tests will now be able to eliminate
most of the sleep as we gain a new syntonization point into the
propagation of information within the test network. Additionally, this
also paves the way for client side software to dynamically visualize
the channel graph in real-time as nodes+channels are updated.
This commit modifies the path finding routines to properly use the new
channel edge related API exposed by the database. Additionally, a new
type `ChannelHop` has been introduced which couples an edges routing
policy with the capacity and origin chain of the channel.
This commit fixes bug that could result in the panicking or crashing of
nodes in the case of an at-funding-time reorganization within the
network. In order to avoid such a case, we now ensure that the
advertised transaction index is within the bounds of the block before
attempting to access it.
Note that this is a temporary patch commit until full advertisement
validation which is implemented in the discovery PR lands in master.
Additionally, better reorg handling during the funding process is being
specified within the spec and will properly be implemented within lnd
at a later date.
This commit fixes a prior block propagation race-condition by detecting
and properly processing “premature” announcements. A premature
announcement is one that’s received with an anchored block height which
is beyond our chain tip. Once received, we now store these
announcements in a special map that’s caches them in memory. Once a new
block arrives, we check the map for the existence of any entries,
processing them as normal if so.
This fixes the bug reproduced in #114. The prevHop map in newRoute may
include many more edges than what is used to produce the final route, and
thus the check prior to building the route could result with incorrect
errors being reported. We move this check to after the number of edges
to be used for the route is deduced.
This commit moves much of the logic for querying for a potential route,
constructing the HTLC including the Sphinx packet, and sending the
ultimate payment from the rpcServer to the ChannelRouter.
This movement paves the way for muilt-path path finding as well as
adding automatic retry logic to the ChannelRouter. Additionally, by
having the ChannelRouter construct the Sphinx packet, we’ll be able to
also include the proper time-lock and general per-hop-payload
information properly in the future.
This commit adds new behavior to the ChannelRouter struct: we know
rebroadcast our outgoing channels every 30 minutes. This new behavior
should ensure that both directions of an advertised channel edge are
always propagated though the network, fixing the issue of “ghost” edges
which exist but aren’t advertised.
This commit fixes a slight bug in the storage of the capacity of a
channel. Previously, we were subtracting a the hard coded fee amount
without first casting the integer to a btcutil.Amount which results in
a display/rounding error when the amount is converted to BTC.
This commit is similar to the prior commit to channeldb: we no longer
assume that _both_ edges of a channel will always be advertised. Such
an assumption resulted in the inability for a node to sync graph state
since we were previously returning an error when _both_ edges weren’t
found within the graph database.
To remedy this bug, we now carefully ensure that if one edge doesn’t
exist, then we still sync the other.
This commit fixes a bug in the test data that was uncovered due to the
recent bug fix within the AddChannelEdge method within the ChannelGraph
struct of channeldb.
The storage and assertion of unique channel ID’s wasn’t correct due to
bug in the channeldb which caused the defect in the test data to go
unnoticed.
This commit seems to fix a sporadic error within the integration tests
which would at times cause a panic when a payment as initiated.
This issue was with the way were deleting from the middle of the slice
of unvisited nodes within the graph. Assigning the last element to the
middle would at times cause a panic the last element may be nil. To fix
this, we now manually copy every item over by one, preserving the order
of the slice, and possibly fixing the panic once and for all.
This commit makes a large number of minor changes concerning API usage
within the deamon to match the latest version on the upstream btcsuite
libraries.
The major changes are the switch from wire.ShaHash to chainhash.Hash,
and that wire.NewMsgTx() now takes a paramter indicating the version of
the transaction to be created.
This commit slightly refactors the logic for the new outgoing payment
related RPC’s to more closely match the style of the rest of the
codebase. Additionally the tests have been updated to reflect the
changes to the protos of the new RPC’s.
This commit overhauls the routing package significantly to simplify the
code, conform to the rest of the coding style within the package, and
observe the new authenticated gossiping scheme outlined in BOLT07.
As a major step towards a more realistic path finding algorithm, fees
are properly calculated and observed during path finding. If a path has
sufficient capacity _before_ fees are applied, but afterwards the
finalized route would exceed the capacity of a single link, the route
is marked as invalid.
Currently a naive weighting algorithm is used which only factors in the
time-lock delta at each hop, thereby optimizing for the lowest time
lock. Fee calculation also isn’t finalized since we aren’t yet using
milli-satoshi throughout the daemon. The final TODO item within the PR
is to properly perform a multi-path search and rank the results based
on a summation heuristic rather than just return the first (out of
many) route found.
On the server side, once nodes are initially connected to the daemon,
our routing table will be synced with the peer’s using a naive “just
send everything scheme” to hold us over until I spec out some a
efficient graph reconciliation protocol. Additionally, the routing
table is now pruned by the channel router itself once new blocks arrive
rather than depending on peers to tell us when a channel flaps or is
closed.
Finally, the validation of peer announcements aren’t yet fully
implemented as they’ll be implemented within the pending discovery
package that was blocking on the completion of this package. Most off
the routing message processing will be moved out of this package and
into the discovery package where full validation will be carried out.
Use [33]byte for graph vertex representation.
Delete unneeded stuff:
1. DeepEqual for graph comparison
2. EdgePath
3. 2-thread BFS
4. Table transfer messages and neighborhood radius
5. Beacons
Refactor:
1. Change ID to Vertex
2. Test use table driven approach
3. Add comments
4. Make graph internal representation private
5. Use wire.OutPoint as EdgeId
6. Decouple routing messages from routing implementation
7. Delete Async methods
8. Delete unneeded channels and priority buffer from manager
9. Delete unneeded interfaces in internal graph realisation
10. Renamed ID to Vertex
