Commit Graph

517 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
486b464e1c
routing: move path caching into FindRoutes
This commit modifies the way we do path caching. Rather than only
caching within SendPayment, we now cache routes within FindRoutes. This
is more natural as SendPayment eventually calls FindRoute. As a result
of this commit, queries to FindRoute are now properly cached, speeding
up applications which are focused on graph visualization or querying
rather than sending payments.
2017-10-02 22:14:13 -07:00
John Griffith
1057a1a7c3 routing: handle onion errors in ChannelRouter 2017-10-02 22:13:05 -07:00
Jim Posen
d8a2ed27b8 routing/chainview: Fix data race in block disconnected callback. 2017-09-29 13:53:02 -07:00
Olaoluwa Osuntokun
0e626ce42c
routing: add a select+quit case when receiving error to ensure graceful shutdown 2017-09-25 20:55:09 -07:00
Olaoluwa Osuntokun
e5f3ee0fb6
chainntnfs+routing/chainview: reduce neutrino.WaitForMoreCFHeaders value
This commit reduces the neutrino.WaitForMoreCFHeaders parameter when
instantiating a neutrino instance as a lower value will allow the tests
to complete more quickly.
2017-09-13 16:46:11 +02:00
Olaoluwa Osuntokun
b07e7fb7cc
routing: hop-payload for last hop should be the absolute timeout, not delta
This commit fixes an oversight in the path finding code when converting
a path into a route. Currently, for the last hop, we’d emplace the
expiry delta of the last hop within the per-hop payload. This was left
over from a prior version of the specification.

To fix this, we’ll now emplace the _absolute_ final HTLC expiry with
the payload, such that, the final hop that verify that the HTLC has not
been tampered with in flight.
2017-09-12 21:27:47 +02:00
Olaoluwa Osuntokun
9f0efddc20
multi: switch from btcrpcclient to rpcclient 2017-08-24 18:54:24 -07:00
Olaoluwa Osuntokun
321cc28cd8
routing: in findPath skip edge if incoming edge isn't advertised 2017-08-22 00:54:15 -07:00
Olaoluwa Osuntokun
f5d221012d
routing: update ChannelGraphSource due to latest API changes 2017-08-22 00:53:36 -07:00
Olaoluwa Osuntokun
bc4ad34190
routing: don't re-validate a channel update's edge if it already exists
By avoiding re-validating the channel edge, we avoid wasted network
bandwidth and queries.
2017-08-22 00:53:33 -07:00
Olaoluwa Osuntokun
5301da790c
routing: fix path finding, bug use the proper policy during path finding
This commit fixes an lingering bug within the path finding logic of the
router. Previously we used the edge policy directly attached to the
outgoing channel of the node we were traversing to calculate the fees
and time lock information. This is incorrect, as we instead should be
using the policy of the *connecting* node as we’ll need to pay for
transit as they dictate.

To remedy this, we now grab the incoming+outgoing edges and use those
accordingly when building the initial path.
2017-08-22 00:53:15 -07:00
Olaoluwa Osuntokun
6467fdd829
routing: update path finding and notifications to use mSAT 2017-08-22 00:53:12 -07:00
Olaoluwa Osuntokun
5ef077e5c8
routing: cap number of yen's algorithm iterations at 100
This commit makes a precautionary change in order to ensure that the
upper bound on the number of iteration’s within our version of Yen’s
algorithm is fixed.
2017-08-15 19:56:41 -07:00
Olaoluwa Osuntokun
8c3441b30f
routing: update test to account for proper time locks 2017-08-02 21:07:35 -07:00
Olaoluwa Osuntokun
67f17b319a
routing: invalidate routing cache on each new block
This commit makes the routing cache invalidation a bit more aggressive.
We now invalidate the cache on each new block as the routes in the
cache are based on the current block height. Using the cached items may
cause our routes to fail due to them having time locks which have
already expired.
2017-08-02 21:07:06 -07:00
Olaoluwa Osuntokun
f61d977176
routing: obtain current height when creating a new route 2017-08-02 21:02:24 -07:00
Olaoluwa Osuntokun
d331ddd2f4
routing: when creating a route, base time locks off current height
This commit implements some missing functionality, namely before all
time locks were calculated off of a base height of 0 essentially.
That’s incorrect as all time locks within HTLC’s would then be already
expired. We remedy this requesting the latest height when creating a
route to ensure that our time locks are set properly.
2017-08-02 21:01:54 -07:00
Johan T. Halseth
39a59bbe6f routing: Require adding edge to node before adding node.
This commit introduces the requirement specified in BOLT#7,
where we ignore any node announcements for a specific node
if we yet haven't seen any channel announcements where this
node takes part. This is to prevent someone DoS-ing the
network with cheap node announcements. In the router this
is enforced by requiring a call to AddNode(node_id) to
be preceded by an AddEdge(edge_id) call, where node_id is
one of the nodes in edge_id.
2017-08-02 15:58:58 -07:00
Conner Fromknecht
14a06526b8 routing/notifs: order invariant testing of ntfn delivery (#238)
Modifies the test cases in `TestEdgeUpdateNotification` and
`TestNodeUpdateNotification` to check for the possibility of notifications
being delivered out of order.  This addresses some sporadic failures that
were observed when running the test suite. 

I looked through some of the open issues but didn't see any addressing this
issue in particular, but if someone could point me to any relevant issues
that would be much appreciated!

Issue
-----
Currently the test suite validates notifications received in the order they
are submitted. The check fails because the verification of each
notification is statically linked to the order in which they are delivered,
seen
[here](1be4d67ce4/routing/notifications_test.go (L403))
and
[here](1be4d67ce4/routing/notifications_test.go (L499))
in `routing/notifications_test.go`.  The notifications are typically
delivered in this order, but causes the test to fail otherwise.

Proposed Changes
-------------------
Construct an index that maps a public key to its corresponding edges and/or
nodes.  When a notification is received, use its identifying public key and
the index to look up the edge/node to use for validation. Entries are
removed from the index after they are verified to ensure that the same
entry is validated twice. The logic to dynamically handle the verification
of incoming notifications rests can be found here
[here](https://github.com/cfromknecht/lnd/blob/order-invariant-ntfns/routing/notifications_test.go#L420)
and
[here](https://github.com/cfromknecht/lnd/blob/order-invariant-ntfns/routing/notifications_test.go#L539).

Encountered Errors
--------------------
 * `TestEdgeUpdateNotification`: notifications_test.go:379: min HTLC of
   edge doesn't match: expected 16.7401473 BTC, got 19.4852751 BTC
 * `TestNodeUpdateNotification`: notifications_test.go:485: node identity
   keys don't match: expected
   027b139b2153ac5f3c83c2022e58b3219297d0fb3170739ee6391cddf2e06fe3e7, got
   03921deafb61ee13d18e9d96c3ecd9e572e59c8dbd0bb922b5b6ac609d10fe4ee4


Recreating Failing Behavior
---------------------------
The failures can be somewhat difficult to recreate, I was able to reproduce
them by running the unit tests repeatedly until they showed up.  I used the
following commands to bring them out of hiding:

```
./gotest.sh -i
go test -test.v ./routing && while [ $? -eq 0 ]; do go test -test.v ./routing; done
```

I was unable to recreate these errors, or any others in this package, after
making the proposed changes and leaving the script running continuously for
~30 minutes. Previously, I could consistently generate an error after ~20
seconds had elapsed on the latest commit in master at the time of writing:
78f6caf5d2e570fea0e5c05cc440cb7395a99c1d. Moar stability ftw!
2017-07-31 21:38:03 -07:00
Andrey Samokhvalov
2d378b3280 htlcswitch+router: add onion error obfuscation
Within the network, it's important that when an HTLC forwarding failure
occurs, the recipient is notified in a timely manner in order to ensure
that errors are graceful and not unknown. For that reason with
accordance to BOLT №4 onion failure obfuscation have been added.
2017-07-14 19:08:04 -07:00
Olaoluwa Osuntokun
9daa659bb3
routing/chainview: convert chainview integration tests to use sub-tests 2017-07-04 15:53:58 -07:00
Andrey Samokhvalov
8fa2b95c12 lnd: remove seelog logger
The btclog package has been changed to defining its own logging
interface (rather than seelog's) and provides a default implementation
for callers to use.

There are two primary advantages to the new logger implementation.

First, all log messages are created before the call returns.  Compared
to seelog, this prevents data races when mutable variables are logged.

Second, the new logger does not implement any kind of artifical rate
limiting (what seelog refers to as "adaptive logging").  Log messages
are outputted as soon as possible and the application will appear to
perform much better when watching standard output.

Because log rotation is not a feature of the btclog logging
implementation, it is handled by the main package by importing a file
rotation package that provides an io.Reader interface for creating
output to a rotating file output.  The rotator has been configured
with the same defaults that btcd previously used in the seelog config
(10MB file limits with maximum of 3 rolls) but now compresses newly
created roll files.  Due to the high compressibility of log text, the
compressed files typically reduce to around 15-30% of the original
10MB file.
2017-06-25 14:19:56 +01:00
Olaoluwa Osuntokun
9b9f419e51
routing: fix vet error, ensure wait group in topologyClient isn't copied 2017-06-25 13:48:52 +01:00
Olaoluwa Osuntokun
5c45d52ab6
routing: wait for topology clients to fully exit before closing ntfn chan
This commit fixes a send on closed channel panic by adding additional
synchronization when cancelling the notifications for a particular
topology client. We now ensure that all goroutines belonging to a
particular topology client exit fully before we close the notification
channel in order to avoid a panic.
2017-06-25 13:31:55 +01:00
Olaoluwa Osuntokun
1be4d67ce4
multi: run all test instances in parallel 2017-06-17 01:00:07 +02:00
Olaoluwa Osuntokun
2452f2ed82
routing: update test to assert correctness of per-hop payloads 2017-06-16 22:45:30 +02:00
Olaoluwa Osuntokun
fca51d6165
routing: use converted hop payloads for sphinx packet when creating onion 2017-06-16 22:43:00 +02:00
Olaoluwa Osuntokun
4d8bb21d9d
routing: add ToHopPayloads method to routing.Route
This commit adds a new method to the routing.Route struct:
ToHopPayloads. This function will converts a complete route into the
series of per-hop payloads that is to be encoded within each HTLC using
an opaque Sphinx packet.

We can now use this function when creating the sphinx packet to
properly encoded the hop payload for each hop in the route.
2017-06-16 22:37:47 +02:00
Olaoluwa Osuntokun
dfd37df0ed
routing: set expiry on initial outgoing HTLC using total time lock 2017-06-16 22:35:14 +02:00
Olaoluwa Osuntokun
62cd6ee046
routing: populate the OutgoingTimeLock field within route hops
This commit inches towards fully validation+adherance of the per-hop
payloads within an HTLC’s route by properly calculating the outgoing
time lock value for each hop according to the current draft
specification.
2017-06-16 22:33:59 +02:00
Olaoluwa Osuntokun
e4563ca13b
routing/chainview: make filter updates synchronous for neutrino
This commit fixes a possible race condition wherein a call to
FilterBlock after a call to UpdateFilter would result in the call to
FilterBlock not yet using the updated filter. We fix this by ensuring
the internal chain filter is updated by the time the call to
FilterBlock returns.
2017-06-09 12:18:33 -07:00
Olaoluwa Osuntokun
0c134a8cb3
routing/chainview: if the chain filter is empty, don't fetch the cfilter 2017-06-08 23:11:24 -07:00
Olaoluwa Osuntokun
38beeebe3d
routing/chainview: optimize neutrino implementation of FilterBlock
This commit optimizes the neutrino implementation of FilterBlock method
of the ChainView interface. The old implementation would _always_ fetch
the entire block and manually scan through it. Instead, we can just
fetch the filter, and then if the items match, fetch the block itself.
This will save bandwidth during a lnd node’s pruning of the channel
graph after a period of dormancy.
2017-06-08 22:24:03 -07:00
Olaoluwa Osuntokun
7d30634757
routing/chainview: fix race condition in neutrino implementation 2017-06-06 12:01:24 -07:00
Olaoluwa Osuntokun
9fd70958f4
multi: fix linter errors 2017-06-05 19:45:01 -07:00
Olaoluwa Osuntokun
cbf7cd48f5
multi: update to latest neutrino API changes 2017-06-05 19:41:07 -07:00
Olaoluwa Osuntokun
8f81133d6c
chainntnfs/neutrinonotify: add ChainNotifier implementation for neutrino
This commit adds an initial rough implementation father ChainNotifier
interface for neutrino, our new light client implementation. This
implementation largely borrows from the existing BtcdNotifier
implementation. As a result, a follow up commit will perform two
refactoring in order to further consolidate code.
2017-06-05 19:04:56 -07:00
Olaoluwa Osuntokun
48ea8a3e70
routing/chainview: add neutrino light client to set of interface tests 2017-06-05 18:58:53 -07:00
Olaoluwa Osuntokun
cc25049684
routing/chainview: add new light client p2p version of FilteredChainView
This commit adds a new implementation of the FilteredChainView
interface. This implementation speaks purely to the p2p network and is
backed by a new experimental light client implementation.
2017-06-05 18:57:36 -07:00
Olaoluwa Osuntokun
461d84dd75
routing/chainview: report line # in test failures, bump up timeouts 2017-06-05 18:54:58 -07:00
bryanvu
4ac7cc719f lnwallet: replace hard-coded fees and adjust tests accordingly
This commit replaces the hard-coded 5000 satoshi fees with calls to the
FeeEstimator interface. This should provide a way to cleanly plug in
additional fee calculation algorithms in the future. This change
affected quite a few tests. When possible, the tests were changed to
assert amounts sent rather than balances so that fees wouldn't need to
be taken into account. There were several tests for which this wasn't
possible, so calls to the static fee calculator were made.
2017-05-15 20:26:11 -07:00
Olaoluwa Osuntokun
a4ba72c08c
routing: close exit chan before ntfnChan to cancel topology clients
This commit fixes a panic due to a send on a closed channel that could
possibly occur depending on the order of channel closes when a client
goes to cancel a topology notification client.

Previously we closed the ntfnChan first, this would possible result in
a panic as the goroutine may have succeeded on a send at the same time
the channel was closed. Instead, we now close the `exit` channel first
which is meant to be a signal to the goroutine that the client has been
canceled.
2017-05-15 18:47:26 -07:00
Olaoluwa Osuntokun
bfc869739d
routing: fix linter error 2017-05-15 18:21:13 -07:00
Olaoluwa Osuntokun
40a7523b8f
routing: fix incorrect logging fmt directive, log chanPoint for undo fetch failure 2017-05-15 18:20:49 -07:00
Olaoluwa Osuntokun
cc19695dad
routing: reject channel announcement if connected vertexes are unknown
This commit modifies the processing in the routing package eo new
announcements. Previously,  if we cgot a cnew channel announcement but
didn’t yet know of the verses that the chanell connected, the
cnnounacment would be accepted. This behavior was eronoues as if the
channel were to be queried for, the DB query would fail as we would be
unable to retrieve the two nodes involved int he channel.

To avoid such an error case, we will now _reject_ any channel
announcements in which we don’t yet have a valid node announcement for
the connected nodes. This case has been inserted into the handling of
channel announcement, a new test has been added, and finally older
tests have also been updated to ensure that nodes are added to the
database _before_ the edge is.
2017-05-14 19:27:24 -07:00
Olaoluwa Osuntokun
31cb1cb65b
routing/chainview: linter fixes 2017-05-11 15:36:47 -07:00
Olaoluwa Osuntokun
828d28581a
routing: abandon ChainNotifier for FilteredChainView
This commit modifies the routing package to no longer use the
ChainNotifier for pruning the channel graph. Instead, we now use the
FilteredChainView interface to more (from the ChannelRouter’s PoV)
efficiently maintain the channel graph.

Rather than scanning the _entire_ block manually, we now rely on the
FilteredChainView to provide us with FilteredBlocks which include
_only_ the relevant transactions that we care about.
2017-05-11 15:20:48 -07:00
Olaoluwa Osuntokun
2ec0f5788e
routing/chainview: add behavioral interface level tests
This commit adds a new set of behavioral interface level tests to the
chain view package. This set of tests can now be used in order to check
proper conformity to this “specification” for all future
implementations of the chain view package.
2017-05-11 15:20:45 -07:00
Olaoluwa Osuntokun
7a42e31a44
routing/chainview: add btcd-websockets impl of FilteredChainView
This commit adds the first concrete implementation of the
chainview.FilteredChainView interface. The implementation of this
interface, BtcdFilteredChainView is backed by a web sockets connection
to an active btcd instance.
2017-05-11 15:20:43 -07:00
Olaoluwa Osuntokun
7bdd7023f4
routing/chainview: add new chainview package for watching subsets of the UTXO set
This commit creates a new package as sub-package within the routing
package: chainview. This package is centered around a single interface
definition: the FilteredChainView. This interface is to be used to
allow the routing package to watch a _subset_ of the UTXO set for any
modifications. In the case of LN, the subset of the UTXO set that we
care about is the set of currently opened channels.

In a future commit the routing package will be modified to remove the
current full block scanning with processing of FilteredBlock
notification, and proper updates to the filter as observed by the
FilteredChainView.
2017-05-11 15:20:41 -07:00
Olaoluwa Osuntokun
a0c2278a69
routing: add TODO in newRoute for more sanity checks 2017-04-16 15:24:17 -07:00
Olaoluwa Osuntokun
1bb225a9bd
routing: print proper error msg on payment send fail 2017-04-14 17:12:04 -07:00
Olaoluwa Osuntokun
79807022a5
routing: update graph traversal to use latest API 2017-04-14 13:15:00 -07:00
Olaoluwa Osuntokun
c8bf521c75
routing: display channel capacity in info logging message 2017-04-14 11:48:09 -07:00
Olaoluwa Osuntokun
5442e42cc1
routing: fix slice mutation bug that could result in an infinite loop
This commit fixes a pretty nasty unnoticed bug within the main
k-shortest paths algorithm loop. After a new candidate path is found,
the rootPath (the path up to the pivot node) and the spurPath (the
_new_ path after the pivot node) are to be combined into a new candiate
shortest path. The prior logic simply appended the spurPath onto the
end of the rootPath to create a slice. However, if the case that the
currnet rootPath is really a sub-path in a larger slice, then this will
mutate the underlying slice.

This bug would manifest when doing path finding and cause an infinite
loop as the slice kept growing with new spurPaths, causing the loop to
never terminate. We remedy this bug by properly create a new backing
slice, and adding the elements to them rather than incorrectly mutating
an underlying slice.
2017-04-13 14:48:17 -07:00
Olaoluwa Osuntokun
a4e26eaa4a
routing: fix bug in path finding when len(rootPath) > len(shortestPath)
This commit fixes a bug within the k-shortest paths routine which could
result in a daemon panic when traversing a graph with particular
characteristics. Before referencing the path to create a sub-slice, we
we’re properly asserting that the length of the path was at least as
long as the current rootPath in question. We fix this by simply
ensuring the length of the slice is adequate before proceeding with the
operation.
2017-04-13 14:44:59 -07:00
Olaoluwa Osuntokun
2d10d83f07
routing: assert that paths have same length in isSamePath 2017-04-13 14:42:35 -07:00
Olaoluwa Osuntokun
41a54145a7
routing: capitalize first letter of new error messages 2017-04-11 22:02:17 -07:00
Olaoluwa Osuntokun
ca053e5273
multi: minor coding style and comment clean ups post-discovery merge
This commit implements some minor coding style, commenting and naming
clean up after the recent major discovery service was merged into the
codebase.

Highlights of the naming changes:
* fundingManager.SendToDiscovery -> SendAnnouncement
* discovery.Discovery -> discovery.AuthenticatedGossiper

The rest of the changes consist primary of grammar fixes and proper
column wrapping.
2017-04-01 20:14:05 +02:00
Andrey Samokhvalov
07076cce21 routing+channeldb: add AddProof method
Originally we adding the edge without proof in order to able to use it
for payments path constrcution. This method will allow us to populate
the announcement proof after the exchange of the half proofs and
constrcutrion of full proof finished.
2017-03-29 19:49:05 -07:00
Andrey Samokhvalov
4bca54e30c routing: add validation of utxo
Add check that the edge that was received really exist in the blockchain
and that the announced funding keys and capcity corresponds to reality.
2017-03-29 19:49:05 -07:00
Andrey Samokhvalov
19174ebdfd channeldb: add storing of node signature and add edge signature
In order to properly announce the channel the announcements proofs
should be persistent in boltdb.
2017-03-29 19:49:05 -07:00
Andrey Samokhvalov
c3b2854428 lnwire: converge discovery part of messages with specification
Change the name of fields of messages which are belong to the discovery
subsystem in a such way so they were the same with the names that are
defined in the specification.
2017-03-29 19:49:05 -07:00
Andrey Samokhvalov
4c52b6e6a4 lnd: replace 'routing' with 'discovery' package
Add usage of the 'discovery' package in the lnd, now discovery service
will be handle all lnwire announcement messages and send them to the
remote party.
2017-03-29 19:49:05 -07:00
Andrey Samokhvalov
b4ac7071ff discovery+routing: split 'routing' package on 'routing' and 'discovery'
In this commit the routing package was divided on two separete one,
this was done because 'routing' package start take too much responsibily
on themself, so with following commit:

Routing pacakge:
Enitites:
* channeldb.ChannelEdge
* channeldb.ChannelPolicy
* channeldb.NodeLightning

Responsibilities:
* send topology notification
* find payment paths
* send payment
* apply topology changes to the graph
* prune graph
* validate that funding point exist and corresponds to given one
* to be the source of topology data

Discovery package:
Entities:
* lnwire.AnnounceSignature
* lnwire.ChannelAnnouncement
* lnwire.NodeAnnouncement
* lnwire.ChannelUpdateAnnouncement

Responsibilities:
* validate announcement signatures
* sync topology with newly connected peers
* handle the premature annoucement
* redirect topology changes to the router susbsystem
* broadcast announcement to the rest of the network
* exchange channel announcement proofs

Before that moment all that was in the 'routing' which is quite big for
one subsystem.

split
2017-03-29 19:49:05 -07:00
bryanvu
085b7333cb lnwire: add support for Features in NodeAnnouncement
Add support for Features in NodeAnnouncment according to spec.
2017-03-29 12:03:43 -07:00
bryanvu
9ffac9eae1 lnwire: update NodeAnnouncement to handle multiple addresses
This commit modifies address handling in the NodeAnnouncement struct,
switching from net.TCPAddr to []net.Addr. This enables more flexible
address handling with multiple types and multiple addresses for each
node. This commit addresses the first part of issue #131 .
2017-03-29 12:03:43 -07:00
Olaoluwa Osuntokun
6b3a258e86
multi: fix formatting issues in packge README's 2017-03-27 16:25:25 -07:00
Olaoluwa Osuntokun
69b257154f
routing: fix build on go 1.7.5 by using sort.Interface instead of sort.Lice
This commit fixes the issue of broken builds in versions other than go
1.7.5 by sorting according to the sort.Interface interface rather than
the newly available sort.Slice function.
2017-03-21 12:21:00 -07:00
Olaoluwa Osuntokun
3da8cd7551
routing: add a caching layer in front of the KSP algorithm
This commit adds caching to our route finding. Caching is done on a
tuple-basis mapping a (dest, amt) pair to a previously calculated set
of shortest paths. The cache invalidated on two occasions: when a block
closes a set of transactions, or we received a new channel update or
channel announcement message.

With this change, payments are now snappier from the PoV of an
application developer as we no longer need to do a series of disk-seeks
before we dispatch each payment.
2017-03-21 12:20:55 -07:00
Olaoluwa Osuntokun
b126298b2b
routing: implement route failure fallback in SendPayment
This commit adds payment route failure fallback to SendPayment. By
this, we mean that we now take all the possible routes found during
path finding and try them in series. Either a route fails and we move
onto the next one, or the route is successful and we terminate early.

With this commit, sending payments using lnd is now much more robust as
if there exists an eligible route with sufficient capacity, it will be
utilized.
2017-03-21 12:20:45 -07:00
Olaoluwa Osuntokun
9818f662cf
routing: exit early in FindRoutes if unable to convert any paths to a route 2017-03-21 12:20:42 -07:00
Olaoluwa Osuntokun
1df5bece85
routing: FindRoute is now FindRoutes, uses our KSP algo, ranks by fee
This commit modifies the existing FindRoute method on the ChannelRouter
to now use the KSP implementation added in a prior commit.

This new method FindRoutes, is able to find all the possible paths
between a source and destination. The method takes all paths reported
by findPaths, and attempt to turn each of them into a route. A route
differs from a path in that is has complete time-lock and fee
information. Some paths may not be able to be turned into routes as
once fees are accounted for the have an insufficient flow. We then take
the routes, sort them by total fee (with time-lock being a
time-breaker), then return them in sorted order.
2017-03-21 12:20:40 -07:00
Olaoluwa Osuntokun
aaa04bb2e5
routing: add findPaths function whcih implements Yen's Algorithm
With this commit we make our routing more robust by looking for the
k-shortest paths rather than a single shortest path and using that
unconditionally. In a nut shell Yen’s algorithm does the following:
   * Find the shortest path from the source to the destination
   * From k=1…K, walk the kth shortest path and find possible
divergence from each node in the path

Our version of Yen’s implemented is slightly modified, rather than
actually increasing the edge weights or remove vertexes from the graph,
we instead use two black-lists: one for edges and the other for
vertexes. Our modified version of Djikstra’s algorithm is then made
aware of these black lists in order to more efficiently implement the
path iteration via spur and root node.
2017-03-21 12:20:37 -07:00
Olaoluwa Osuntokun
56d27f2d58
routing: rename findRoute to findPath, return path in forwards order
This commit modifies the findRoute method by first calling it findPath,
but also making the following modifications.

First, two new black-listing maps are now passed in. These two maps
contain vertexes but also edges to ignore while performing path
finding. These maps will be used in order to ensure that we don’t
duplicate paths or back-track when executing our KSP algorithm.

Next, we now ensure that the path returned from the findPath function
is ordered properly in the direction of source to target. Such a change
is required for our KSP algorithm to function properly.
2017-03-21 12:20:35 -07:00
Olaoluwa Osuntokun
fd18c2d036
routing: newRoute now expects the path in forwards order 2017-03-21 12:20:33 -07:00
Olaoluwa Osuntokun
9f96ceb1e0
routing: introduce new heap for storing candidate shortest paths
This commit adds a new heap structure to heap.go which will be used for
storing candidate paths during the iterations of the k-shortest paths
algorithm.
2017-03-21 12:20:30 -07:00
Olaoluwa Osuntokun
e0ef63e4e0
routing: findRoute now returns a slice of selected hops in reverse order
This commit modifies the findRoute function to decouple the
validation+creation of a route, from the path finding algorithm itself.
When we say “route”, we mean the full payment route complete with
time-lock and fee information. When we say “path” we simple mean an
ordered set of channel edges from one node to another target node.

With this commit we can now perform path finding independent of route
creation which will be needed in the up coming refactor to implement a
new modified k-shortest paths algorithm.
2017-03-21 12:20:28 -07:00
Olaoluwa Osuntokun
c6c56173a8
routing: modify findRoute to accept starting node as a paramter
This commit slightly modified findRoute to accept the node which should
be used as the starting point in our path finding algorithm. With this
change, as we move to a k-shortest paths algorithm this modification
will be needed as all of our path finding attempts won’t always
originate from a the same starting point.
2017-03-21 12:20:25 -07:00
Olaoluwa Osuntokun
b6199f27da
routing: replace linear vertex scan with distanceHeap usage
In this commit we now utilize the node distance heap that was added in
a prior commit into our core path finding logic. With this new data
structure, we no longer linearly scan the distance of all vertexes from
the source node when deciding which one to greedily explore.

Instead, we now start with the source added to our distance heap, then
new vertexes are progressively added to our heap as their edges are
explored. With this change, we move the computational complexity of our
path finding algorithm closer to the theoretical limit.
2017-03-21 12:20:23 -07:00
Olaoluwa Osuntokun
20aba8060f
routing: add sufficient link capacity to our relaxation condition
This commit modifies our modified version of Dijkstra's to include
sufficient link capacity within the algorithm’s relaxation condition.
With this change, we’ll now avoid exploring the graph in the direction
of a link that has insufficient capacity, allowing us to terminate path
finding more quickly.
2017-03-21 12:20:21 -07:00
Olaoluwa Osuntokun
606b23df43
routing: introduce a heap to keep track of closest nodes during pathfinding
This commit introduces a new heap struct that will be used to keep
track of the next closest node to the source during path finding within
our modified Dijkstra's algorithm.
2017-03-21 12:20:18 -07:00
Olaoluwa Osuntokun
d0509955a7
routing: only log topology client sends if clients are active 2017-03-16 12:16:32 -07:00
Olaoluwa Osuntokun
8ed79ae497
routing: fix bug when sending topology ntfns to multiple clients
This commit fixes a bug which was originally introduced when the
topology notifications were added to the channel router. The issue was
that a pointer to the loop-scope range variable was being passed into
the goroutine which dispatches the notification rather than the value
itself. It seems that the memory location is re-used between range
iterations causing the same client to receive _all_ the notifications.

This bug is fixed by passing a copy of the client struct rather than a
pointer to the range variable.

In the process, we also add some additional debug logging messages, and
remove the Curve parameter from any public keys involved in a
notification so the pretty print properly.
2017-03-14 20:07:55 -07:00
Olaoluwa Osuntokun
f6ab1390ed
routing: include the channel point of chan within edge update ntfn
This commit modifies the `ChannelEdgeUpdate` struct to include the
channel point itself within the notifications. Such a change improves
the notificaiton experience for callers as it allows them to filter out
update notifications based on a familiar object within the codebase: a
channel point.
2017-03-14 20:07:33 -07:00
Andrey Samokhvalov
61991a1c89 lnd: fix latest goclean.sh lint warning 2017-03-13 16:30:23 -07:00
Andrey Samokhvalov
fd97a4bd19 lnd: partially fix golint warnings 2017-03-13 16:30:23 -07:00
Andrey Samokhvalov
f2843dd4c9 lnd: fix gofmt warnings 2017-03-13 16:30:23 -07:00
Olaoluwa Osuntokun
addab3273f
routing: add a set of test cases for client topology notifications 2017-03-08 14:46:18 -08:00
Olaoluwa Osuntokun
1a78c73f7e
routing: add new notificaiton client for topology changes
This commit adds some new functionality to the channel router: the
ability to dispatch notification to registered clients upon either a
channel being closed, a new node appearing, or an exiting client being
updated or opened for the first time.

With this change, the integration tests will now be able to eliminate
most of the sleep as we gain a new syntonization point into the
propagation of information within the test network. Additionally, this
also paves the way for client side software to dynamically visualize
the channel graph in real-time as nodes+channels are updated.
2017-03-08 14:46:14 -08:00
Olaoluwa Osuntokun
7bdf02bc9e
routing: modify path finding routines use new EdgeInfo/EdgePolicy
This commit modifies the path finding routines to properly use the new
channel edge related API exposed by the database. Additionally, a new
type `ChannelHop` has been introduced which couples an edges routing
policy with the capacity and origin chain of the channel.
2017-03-08 14:25:05 -08:00
Olaoluwa Osuntokun
2dfab8c6d7
routing+lnd: provide payment premiere as response to SendPayment 2017-02-21 01:43:48 -08:00
Olaoluwa Osuntokun
78561c3e35
routing: fix compile error in definition due to chainntfns API change 2017-02-21 01:43:01 -08:00
Olaoluwa Osuntokun
12f69692aa
multi: update sub-systems to use latest iteration of wire messages 2017-02-21 01:42:37 -08:00
Olaoluwa Osuntokun
5560f032f6
rpc+routing: fix panic when channel edge but no edge info in db 2017-02-16 19:35:32 +08:00
Olaoluwa Osuntokun
384fe61e73
multi: fix go vet warnings throughout code base 2017-02-16 19:33:19 +08:00
Olaoluwa Osuntokun
a094681dae
routing: only log pre-mature announcements if we have any 2017-02-07 18:38:48 -08:00
Olaoluwa Osuntokun
bf78122dc7
routing: ensure advertised transaction index is within bounds
This commit fixes bug that could result in the panicking or crashing of
nodes in the case of an at-funding-time reorganization within the
network. In order to avoid such a case, we now ensure that the
advertised transaction index is within the bounds of the block before
attempting to access it.

Note that this is a temporary patch commit until full advertisement
validation which is implemented in the discovery PR lands in master.
Additionally, better reorg handling during the funding process is being
specified within the spec and will properly be implemented within lnd
at a later date.
2017-02-06 14:54:57 -08:00
Olaoluwa Osuntokun
9cef2f8657
routing: add support for detection+processing of premature announcements
This commit fixes a prior block propagation race-condition by detecting
and properly processing “premature” announcements. A premature
announcement is one that’s received with an anchored block height which
is beyond our chain tip. Once received, we now store these
announcements in a special map that’s caches them in memory. Once a new
block arrives, we check the map for the existence of any entries,
processing them as normal if so.
2017-02-02 17:44:25 -08:00
Christopher Jämthagen
24a569b047 routing: add proper test for excessive number of hops 2017-02-02 00:24:16 -08:00
Christopher Jämthagen
4200d8a7e0 routing: check route length after edges for route is collected
This fixes the bug reproduced in #114. The prevHop map in newRoute may
include many more edges than what is used to produce the final route, and
thus the check prior to building the route could result with incorrect
errors being reported. We move this check to after the number of edges
to be used for the route is deduced.
2017-02-02 00:24:16 -08:00
Olaoluwa Osuntokun
08f0d0fbea
routing+rpcserver: move route querying+sending to SendPayment
This commit moves much of the logic for querying for a potential route,
constructing the HTLC including the Sphinx packet, and sending the
ultimate payment from the rpcServer to the ChannelRouter.

This movement paves the way for muilt-path path finding as well as
adding automatic retry logic to the ChannelRouter. Additionally, by
having the ChannelRouter construct the Sphinx packet, we’ll be able to
also include the proper time-lock and general per-hop-payload
information properly in the future.
2017-02-01 18:29:55 -08:00
Olaoluwa Osuntokun
db40b4322e
routing: rebroadcast our outgoing channels with a long period
This commit adds new behavior to the ChannelRouter struct: we know
rebroadcast our outgoing channels every 30 minutes. This new behavior
should ensure that both directions of an advertised channel edge are
always propagated though the network, fixing the issue of “ghost” edges
which exist but aren’t advertised.
2017-01-22 14:40:09 -08:00
Olaoluwa Osuntokun
8fd4d7ea6b
routing: fix bug in channel capacity display by ensuring properly typed arithmetic
This commit fixes a slight bug in the storage of the capacity of a
channel. Previously, we were subtracting a the hard coded fee amount
without first casting the integer to a btcutil.Amount which results in
a display/rounding error when the amount is converted to BTC.
2017-01-22 14:21:58 -08:00
Trevin Hofmann
40c7bac3aa multi: fix a variety of typos throughout the repo 2017-01-17 17:02:56 -08:00
Olaoluwa Osuntokun
7312565644
routing: allow full syncing graph state with partially advertised edges
This commit is similar to the prior commit to channeldb: we no longer
assume that _both_ edges of a channel will always be advertised. Such
an assumption resulted in the inability for a node to sync graph state
since we were previously returning an error when _both_ edges weren’t
found within the graph database.

To remedy this bug, we now carefully ensure that if one edge doesn’t
exist, then we still sync the other.
2017-01-17 13:18:05 -08:00
Olaoluwa Osuntokun
0c7fcb1755
routing: fix nil pointer panic when node has no outgoing channels 2017-01-17 13:07:05 -08:00
Trevin Hofmann
a13ac90d46 multi: add link to LICENSE in README license badges (#100) 2017-01-12 16:31:08 -08:00
Olaoluwa Osuntokun
4ccdad0d66
multi: add README's for all sub-packages 2017-01-10 15:02:37 -08:00
Olaoluwa Osuntokun
f1e23748c9
routing/testdata: ensure test channels have unique chan ID's
This commit fixes a bug in the test data that was uncovered due to the
recent bug fix within the AddChannelEdge method within the ChannelGraph
struct of channeldb.

The storage and assertion of unique channel ID’s wasn’t correct due to
bug in the channeldb which caused the defect in the test data to go
unnoticed.
2017-01-09 19:47:52 -08:00
Olaoluwa Osuntokun
ec0c7c5989
routing: fix panic in inner loop of path finding
This commit seems to fix a sporadic error within the integration tests
which would at times cause a panic when a payment as initiated.

This issue was with the way were deleting from the middle of the slice
of unvisited nodes within the graph. Assigning the last element to the
middle would at times cause a panic the last element may be nil. To fix
this, we now manually copy every item over by one, preserving the order
of the slice, and possibly fixing the panic once and for all.
2017-01-07 21:22:23 -08:00
Olaoluwa Osuntokun
5affed38fc
multi: update btcsuite API's to latest upstream changes
This commit makes a large number of minor changes concerning API usage
within the deamon to match the latest version on the upstream btcsuite
libraries.

The major changes are the switch from wire.ShaHash to chainhash.Hash,
and that wire.NewMsgTx() now takes a paramter indicating the version of
the transaction to be created.
2017-01-05 13:56:34 -08:00
Olaoluwa Osuntokun
82815b703e
rpcserver: refactor logic for ListPayments/DeleteAllPayments
This commit slightly refactors the logic for the new outgoing payment
related RPC’s to more closely match the style of the rest of the
codebase. Additionally the tests have been updated to reflect the
changes to the protos of the new RPC’s.
2016-12-30 16:42:10 -08:00
Olaoluwa Osuntokun
e327ffe954
routing: rewrite package to conform to BOLT07 and factor in fees+timelocks
This commit overhauls the routing package significantly to simplify the
code, conform to the rest of the coding style within the package, and
observe the new authenticated gossiping scheme outlined in BOLT07.

As a major step towards a more realistic path finding algorithm, fees
are properly calculated and observed during path finding. If a path has
sufficient capacity _before_ fees are applied, but afterwards the
finalized route would exceed the capacity of a single link, the route
is marked as invalid.

Currently a naive weighting algorithm is used which only factors in the
time-lock delta at each hop, thereby optimizing for the lowest time
lock. Fee calculation also isn’t finalized since we aren’t yet using
milli-satoshi throughout the daemon. The final TODO item within the PR
is to properly perform a multi-path search and rank the results based
on a summation heuristic rather than just return the first (out of
many) route found.

On the server side, once nodes are initially connected to the daemon,
our routing table will be synced with the peer’s using a naive “just
send everything scheme” to hold us over until I spec out some a
efficient graph reconciliation protocol. Additionally, the routing
table is now pruned by the channel router itself once new blocks arrive
rather than depending on peers to tell us when a channel flaps or is
closed.

Finally, the validation of peer announcements aren’t yet fully
implemented as they’ll be implemented within the pending discovery
package that was blocking on the completion of this package. Most off
the routing message processing will be moved out of this package and
into the discovery package where full validation will be carried out.
2016-12-27 16:44:22 -08:00
Olaoluwa Osuntokun
b49d2827a2
lnwallet/btcwallet: add compile-time interface assertions for Signer+BlockChainIO 2016-12-27 16:43:31 -08:00
Olaoluwa Osuntokun
ddc1eb4c8a
lnwallet: correct comment for logging object 2016-12-27 16:43:07 -08:00
BitfuryLightning
327768f4ad routing: Move tools inside lnd. Refactor and delete unneeded stuff
Use [33]byte for graph vertex representation.
Delete unneeded stuff:
1. DeepEqual for graph comparison
2. EdgePath
3. 2-thread BFS
4. Table transfer messages and neighborhood radius
5. Beacons

Refactor:
1. Change ID to Vertex
2. Test use table driven approach
3. Add comments
4. Make graph internal representation private
5. Use wire.OutPoint as  EdgeId
6. Decouple routing messages from routing implementation
7. Delete Async methods
8. Delete unneeded channels and priority buffer from manager
9. Delete unneeded interfaces in internal graph realisation
10. Renamed ID to Vertex
2016-11-23 20:37:43 -06:00