Commit Graph

413 Commits

Author SHA1 Message Date
Conner Fromknecht
59b459674d
multi: init subsystem loggers via build pkg 2018-10-05 13:04:45 +09:00
Olaoluwa Osuntokun
6afee3d099
Merge pull request #1856 from maurycy/typos
multi: fix various typos in comments
2018-09-27 20:38:10 -07:00
Olaoluwa Osuntokun
25145acc46
Merge pull request #1738 from joostjager/validation
payment failure message with attached channel_update not validated
2018-09-17 21:11:02 -07:00
Wilmer Paulino
2fa64dd6ff multi: update to new BitcoindClient API 2018-09-12 21:05:19 -07:00
Joost Jager
a7fec827cc
routing: validate channel update in failure message 2018-09-11 20:12:09 +02:00
Joost Jager
b5fd32ff77
discovery+routing: move validation logic to routing package
Previously, gossiper was the only object that validated channel
updates. Because updates can also be received as part of a
failed payment session in the routing package, validation logic
needs to be available there too. Gossiper already depends on
routing and having routing call the validation logic inside
gossiper would be a circular dependency. Therefore the validation
was moved to routing.
2018-09-11 20:12:07 +02:00
Joost Jager
ac8a3af84c
routing: add channel update validation test 2018-09-11 20:10:15 +02:00
Joost Jager
47d2e1e024
routing: add option to specify explicit id for test channels 2018-09-11 20:10:15 +02:00
Joost Jager
c5c580ab54
routing: add private key map to test graph instance 2018-09-11 20:10:14 +02:00
Joost Jager
5daf75b264
routing: restructure test context creation 2018-09-11 20:10:14 +02:00
Joost Jager
2d255e3bc3
routing: use fixed time value in tests 2018-09-11 20:10:14 +02:00
maurycy
ac24b12bf2
multi: fix various typos in comments 2018-09-07 06:51:49 +02:00
Conner Fromknecht
89113654fe
routing/conf: add experimental assume valid conf 2018-09-03 20:15:18 -07:00
Conner Fromknecht
1e473b2364
routing/router: add assume chan valid 2018-09-03 20:15:12 -07:00
Olaoluwa Osuntokun
85c0dd3de0
routing/chainview: update to latest API changes 2018-08-24 03:36:25 -07:00
Wilmer Paulino
191c4f3e28
multi: enable txindex on miner harness 2018-08-24 03:36:25 -07:00
Johan T. Halseth
4b04e1afd1
routing: return ErrRejected if found in reject cache
We make sure to return an error other than ErrIgnored, as ErrIgnored is
expected to only be returned for updates where we already have the
necessary information in the database.

In case of a channel ID found in the rejectCache, there was a
possibility that we had rejected an invalid update for this channel
earlier, and when attempting to add the current update we wouldn't
distinguish the failure to add from an outdated/ignored update.
2018-08-23 07:40:40 +02:00
Johan T. Halseth
18e680f192
router: simplify edge outdated logic, return ErrOutdated if old. 2018-08-23 07:40:39 +02:00
Wilmer Paulino
6c903393e5 router+server: use first hop channel id rather than node key 2018-08-20 21:04:52 -07:00
Olaoluwa Osuntokun
518b0d2cf7
Merge pull request #1679 from cfromknecht/move-linter-v2
Makefile: move linter v2
2018-08-09 21:03:05 -07:00
Olaoluwa Osuntokun
3438baaf0c
routing: eliminate nested db transactions by using new FetchOtherNode method 2018-08-09 20:48:06 -07:00
Joost Jager
29b6bae4ed routing: convert fee limit test to table-driven 2018-08-09 20:23:40 -07:00
Joost Jager
3e7473f4f0 routing: backward searching 2018-08-09 20:23:40 -07:00
Joost Jager
e52d829168 routing: path finding test refactored 2018-08-09 20:23:40 -07:00
Joost Jager
6c918a1806 channeldb: store unknown policies in database
The commit ensures that for every channel, there will always
be two entries in the edges bucket. If the policy from one or
both ends of the channel is unknown, it is marked as such.

This allows efficient lookup of incoming edges. This is
required for backwards payment path finding.
2018-08-09 20:23:40 -07:00
Wilmer Paulino
9d2eeb6304
multi: update to latest fee estimation interface 2018-08-09 17:29:52 -07:00
Wilmer Paulino
936fcc1f16
multi: update to latest BitcoindClient interface
In this commit, we introduce a nice optimization with regards to lnd's
interaction with a bitcoind backend. Within lnd, we currently have three
different subsystems responsible for watching the chain: chainntnfs,
lnwallet, and routing/chainview. Each of these subsystems has an active
RPC and ZMQ connection to the underlying bitcoind node. This would incur
a toll on the underlying bitcoind node and would cause us to miss ZMQ
events, which are crucial to lnd. We remedy this issue by sharing the
same connection to a bitcoind node between the different clients within
lnd.
2018-08-08 20:26:30 -07:00
Conner Fromknecht
a76b9a86cd
routing/chainview/interface_test: use EdgePoint field names 2018-08-07 16:57:40 -07:00
Conner Fromknecht
9eff876f52
routing/pathfind_test: fmt whitespace 2018-08-02 18:20:49 -07:00
Conner Fromknecht
614b3b3862
routing/pathfind: fmt whitespace 2018-08-02 18:20:49 -07:00
Conner Fromknecht
1ded697e8d
multi: sort import paths with gofmt 2018-08-02 18:20:49 -07:00
Olaoluwa Osuntokun
3c60861005
routing/chainview: give neutrino backend more time to catch up for re-org test
In this commit, we modify the test to explitlcy give the neutrino
backend more time to catch up compared to the RPC backends. We do this
as a recent change has been made in the neutrino backend to wait for the
filter headers to finish syncing before proceeding with the rescan
itself. As a result, we'll need to account for this in the test and
sleep enough to give the backend a chance to catch up.
2018-07-31 21:29:03 -07:00
Olaoluwa Osuntokun
8c0a151cec
routing: update UTXO verification to use new interface, graph pruning too 2018-07-31 21:28:55 -07:00
Olaoluwa Osuntokun
fb98f59407
routing/chainview: ensure bitcoind impl meets new interface requirements 2018-07-31 21:28:55 -07:00
Olaoluwa Osuntokun
9b9029cf4b
routing/chainview: ensure btcd impl meets new interface requirements 2018-07-31 21:28:55 -07:00
Olaoluwa Osuntokun
7e6997d6d3
routing/chainview: update neutrino API due to recent changes
In this commit, we ensure that the neutrino backend meets the target
interface, and also we update the API usage for the internal neutrino
rescan struct to use the new InputWithScript struct.
2018-07-31 21:28:54 -07:00
Olaoluwa Osuntokun
ec0fb9c537
routing/chainview: modify the UpdateFilter method to take a channeldb.EdgePoint
In this commit, we update the existing UpdateFilter method to take the
new channeldb.EdgePoint struct in place of the prior wire.OutPoint. We
must do this as the caller now typically has this type due to the
preparation to enable lnd to be able to be compatible with the new
neutrino protocol.
2018-07-31 21:28:54 -07:00
Olaoluwa Osuntokun
a6c814010c
routing: exit gracefully if generateSphinxPacket is passed a nil set of hops 2018-07-30 13:41:06 -07:00
Olaoluwa Osuntokun
c903a9a711
routing: add new TestEmptyRoutesGenerateSphinxPacket test 2018-07-30 13:40:25 -07:00
Olaoluwa Osuntokun
e0baa49690
Merge pull request #1387 from wpaulino/send-disable-chan-update
multi: send a channel update with disabled flag set on channel close
2018-07-23 17:46:37 -07:00
Olaoluwa Osuntokun
ef56f76200
routing: demote channel update logging to trace 2018-07-23 17:01:39 -07:00
Olaoluwa Osuntokun
2e75499787
channeldb+routing: move adding shell nodes into db txn of AddChannelEdge
In this commit, we fix a slight race condition that can occur when we go
to add a shell node for a node announcement, but then right afterwards,
a new block arrives that causes us to prune an unconnected node. To
ensure this doesn't happen, we now add shell nodes within the same db
transaction as AddChannelEdge. This ensures that the state is fully
consistent and shell nodes will be added atomically along with the new
channel edge.

As a result of this change, we no longer need to add shell nodes within
the ChannelRouter, as the database will take care of this operation as
it should.
2018-07-22 21:02:53 -07:00
Olaoluwa Osuntokun
f8cbe34e93
routing: prune nodes from the channel graph on start up 2018-07-21 19:52:26 -07:00
Olaoluwa Osuntokun
505544e1b3
routing/chainview: update API usage for neutrino due to recent changes 2018-07-13 17:15:12 -07:00
Olaoluwa Osuntokun
6f60f139f4 multi: switch over import paths from roasbeef/* to btcsuite/* 2018-07-13 17:05:39 -07:00
Wilmer Paulino
191e111661
routing: include disabled field in topology updates 2018-07-11 17:07:02 -07:00
Olaoluwa Osuntokun
16df90fc23
Merge pull request #1504 from cfromknecht/chainview-filter-locking
[routing/chainview] improve filter locking granularity in btcd/bitcoind
2018-07-10 17:19:36 -07:00
maurycy
3915c6a569 routing: do not reacquire the lock for topology notifications 2018-07-10 17:09:52 -07:00
Olaoluwa Osuntokun
e13f96e64b
routing: ensure we create a pruneViewSnapshot for route sessions
In this commit, we fix an existing bug that could at times lead to a
panic if a user manually crafts a route via SendToRoute, and that route
results in a payment error. The fix is simple: create the map even
though it won't be used in the sessions since the user is feeding the
router manual routes.
2018-07-08 16:34:25 -07:00
Conner Fromknecht
daa28e0d0f
routing/chainview/bitcoind: improve filterMtx locking
In this commit, we modify the granularity of the locking
around the filterMtx in the bitcoind chainview, such that
we only lock once per block connected or filter update.
Currently, we acquire and release the lock for every
update to the map.

We also fix a bug that would cause us to not fully remove
all previous outpoints spent by a txn when doing manual
filter, as we previously would only remove the first output
detected.
2018-07-06 19:08:17 -07:00
Conner Fromknecht
0f6bc3538d
routing/chainview/btcd: improve filterMtx locking
In this commit, we modify the granularity of the locking
around the filterMtx in the btcd chainview, such that we
only lock once per block connected or filter update.
Currently, we acquire and release the lock for every
update to the map.

We also fix a bug that would cause us to not fully remove
all previous outpoints spent by a txn when doing manual
filter, as we previously would only remove the first output
detected.
2018-07-06 19:08:17 -07:00
Olaoluwa Osuntokun
39c1f46f85
routing: new newLogClosure to delay spew evaluation until log print time
In this commit, we update the generateSphinxPacket to use newLogClosure
to delay the spew evaluation until log print time. Before this commit,
even if we weren't on the trace logging level, the spew call would
always be evaluated.
2018-06-30 17:14:28 -05:00
Olaoluwa Osuntokun
c045defd0f
Merge pull request #1358 from joostjager/newfee
routing: routing may come up with suboptimal routes (weight function)
2018-06-28 19:18:13 -07:00
Joost Jager
b52796749f routing: new weight function
In this commit, a new weight function is introduced. This will create a
meaningful effect of time lock on route selection. Also, removes the
squaring of the fee term. This led to suboptimal routes.

Unit test added that covers the weight function and asserts that the
lowest fee route is indeed returned.
2018-06-28 14:52:39 +02:00
Joost Jager
416f368979 routing: add time lock asserts to NewRoute test
This comment extends the unit tests for NewRoute with checks
on the total time lock for a route as well as the expected time
lock values for every hop along the route.
2018-06-28 14:04:02 +02:00
Joost Jager
d4471878df routing: fix route fee calculation and channel capacity check
This commit fixes the logic inside the newRoute function to
address the following problems:

- Fee calculation for a hop does not include the fee that needs
  to be paid to the next hop.

- The incoming channel capacity "sanity" check does not include
  the fee to be paid to the current hop.
2018-06-28 13:53:45 +02:00
Olaoluwa Osuntokun
6a6c29cd00
routing: fix incorrect expiry values in spec_example.json
In this commit, we fix the incorrect expiry values in the
spec_example.json test file. Many of the time locks were incorrect which
allowed bugs within the path finding logic related to CLTV deltas to go
un-detected.
2018-06-26 17:29:42 -07:00
Olaoluwa Osuntokun
6c986864ff
routing: fix bug in newRoute, use time lock delta of prior hop, not current
In this commit, we fix an existing bug in the newRoute method. Before
this commit we would use the time lock delta of the current hop to
compute the outgoing time lock for the current hop. This is incorrect as
the time lock delta of the _outgoing_ hop should be used, as this is
what we're paying for "transit" on. This is a bug left over from when we
switched the meaning of the CLTV delta on the ChannelUpdate message
sometime last year.

The fix is simple: use the CLTV delta of the prior (later in the route)
hop.
2018-06-26 17:29:42 -07:00
Wilmer Paulino
ddf8f2cb01
routing: allow specifying a fee limit during route construction 2018-06-12 18:14:48 -07:00
Wilmer Paulino
6a6de812ba
routing: denote a payment's fee limit in millisatoshis instead 2018-06-12 18:14:47 -07:00
Sebastian Delgado
6746609ec6
Add constant and percentage-based fee limits to payments
- Extend SendRequest and QueryRoutesRequest protos
- newRoute function takes fee limit and cuts off routes that exceed it
- queryRoutes, payInvoice and sendPayment commands take the feeLimit inputs and pass them down to newRoute
- When no feeLimit is included, don't enforce any feeLimits at all (by setting feeLimit to maxValue)
2018-06-12 18:14:43 -07:00
Olaoluwa Osuntokun
eb65b0cd5a
routing: check to see if pubkey is nil before logging in sendPayment
In this commit, we now need to check to see if the pubkey is nil as if
the user specified a set of manual routes, then the dest pubkey would be
empty.
2018-06-06 20:45:56 -07:00
Olaoluwa Osuntokun
0113035f6d
routing: add new NewPaymentSessionFromRoutes function instead of interface
In this commit, we modify the recent refactoring of the mission control
sub-system to overload the existing payment session, rather than create
a brand new one. This allows us to re-use more of the existing logic, and
also feedback into mission control the failures incurred by any user
selected routes.
2018-06-06 20:45:56 -07:00
t4sk
53eb396ae9 routing/router: add SendToRoute 2018-06-06 20:45:00 -07:00
maurycy
3be08e69cf multi: 64bit aligment of atomic vars on arm/x86-32 2018-06-04 20:02:34 -07:00
Olaoluwa Osuntokun
f494433cbf
routing+server: add new QueryBandwidth method to reduce outbound failures
In this commit, we introduce a new method to the channel router's config
struct: QueryBandwidth. This method allows the channel router to query
for the up-to-date available bandwidth of a particular link. In the case
that this link emanates from/to us, then we can query the switch to see
if the link is active (if not bandwidth is zero), and return the current
best estimate for the available bandwidth of the link. If the link,
isn't one of ours, then we can thread through the total maximal
capacity of the link.

In order to implement this, the missionControl struct will now query the
switch upon creation to obtain a fresh bandwidth snapshot. We take care
to do this in a distinct db transaction in order to now introduced a
circular waiting condition between the mutexes in bolt, and the channel
state machine.

The aim of this change is to reduce the number of unnecessary failures
during HTLC payment routing as we'll now skip any links that are
inactive, or just don't have enough bandwidth for the payment. Nodes
that have several hundred channels (all of which in various states of
activity and available bandwidth) should see a nice gain from this w.r.t
payment latency.
2018-05-14 16:23:54 -07:00
Conner Fromknecht
1b1f5f197a
routing/chainview/neutrino: cache filter entries
This commit alters the neutrino chainview such that it
caches the filter entries corresponding to watched
outpoints at the moment they are added to the filter.
Previously, we would rederive each filter entry when
reconstructing the relevant filter entries, which
would lead to unnecessary work on the gc. Now, each is
created at most once, and reused across subsequent
reconstructions.
2018-05-08 20:46:46 -07:00
Conner Fromknecht
99e7ec0895
routing/validation_barrier_test: test semaphore and quit
Adds test checking the basic functionality of the barrier's
semaphore and behavior during shutdown.
2018-05-08 16:32:08 -07:00
Conner Fromknecht
995e3fa85f
routing/validation_barrier: adds ErrVBarrierShuttingDown
Adds a new error ErrVBarrierShuttingDown that is returned
from WaitForDependants if the validation barrier's quit
chan is closed. This allows any blocked goroutines to
distinguish whether the dependent task has been completed,
or if validation should be aborted entirely.
2018-05-08 16:32:08 -07:00
Conner Fromknecht
eaa8cdf916
routing/router: improve validation barrier shutdown
This commit improves the shutdown of the router's
pending validation tasks, by ensuring the pending
tasks exit early if the validation barrier
receives a shutdown request.

Currently, any goroutines blocked by WaitForDependants
will continue execution after a shutdown is signaled.
This may lead to unnexpected behavior as the relation
between updates is no longer upheld. It also has the
side effect of slowing down shutdown, since we
continue to process the remaining updates.

To remedy this, WaitForDependants now returns an error
that signals if a shutdown was requested. The blocked
goroutines can exit early upon seeing this error,
without also signaling completion of their task to
the dependent tasks, which should will now properly
wait to read the validation barrier's quit signal.
2018-05-08 16:32:08 -07:00
Kenneth Perry
f7c5a7a19e routing/chainview/neutrino: error if we get a nil cfilter back (#1176) 2018-05-03 15:21:56 -07:00
Olaoluwa Osuntokun
76b9501e04
routing: update TestSendPaymentErrorPathPruning due to recent UnknownPeer change
In this commit, we update the TestSendPaymentErrorPathPruning test to
reflect the new behavior w.r.t how we respond to UnknownPeer errors. In
this new test, we expect that we'll find alternative route in light of
us getting an UnknownPeer error "pointing" to our destination node.
2018-04-23 17:53:46 -07:00
Olaoluwa Osuntokun
bd9f1b597e
routing: prune edges instead of vertexes in response to an FailUnknownNextPeer error
In this commit we fix an lingering bug in the Mission Control logic we
execute in response to the FailUnknownNextPeer error. Historically, we
would treat this as the _next_ node not being online. As a result, we
would then prune away the vertex from the current reachable graph all
together. It was recently realized, that this would at times be a bit
_tooo_ aggressive if the channel we attempt to route over was faulty,
down, or the incoming node had connectivity issues with the outgoing
node.

In light of this realization, we'll now instead only prune the _edge_
that we attempted to route over. This ensures that we'll continue to
explore the possible edges. Additionally, this guards us against failure
modes where nodes report FailUnknownNextPeer to other nodes in an
attempt to more closely control our retry logic.

This change is a stop gap on the path to a more intelligent set of
autopilot heuristics.

Fixes #1114.
2018-04-23 17:51:02 -07:00
Wilmer Paulino
beb08b3887
routing: add test for finding paths with additional edges 2018-04-20 04:01:33 -04:00
Wilmer Paulino
4598df054e
routing: modify findPath to take into account additional edges
In this commit, we modify our path finding algorithm to take an
additional set of edges that are currently not known to us that are
used to temporarily extend our graph with during a payment session.
These edges should assist the sender of a payment in successfully
constructing a path to the destination.

These edges should usually represent private channels, as they are not
publicly advertised to the network for routing.
2018-04-20 04:01:32 -04:00
Wilmer Paulino
7965247db1
routing: add additionalEdges field to a missionControl's paymentSession
In this commit, we introduce the ability for payment sessions to store
an additional set of edges that can be used to assist a payment in
successfully reaching its destination.
2018-04-20 04:01:31 -04:00
Wilmer Paulino
5ddee85479
routing: prevent logging the public key params within the routing hints 2018-04-20 04:01:30 -04:00
Wilmer Paulino
1e5659e5f4
routing: introduce routing hints to lightning payments
In this commit, we add a new field of routing hints to payments over the
Lightning Network. These routing hints can later be used within the path
finding algorithm in order to craft a path that will reach the
destination succesfully.
2018-04-20 04:01:29 -04:00
Dimitris Tsapakidis
4009f7f874 multi: fix typos in comments 2018-04-17 19:03:27 -07:00
practicalswift
663c396235 multi: fix a-vs-an typos 2018-04-17 19:02:04 -07:00
Olaoluwa Osuntokun
a6ffe999c6
routing: prune vertex, not ege after repeated FeeInsufficientErrors
In this commit, we modify the way we handle FeeInsufficientErrors to
more aggressively route around nodes that repeatedly return the same
error to us. This will ensure we skip older nodes on the network which
are running a buggier older version of lnd. Eventually most nodes will
upgrade to this new version, making this change less needed.

We also update the existing test to properly use a multi-hop route to
ensure that we route around the offending node.
2018-04-06 14:52:02 -07:00
Olaoluwa Osuntokun
1a05b48080
routing: add new test case to ensure we route around time lock failures 2018-03-23 17:55:16 -07:00
Olaoluwa Osuntokun
62b8ddb839
routing: update TestAddEdgeUnknownVertexes due to additional node in test graph 2018-03-23 17:55:15 -07:00
Olaoluwa Osuntokun
d5721165de
routing: fix TestPathInsufficientCapacity to actually send 1 BTC 2018-03-23 17:55:15 -07:00
Olaoluwa Osuntokun
de1150b78b
routing: add new vertex and to new channels to basic_graph.json
In this commit, we add a new node to the current default test graph
that we use for our path finding tests. This new node connects roasbeef
to sophon via a new route with very high fees. With this new node and
the two channels it adds, we can properly test that we’ll route around
failures that we run into during payment routing.
2018-03-23 17:55:15 -07:00
Olaoluwa Osuntokun
1365392d2b
routing: don't traverse links that advertise a timelock delta of zero 2018-03-23 17:55:14 -07:00
Olaoluwa Osuntokun
f935bd0baf
routing: add vertex pruning for non-final CLTV related errors
In this commit, we add vertex pruning for any non-final CLTV error.
Before this commit, we assumed that any source of this error was due to
the local node setting the incorrect time lock. However, it’s been
recently noticed on main net that there’re a set of nodes that seem to
not be properly scanned to the chain. Without this patch, users aren’t
able to route successfully as atm, we’ll stop all path finding attempts
if we encounter this.
2018-03-23 17:55:14 -07:00
Olaoluwa Osuntokun
635780858e
routing: add rejected spent outputs to the reject cache 2018-03-23 15:50:27 -07:00
Olaoluwa Osuntokun
93b04b39fe
routing: account for remaining routing onion errors in SendPayment
In this commit, we address a number of edge cases that were unaccounted
for when responding to errors that can be sent back due to an HTLC
routing failure. Namely:

   * We’ll no longer stop payment attempts if we’re unable to apply a
channel update, instead, we’ll log the error, prune the channel and
continue.
    * We’ll no remember which channels were pruned due to insufficient
fee errors. If we ever get a repeat fee error from a channel, then we
prune it. This ensure that we don’t get stuck in a loop due to a node
continually advertising the same fees.
   * We also correct an error in which node we’d prune due to a
temporary or permanent node failure. Before this commit, we would prune
the next node, when we should actually be pruning the node that sent us
the error.

Finally, we also add a new test to exercise the fee insufficient error
handling and channel pruning.

Fixes #865.
2018-03-21 15:23:48 -07:00
Olaoluwa Osuntokun
363cec7012
routing: add new PayAttemptTimeout field to LightningPayment
In this commit, we add a new field to the LightningPayment struct:
PayAttemptTimeout. This new field allows the caller to control exactly
how much time should be spent attempting to route a payment to the
destination. The default value we’ll use is 60 seconds, but callers are
able to specify a diff value. Once the timeout has passed, we’ll
abandon th e payment attempt, and return an error back to the original
caller.
2018-03-21 15:23:45 -07:00
Alex
948cb09c14 routing/chainview: bitcoind back-end now requires explicit NotifyBlocks() 2018-03-14 12:16:40 -06:00
Olaoluwa Osuntokun
800eea931f
build+multi: switch from bolt to bbolt
In this commit, we switch from boltbd/bolt to coreos/bbolt as the
former is no longer being actively maintained.
2018-03-10 19:01:13 -08:00
Olaoluwa Osuntokun
aa0410c90a
routing: add new methods to check the freshness of an edge/node
In this commit, we add a set of new methods to check the freshness of
an edge/node. This will allow callers to skip expensive validation in
the case that the router already knows of an item, or knows of a
fresher version of that time.

A set of tests have been added to ensure basic correctness of these new
methods.
2018-03-06 13:34:48 -05:00
t4sk
55f751fd37 routing/router Fix FindRoutes in router test
In router_test FindRoutes is passing DefaultFinalCLTVDelta in place
where numPaths is expected. This commit passes a default numPaths for
function calls to FindRoutes so that final cltv delta are correctly
passed.
2018-02-28 19:25:19 -08:00
Olaoluwa Osuntokun
7abdd30a87
routing: correct recent type change in heap_test.go 2018-02-12 16:44:42 -08:00
Olaoluwa Osuntokun
528aa67df7
routing: weight findPath more heavily towards fees during edge relaxation
In this commit, we modify the edgeWeight function that’s used within
the findPath method to weight fees more heavily than the time lock
value at an edge. We do this in order to greedily prefer lower fees
during path finding. This is a simple stop gap in place of more complex
weighting parameters that will be investigated later.

We also modify the edge distance to use an int64 rather than a float.
Finally an additional test has been added in order to excessive this
new change. Before the commit, the test was failing as we preferred the
route with lower total time lock.
2018-02-12 16:27:38 -08:00
Olaoluwa Osuntokun
ad0f5b31f6
routing: return cached route response in FindRoutes if enough routes
In this commit, we modify the caching structure to return a set of
cached routes for a request if the number of routes requested is less
than or equal to the number of cached of routes.
2018-02-12 16:22:28 -08:00
Olaoluwa Osuntokun
8a29c01354
routing: modify FindRoutes to take a max number of routes to return 2018-02-12 16:19:52 -08:00
Olaoluwa Osuntokun
33ba60ce66
routing: modify findPaths to take an upper limit on the number of routes
In this commit, we modify the findPaths method to take the max number
of routes to return. With this change, FindRoutes can eventually itself
also take a max number of routes in order to make the function useable
again.
2018-02-12 16:18:49 -08:00
Olaoluwa Osuntokun
b54b8dd7f1
routing: reject any new announcements which were pruned as zombies 2018-02-06 20:14:34 -08:00
Olaoluwa Osuntokun
6d05cb5aae
routing: extract zombie pruning to distinct method 2018-02-06 20:14:34 -08:00
Olaoluwa Osuntokun
1f3124f48a
routing: use [33]byte instead of *btcutil.Publickey for SendToSwitch
With this change, we can avoid unnecessarily serializing a public key.
2018-02-06 20:14:33 -08:00
Olaoluwa Osuntokun
6751cd8b9f
routing: update package to account for recent channeldb API changes 2018-02-06 20:14:32 -08:00
Daniel McNally
8543497dcc multi: fixing it's/its typos in comments 2018-02-06 19:13:07 -08:00
Alex Bosworth
bbc2ecd580 routing: typo fix
"we're closed" should be "were closed"
2018-02-06 19:12:10 -08:00
practicalswift
a93736d21e multi: comprehensive typo fixes across all packages 2018-02-06 19:11:11 -08:00
Olaoluwa Osuntokun
0a4de859a2
discovery+routing: reduce number of active validation barrier jobs
In order to reduce high CPU utilization during the initial network view
sync, we slash down the total number of active in-flight jobs that can
be launched.
2018-01-28 14:55:32 -08:00
Johan T. Halseth
b07f242dc2
routing: use multimutex.Mutex instead of package internal mutex 2018-01-24 10:26:40 +01:00
Olaoluwa Osuntokun
eba6ba0760
routing: properly prune edges that return a FailPermanentChannelFailure
In this commit, we now account for a case where a node sends us a
FailPermanentChannelFailure during a payment attempt. Before this
commit, we wouldn’t properly prune the edge to avoid re-using it. We
remedy this by properly attempting to prune the edge if possible.
Future changes well send a FailPermanentChannelFailure in the case that
we ned to go on-chain for an outgoing HTLC, and cancel back the
incoming HTLC.
2018-01-22 19:19:34 -08:00
Alex
187f59556a multi: add bitcoind drivers and tests 2018-01-15 13:59:34 -08:00
Johan T. Halseth
0b8e7ff836
routing: embed FeeSchema in ChannelPolicy
This commit embeds the FeeSchema within the new
struct ChannelPolicy, which also contains the
TimeLockDelta for a channel.
2018-01-12 22:56:31 +01:00
Olaoluwa Osuntokun
d70e4bb0a0
routing: account for case where final destination send TemporaryChannelFailure
In this commit, we fix an existing bug that could cause lnd to crash if
we sent a payment, and the *destination* sent a temp channel failure
error message. When handling such a message, we’ll look in the nextHop
map to see which channel was *after* the node that sent the payment.
However, if the destination sends this error, then there’ll be no entry
in this map.

To address this case, we now add a prevHop map. If we attempt to lookup
a node in the nextHop map, and they don’t have an entry, then we’ll
consult the prevHop map.

We also update the set of tests to ensure that we’re properly setting
both the prevHop map and the nextHop map.
2018-01-10 15:15:55 -08:00
Johan T. Halseth
0571918d3d
routing: process only single ChannelEdgePolicy per channelID concurrently
This commit adds synchronization around the processing
of multiple ChannelEdgePolicy updates for the same
channel ID at the same time.

This fixes a bug that could cause the database access
HasChannelEdge to be out of date when the goroutine
came to the point where it was calling UpdateEdgePolicy.
This happened because a second goroutine would have
called UpdateEdgePolicy in the meantime.

This bug was quite benign, as if this happened at
runtime, we would eventually get the ChannelEdgePolicy
we had lost again, either from a peer sending it to
us, or if we would fail a payment since we were using
outdated information. However, it would cause some of
the tests to flake, since losing routing information
made payments we expected to go through fail if this
happened.

This is fixed by introducing a new mutex type, that
when locking and unlocking takes an additional
(id uint64) parameter, keeping an internal map
tracking what ID's are currently locked and the
count of goroutines waiting for the mutex. This
ensure we can still process updates concurrently,
only avoiding updates with the same channel ID from
being run concurrently.
2018-01-09 13:09:29 +01:00
Olaoluwa Osuntokun
98f63cdce1
routing: add new paymentSession companion struct to missionControl
In this commit, we modify the pruning semantics of the missionControl
struct. Before this commit, on each payment attempt, we would fetch a
new graph pruned view each time. This served to instantly propagate any
detected failures to all outstanding payment attempts. However, this
meant that we could at times get stuck in a retry loop if sends take a
few second, then we may prune an edge, try another, then the original
edge is now unpruned.

To remedy this, we now introduce the concept of a paymentSession. The
session will start out as a snapshot of the latest graph prune view.
Any payment failures are now reported directly to the paymentSession
rather than missionControl. The rationale for this is that
edges/vertexes pruned as result of failures will never decay for a
local payment session, only for the global prune view. With this in
place, we ensure that our set of prune view only grows for a session.

Fixes #536.
2018-01-08 19:50:24 -08:00
Matt Drollette
adf0d98194 multi: fix several typos in godoc comments 2017-12-17 18:40:05 -08:00
Olaoluwa Osuntokun
7a3116f8a9
routing: ensure we set the TotalFees value on routes properly
Before this commit, we wouldn’t properly set the TotalFees attribute.
As a result, our sorting algorithm at the end to select candidate
routes would simply maintain the time-lock order rather than also sort
by total fees. This commit fixes this issue and also allows the test
added in the prior commit to pass.
2017-12-06 18:45:52 -08:00
Olaoluwa Osuntokun
cc3b32c0d4
routing: extend TestFindRoutesFeeSorting to assert proper fee attributes 2017-12-06 18:39:56 -08:00
Olaoluwa Osuntokun
455568279b
routing: ensure that new blocks connected monotonically increase in height
This commit fixes an existing bug within the ChannelRouter. Prior to
this commit, if the chain view skipped blocks or for some reason we had
a gap in blocks delivered, then we would simply accept them. This had
the potential to cause us to miss on-chain channel closure events. To
remedy this, we won’t process any blocks whose heights aren’t
*strictly* increasing.

A longer term fix would be to have the ChainView take a block height,
and re-dispatch any notifications from that height to the current
height.
2017-12-05 17:46:21 -08:00
Olaoluwa Osuntokun
ffd6b65ec1
routing: ensure access to r.bestHeight is thread-safe 2017-12-05 17:46:20 -08:00
Olaoluwa Osuntokun
a78ff8ba04
routing: during path finding skip edges that are marked disabled
In this commit, we implement adherence of the disabled bit within a
ChannelUpdate during path finding. If a channel is marked as disabled,
then we won’t attempt to route through it. A test has been added to
exercise this new check.
2017-11-30 22:31:27 -08:00
Olaoluwa Osuntokun
b61670fe23
routing: during path finding skip edge if amount < edge.MinHTLC
In this commit, we update path finding to skip an edge if the amount
we’re trying to route through it is below the MinHTLC (in mSAT) value
for that node. We also add a new test to exercise this behavior. In
order for out test to work properly, we’ve modified the JSON to make
the edge to Goku have a higher min HTLC value.
2017-11-30 22:29:45 -08:00
Olaoluwa Osuntokun
73ccb76853
routing: update ChannelUpdate handling to recognize lnwire.ChanUpdateFlag 2017-11-30 22:24:35 -08:00
Olaoluwa Osuntokun
7757721a92
routing: don't dispatch notification if error is non nil after processing ann 2017-11-29 17:53:09 -08:00
Olaoluwa Osuntokun
5bc9f07d12
routing: validate incoming announcements in parallel 2017-11-29 16:44:14 -08:00
Olaoluwa Osuntokun
fcd5e4aa41
routing: when updating chain view filter on restart, use best height as height hint
In this commit, we modify the high value passed into UpdateFilter upon
restart. Before this commit, we would pass in the prune height, which
would cause a full rescan within the FilteredChainView if the best
height as > than the prune height. This was redundant as we would
shortly carry out a manual rescan in the method below. To fix this, we
now pass in the bestHeight, this isn’t an issue as the
syncGraphWithChain method will manually scan up to that best height.
2017-11-29 16:26:39 -08:00
Olaoluwa Osuntokun
978023ab1d
routing: ensure notifyTopologyChange is theadsafe 2017-11-29 16:24:30 -08:00
Olaoluwa Osuntokun
33ce4e5689
routing: add new ValidationBarrier to allow for safe parallel validation of announcements
In this commit, we add a new abstraction, the ValidationBarrier. This
struct will be used to allow parallel validation of announcements
within notes AuthenticatedGossiper as well as the ChannelRouter.
Naively validating the announcement in parallel would run into issues
as it would be possible for validate an update announcement, before
validating the channel announcement itself. We solve this by creating a
waiting dependance using the ValidationBarrier to ensure that the
defendant jobs wait until their parents have been full validated.
2017-11-29 16:24:20 -08:00
Olaoluwa Osuntokun
d2c8f01768
routing/chainview: remove possibility of deadlock in shutdown of blockEventQueue 2017-11-16 16:01:21 -08:00
Olaoluwa Osuntokun
f189e2395a
routing: if the graph has never been pruned, prune with current height
In this commit we ensure that if this is the first time that the
ChannelRouter is starting, then we set the pruned height+hash to the
current best height. Otherwise, it’s possible that we attempt to update
the filter with a 0 prune height, which will restart a historical
rescan unnecessarily.
2017-11-15 18:19:39 -08:00
Olaoluwa Osuntokun
2b052cc889
routing: only update the filter if we have a non-zero chain view
In this commit we ensure that we only update the filter, if we have a
non-zero chain view. Otherwise, a mini rescan may be kicked off
unnecessarily  if we don’t yet know of any channels yet in the greater
graph.
2017-11-15 18:17:51 -08:00
Laura Cressman
7408aa6c8d config+pathfind_test+notifications_test: format and fix newVertex import
Run go fmt so config file is formatted correctly. Also rename
newVertex to NewVertex in pathfind_test and notifications_test
as it is now exported from the routing package.
2017-11-15 16:50:19 -08:00
Laura Cressman
39d38da732 discovery: create deDupedAnnouncements struct in gosspier.go
For Part 1 of Issue #275. Create isolated private struct in
networkHandler goroutine that will de-duplicate
announcements added to the batch. The struct contains maps
for each of channel announcements, channel updates, and
node announcements to keep track of unique announcements.

The struct has a Reset method to reset stored announcements, an
AddMsg(lnwire.Message) method to add a new message to the current
batch, and a Batch method to return the set of de-duplicated
announcements.

Also fix a few minor typos.
2017-11-15 16:50:19 -08:00
Conner Fromknecht
e62bb0319d
routing/chainview/btcd: makes rescan logging statement print to trace 2017-11-11 15:53:42 -08:00
Conner Fromknecht
153794adb0
routing/router: sets bestHeight before filtering chain
This commit alters the behavior of the router's logic on
 startup, ensuring that the chain view is filtered using
 the router's latest prune height. Before, the chain was
 filtered using the bestHeight variable, which was
 uninitialized, benignly forcing a rescan from genesis.
 In tracking down this, we realized that we should
 actually be using the prune height, as this is
 representative of the channel view loaded from disk.
 The best height/hash are now only used during
 startup to determine if we are out of sync.
2017-11-11 15:53:38 -08:00
Olaoluwa Osuntokun
5dbf5084e7
routing: fix linter error 2017-11-06 16:49:36 -08:00
Olaoluwa Osuntokun
9294358b5b
routing: populate the chain filter before calling syncGraphWithChain
In this commit we fix an existing bug within the ChannelRouter. Before
this commit, we would sync our graph prune state, *then* update the
cain filter. This is incorrect as the blocks we manually pruned may
have included channel closing transactions. As a result, we would miss
the pruning of a set of channels, and assume that they were still
active.

In this commit, we fix this by reversing the order: we first update the
chain filter and THEN sync the channel graph.
2017-11-06 16:10:24 -08:00
Olaoluwa Osuntokun
ccf94e9457
routing: add new TestRouterChansClosedOfflinePruneGraph test
In this commit we add a new test to the set of unit tests for the
ChannelRouter: TestRouterChansClosedOfflinePruneGraph. This tests that
if channels are closed while the ChannelRouter is down, then upon
restart the channels are properly recognized as being closed.
2017-11-06 16:06:15 -08:00
Olaoluwa Osuntokun
2824f45aad
routing: update tests to adhere to new newMockChainView API 2017-11-06 16:00:57 -08:00
Olaoluwa Osuntokun
3be905109c
routing: add RestartRouter method to testCtx 2017-11-06 16:00:22 -08:00
Olaoluwa Osuntokun
cdf2f43432
routing: add Reset() method to mockChainView
In this commit, we add a Reset() method to the mockChainView struct.
With this new method tests are able to fully simulate a restart of the
ChannelRouter. This is necessary as the FilteredChainView instances are
assumed to be stateless, and don’t write their state to disk before a
restart.
2017-11-06 15:58:57 -08:00
Olaoluwa Osuntokun
fea8cbf920
routing: add quit channel to mockChainView 2017-11-06 15:57:29 -08:00
Olaoluwa Osuntokun
4f32ee94ea
routing: implement FilterBlock for mockChainView 2017-11-06 15:56:31 -08:00
Olaoluwa Osuntokun
5f915280bc
routing/chainview: if column wrapping on comments in neutrino.go 2017-11-06 15:55:21 -08:00
Alex
1c408adcab routing/chainview: update neutrino driver to comply with BIP 2017-11-02 20:04:07 -07:00
Johan T. Halseth
256db86b02
routing: make channel router aware of stale blocks
This commit make the channel router handle the case where
it wakes up on a stale branch, and also the case where a
reorg happens while it is active.
2017-11-03 00:05:32 +01:00
Johan T. Halseth
8eb994c14b
routing/chainView test: add testFilterBlockDisconnected
This commit adds a test for the FilteredChainView interfaces,
making sure they notify about disconnected/connected blocks
in the correct order during a reorg.
2017-11-03 00:05:31 +01:00
Johan T. Halseth
6d15be5b79
routing/chainview: use blockEventQueue for neutrino block events
This commit makes use of the blockEventQueue within the neutrino
implementation of FilteredChainView to ensure connected and
disconnected blocks are consumed in order by the reader.

It also specifies that neutrino is not to send disconnected blocks
notifications during rescans, making it consistent with the btcd
implementation.
2017-11-03 00:05:30 +01:00
Johan T. Halseth
082f012fcf
routing/chainview: implement staleBlocks for btcd view.
This commit moves btcd view away from using the deprecated
callbacks onBlockConnected/Disconnected, and instead use
onFilteredBlockConnected/disconnected.

This commit also implements the sending of disconnected blocks
over the staleBlocks channel. To send these blocks, the
blockEventQueue is used to ensure the ordering of blocks are
correctly kept.

It also changes the way filter updates are handled. Since we
now load the tx filter to the rpc server itself, we can call
RescanBlocks instead of manually filtering blocks. These
rescanned blocks are also added to the blockEventQueue,
ensuring the ordering is kept.
2017-11-03 00:05:30 +01:00
Johan T. Halseth
e9c2f703d9
routing/chainview: additional comments to FilteredChainView godoc 2017-11-03 00:05:29 +01:00
Johan T. Halseth
a219aca0d3
routing/chainView: add blockEventQueue data structure
blockEventQueue is an ordered queue for block events sent from a
FilteredChainView. The two types of possible block events are
connected/new blocks, and disconencted/stale blocks. The
blockEventQueue keeps the order of these events intact, while
still being non-blocking. This is important in order for the
chainView's call to onFilteredBlockConnected/Disconnected to not
get blocked, and for the consumer of the block events to always
get the events in the correct order.
2017-11-03 00:05:28 +01:00