Commit Graph

587 Commits

Author SHA1 Message Date
Wilmer Paulino
7eb720e535
routing: mark policy as stale if edge remains double disabled
To ensure we don't mark an edge as live again just because an update
with a fresh timestamp was received, we'll ensure that we reject any
new updates for zombie channels if they remain disabled when running
with AssumeChannelValid.
2019-04-18 21:57:35 -07:00
Wilmer Paulino
f23c3b488e
routing: prune channels with disabled bit set on both edges
In this commit, we add an additional heuristic when running with
AssumeChannelValid. Since AssumeChannelValid being present assumes that
we're not able to quickly determine whether channels are valid, we also
assume that any channels with the disabled bit set on both sides are
considered zombie. This should be relatively safe to do, since the
disabled bits are usually set when the channel is closed on-chain. In
the case that they aren't, we'll have to wait until both edges haven't
had a new update within two weeks to prune them.
2019-04-18 21:57:34 -07:00
Wilmer Paulino
78bdcbb115
routing: prune channels only if both edges are present
We do this to ensure we don't prune too aggressively, as it's possible
that we've only received the channel announcement for a channel, but not
its accompanying channel updates.
2019-04-18 21:57:31 -07:00
Wilmer Paulino
29664c9704
routing: make test channel policies optional 2019-04-18 20:47:32 -07:00
Olaoluwa Osuntokun
373927fb3e
Merge pull request #2950 from joostjager/remove-route-cache
routing: remove route cache
2019-04-17 19:34:19 -07:00
Olaoluwa Osuntokun
a3c980b1e8
Merge pull request #2871 from halseth/router-subscribetopology-check-started
routing/notifications: check router started on SubscribeTopology
2019-04-16 21:32:10 -07:00
Johan T. Halseth
2b9efcbaeb
routing/router: define ErrRouterShuttingDown 2019-04-15 16:55:10 +02:00
Joost Jager
3b94627c12
routing: remove route cache
This commit removes the QueryRoutes route cache. It is causing wrong
routes to be returned because not all of the request parameters are
stored.

The cache allowed high frequency QueryRoutes calls to the same
destination and with the same amount to be returned fast. This behaviour
can also be achieved by caching the request on the client side. In case
a route is invalidated because of for example a channel update,
the subsequent SendToRoute call will fail. This is a trigger to call
QueryRoutes again for a fresh route.
2019-04-13 08:10:32 +02:00
Johan T. Halseth
c2af637d3c
routing/notifications: check router startd on SubscribeTopology
If the router is not yet started, return an error to avoid a deadlock
waiting for it to handle the subscription request.
2019-04-02 10:21:08 +02:00
Wilmer Paulino
44a01db0ef
routing: expose VerifyChannelUpdateSignature function 2019-03-27 13:07:47 -07:00
Wilmer Paulino
23796d3247
routing+discovery: extend ChannelGraphSource with zombie index methods 2019-03-27 13:07:30 -07:00
Wilmer Paulino
174645fcba
routing+server: expose DefaultChannelPruneExpiry 2019-03-27 13:07:13 -07:00
Wilmer Paulino
c82d73a826
channeldb+routing: extend edge lookup methods with zombie index check
In this commit, we extend the graph's FetchChannelEdgesByID and
HasChannelEdge methods to also check the zombie index whenever the edge
to be looked up doesn't exist within the edge index. We do this to
signal to callers that the edge is known, but only as a zombie, and the
only information that we have about the edge are the node public keys of
the two parties involved in the edge.

In the event that an edge does exist within the zombie index, we make
an additional check on edge policies to ensure they are not within the
router's pruning window, indicating that it is a fresh update.
2019-03-27 13:06:57 -07:00
Joost Jager
06cd64cbbc
routing: add cltv limit to payment session 2019-03-19 10:36:51 +01:00
Joost Jager
e3bb3d46a2
routing: abstract path finding interface 2019-03-19 10:36:48 +01:00
Joost Jager
6006549ed5
routing: enforce cltv limit in path finding 2019-03-19 10:36:45 +01:00
Joost Jager
c5961d4904
routing: only warn for zero cltv delta edges
This condition may be caused by a bug somewhere else in the system.
Expose it here as a warn log line.
2019-03-19 10:36:43 +01:00
Joost Jager
2be1051fb6
zpay32: move HopHint and DefaultFinalCLTVDelta 2019-03-15 10:08:32 +01:00
Olaoluwa Osuntokun
d86fe393cd
multi: update build to latest version of neutrino+btcwallet
In this commit, we update the build to point to the latest version of
neutrino and btcwallet. The latest version of neutrino includes a number
of bug fixes, and new features like reliably transaction broadcast. The
latest version of btcwallet contains a number of bug fixes related to
properly remove invalid transactions from its database.
2019-03-13 15:03:39 -07:00
Joost Jager
6cc82b4a34
routing: add todo describing route cache bug 2019-03-06 15:31:09 +01:00
Joost Jager
c62c9d64da
routing: add source parameter to query routes
This commit allows execution of QueryRoutes from any source node.
Previously this was restricted to only the self node.
2019-03-06 15:31:03 +01:00
Joost Jager
7719bc432f
routing: take Vertex types as path finding source and target nodes
Currently public keys are represented either as a 33-byte array (Vertex) or as a
btcec.PublicKey struct. The latter isn't useable as index into maps and
cannot be used easily in compares. Therefore the 33-byte array
representation is used predominantly throughout the code base.

This commit converts the argument types of source and target nodes for
path finding to Vertex. Path finding executes no crypto operations and
using Vertex simplifies the code.

Additionally, it prepares for the path finding source parameter to be
exposed over rpc in a follow up commit without requiring conversion back
and forth between Vertex and btcec.PublicKey.
2019-03-06 15:31:00 +01:00
Joost Jager
b09adc3219
lnrpc+routing: add edges and nodes restrictions to query routes
This commit allows the execution of QueryRoutes to be controlled using
lists of black-listed edges and nodes. Any path returned will not pass
through the edges and/or nodes on the list.
2019-03-06 15:30:57 +01:00
Joost Jager
4376f3e1bd
routing: allow nil maps for ignored edges and nodes
This allows removing a lot of empty map initialization code and makes
the code more readable.
2019-03-06 15:30:54 +01:00
Joost Jager
b2b28b49b1
routing: export RestrictParams and EdgeLocator 2019-03-06 15:30:52 +01:00
Joost Jager
4937304732
routing: remove redundant fee limit check in newRoute
This check was a left over from when the fee limit wasn't checked yet in
the path finding algorithm.
2019-03-06 15:30:49 +01:00
Joost Jager
f4cc2e235a
routing: add todo describing fee limit bug 2019-03-06 15:30:46 +01:00
Johan T. Halseth
cee18892b5
Merge pull request #2565 from joostjager/sendpayment-refactor
routing: sendPayment broken down into multiple functions
2019-03-06 10:20:20 +01:00
Valentine Wallace
648adaea69 routing/pathfind: ensure max htlc is considered during path finding
In this commit, we update the path finding logic to
ignore a channel if the HTLC value (including the fees
at the point) exceeds the max HTLC value (if set) of the
link.
2019-03-01 08:05:27 -08:00
Valentine Wallace
348a66ed13 routing: update test edge policy fields to include max htlc + new flags
Since the MaxHTLC field was recently added to the ChannelEdgePolicy struct,
and the Flags field was broken into ChannelFlags and MessageFlags, the
test edge policies should be updated accordingly.
2019-03-01 08:05:27 -08:00
Valentine Wallace
0c6c1040d8 routing/ntfns+rpcserver: include max htlc in topology notifications 2019-02-21 18:39:32 -08:00
Johan T. Halseth
01ea797d78
routing/router test: add TestIgnoreChannelEdgePolicyForUnknownChannel 2019-02-14 14:21:18 +01:00
Johan T. Halseth
23ce82f4e3
routing/router: don't apply channel policy updates for unknown channels 2019-02-14 14:21:18 +01:00
Johan T. Halseth
1259bacd49
routing: remove unused code, nodeIndex and chanIndex 2019-02-13 12:23:45 +01:00
Joost Jager
f8721ffbca
routing: sendPayment broken down into multiple functions 2019-02-11 08:42:41 +01:00
Alex Bosworth
b3ae5bc16e
trivial: typo fix 2019-02-10 21:18:34 -08:00
Joost Jager
7c30a8c493
routing: add outgoing channel restriction 2019-02-08 23:05:56 +01:00
Joost Jager
7b94d9adb0
routing: move payment session to separate file 2019-02-01 09:14:54 +01:00
Joost Jager
9e012ecc93
multi: move Input interface and related code
This commit is a step to split the lnwallet package. It puts the Input
interface and implementations in a separate package along with all their
dependencies from lnwallet.
2019-01-31 13:25:33 +01:00
orbitalturtle
08750f180b multi: deprecate IncorrectHtlcAmount onion error
In this commit, we deprecate the `IncorrectHtlcAmount` onion error.
We'll still decode this error to use when retrying paths, but we'll no
longer send this ourselves. The `UnknownPaymentHash` error has been
amended to also include the value of the payment as well. This allows us
to worry about one less error.
2019-01-29 18:21:13 -08:00
Johan T. Halseth
7d34ce9d08
lnwire+multi: define HasMaxHtlc helper on msgFlags 2019-01-22 08:42:30 +01:00
Valentine Wallace
cced754659
routing/router: when applying onion failure channel update, set max htlc
In this commit, we ensure that when we update an edge
as a result of a ChannelUpdate being returned from an
onion failure, the max htlc portion of the channel update
is included in the edge update.
2019-01-22 08:42:29 +01:00
Valentine Wallace
15168c391e
discovery+routing: validate msg flags and max htlc in ChannelUpdates
In this commit, we alter the ValidateChannelUpdateAnn function in
ann_validation to validate a remote ChannelUpdate's message flags
and max HTLC field. If the message flag is set but the max HTLC
field is not set or vice versa, the ChannelUpdate fails validation.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:27 +01:00
Valentine Wallace
0fd6004958
multi: partition lnwire.ChanUpdateFlag into ChannelFlags and MessageFlags
In this commit:

* we partition lnwire.ChanUpdateFlag into two (ChanUpdateChanFlags and
ChanUpdateMsgFlags), from a uint16 to a pair of uint8's

* we rename the ChannelUpdate.Flags to ChannelFlags and add an
additional MessageFlags field, which will be used to indicate the
presence of the optional field HtlcMaximumMsat within the ChannelUpdate.

* we partition ChannelEdgePolicy.Flags into message and channel flags.
This change corresponds to the partitioning of the ChannelUpdate's Flags
field into MessageFlags and ChannelFlags.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:26 +01:00
Conner Fromknecht
a5b22788bf
routing/router: avoid naked spew of channel policies 2019-01-09 23:54:41 -08:00
Olaoluwa Osuntokun
11c24d3e0f
Merge pull request #1745 from halseth/channeldb-avoid-create-buckets
channeldb: avoid creating empty buckets
2018-12-17 20:01:29 -08:00
Joost Jager
b2cb760c65
routing: check for empty hops list
This commit fixes a crash that could be triggered by sending an empty
hop list to the SendToRoute rpc.
2018-12-10 11:27:07 +01:00
Johan T. Halseth
81fe6e73ed
router+graph: return ErrGraphNodesNotFound if no nodes to prune
Avoids creating a bucket unneccessarily.
2018-12-06 10:22:15 +01:00
Johan T. Halseth
87df6b71c5
router test: only cleanup if context creation succeeds 2018-12-06 10:22:15 +01:00
Joost Jager
3ed16837e4
routing: prune channels unidirectional
In this commit we introduce pruning of channel edges instead of channels.
Channel failures apply to a single direction and it is unnecessarily
restricting to prune both directions.
2018-12-05 14:39:23 +01:00
Joost Jager
8b8e82a171
routing: remove route hop maps
Hop maps were used in a test to verify the population of the hop map
itself and further only in a single function (getFailedChannelID).
Rewrote that function and removed the hop maps completely.
2018-12-05 14:39:23 +01:00
Joost Jager
42debc6c12
routing: fix node ordering in test graph
There is the general assumption that channel edge policy nodes are
ordered such that the node1 pubkey is smaller than the key of node 2. In
the test graph, this assumption didn't hold. This commit fixes the test
graph and also adds a check to prevent this from happening again.
2018-12-05 09:15:27 +01:00
Johan T. Halseth
5f8425cb03
routing/pathfind_test: check error before attempting cleanup 2018-12-04 12:20:45 +01:00
Johan T. Halseth
67b018476c
routing/pathfind test: add TestPathSourceEdgesBandwidth
This commit adds a new test that checks that the bandwidth hints are
considered correclty for local channels, and that disable flags are
ignored in this case.
2018-12-04 12:20:45 +01:00
Johan T. Halseth
0b6a19866b
routing/pathfind: pass ignore disable flags for local channels
To decouple our own path finding from the graph state, we don't consider
the disable bit when attempting to use local channels. Instead the
bandwidth hints will be zero for local inactive channels.

We alos modify the unit test to check that the disable flag is ignored
for local edges.
2018-12-04 12:20:45 +01:00
Johan T. Halseth
20b9d5d000
routing: define graphParams and restrictParams
To avoid the findPath parameter list getting out of hand, we define new
structs that wraps the mandatory and optional parameters to findPath.
2018-12-04 12:20:44 +01:00
Olaoluwa Osuntokun
5075394617
Merge pull request #1706 from joostjager/errorprocessing
routing: make routing retry behaviour consistent
2018-12-03 18:29:07 -08:00
Joost Jager
b6ce03e569
routing: make routing retry behaviour consistent
Fixes the following issues:
- If the channel update of FailFeeInsufficient contains an invalid channel
  update, it is not possible to properly add to the failed channels set.
- FailAmountBelowMinimum may apply a channel update, but does not retry.
- FailIncorrectCltvExpiry immediately prunes the vertex without
  trying one more time.

In this commit, the logic for all three policy related errors is
aligned.
2018-12-03 13:23:27 +01:00
Olaoluwa Osuntokun
1fd3aac925
multi: switch from bolt packge to bbolt package for all imports 2018-11-29 20:33:49 -08:00
Joost Jager
6ba1144528
routing: move failed channels map into payment session
This is a small preparatory step towards moving mission control logic
out of router and reusing the acquired routing result data.
2018-11-29 10:31:38 +01:00
Joost Jager
ac04729cff
routing: use complete route in test
Previously not all route fields were properly populated. Example: prev
and next hop maps.
2018-11-29 10:31:38 +01:00
Joost Jager
71037969b7
routing: remove pruneVertexFailure function 2018-11-29 10:31:38 +01:00
Joost Jager
aca136a91c
routing: remove unused pruneVertexFailure parameters 2018-11-29 10:31:38 +01:00
Joost Jager
dd7e2e9e04
routing: move logging into applyChannelUpdate
To remove code duplicated at all call sites to check err and log.
2018-11-29 10:31:38 +01:00
Wilmer Paulino
4f45e339f1
discovery+routing: add FetchLightningNode to ChannelGraphSource interface 2018-11-11 17:48:05 -08:00
Wilmer Paulino
18ecb31983
routing/router: extend ChannelGraphSource interface with IsPublicNode
method
2018-10-24 17:28:54 -07:00
Olaoluwa Osuntokun
ccc1dffac4
routing: minor style clean ups 2018-10-23 17:16:39 -07:00
Olaoluwa Osuntokun
fbd91feace
Merge pull request #1888 from joostjager/routestruct
lnrpc+routing: fix unmarshallRoute and simplify route structure
2018-10-23 17:07:23 -07:00
Johan T. Halseth
29acb9f1f8
routing/chainview: new neutrino API 2018-10-16 19:28:29 -07:00
Joost Jager
1d97cf1229
htlcswitch+routing: implement expiry_too_far failure
In this commit we add a check to HtlcSatifiesPolicy to verify that the
time lock for the outgoing htlc that is requested in the onion packet
isn't too far in the future.

Without this check, anyone could force an unreasonably long time lock on
the forwarding node.
2018-10-15 08:51:08 +02:00
Joost Jager
a779004a18
lnrpc+routing: fix issues with missing data in unmarshallRoute
In this commit the dependency of unmarshallRoute on edge policies being
available is removed. Edge policies may be unknown and reported as nil.
SendToRoute does not need the policies, but it does need pubkeys of the
route hops. In this commit, unmarshallRoute is modified so that it
takes the pubkeys from edgeInfo instead of channelEdgePolicy.

In addition to this, the route structure is simplified. No more connection
to the database at that point. Fees are determined based on incoming and
outgoing amounts.
2018-10-13 22:59:23 +02:00
Johan T. Halseth
d8863bea8c
routing/chainview: demote logging of new utxos for filter to Trace
It was previously Debug, which would spam the logs at startup, and not
provide very useful (human-readable) information.
2018-10-10 15:00:50 +02:00
Olaoluwa Osuntokun
3fda1029c0
Merge pull request #1786 from Roasbeef/golang-1.1
build+docs: bump to golang 1.11
2018-10-08 13:26:07 +09:00
Olaoluwa Osuntokun
d8e6085c17
multi: update to go 1.11 gofmt 2018-10-07 14:37:38 +09:00
Conner Fromknecht
59b459674d
multi: init subsystem loggers via build pkg 2018-10-05 13:04:45 +09:00
Olaoluwa Osuntokun
6afee3d099
Merge pull request #1856 from maurycy/typos
multi: fix various typos in comments
2018-09-27 20:38:10 -07:00
Olaoluwa Osuntokun
25145acc46
Merge pull request #1738 from joostjager/validation
payment failure message with attached channel_update not validated
2018-09-17 21:11:02 -07:00
Wilmer Paulino
2fa64dd6ff multi: update to new BitcoindClient API 2018-09-12 21:05:19 -07:00
Joost Jager
a7fec827cc
routing: validate channel update in failure message 2018-09-11 20:12:09 +02:00
Joost Jager
b5fd32ff77
discovery+routing: move validation logic to routing package
Previously, gossiper was the only object that validated channel
updates. Because updates can also be received as part of a
failed payment session in the routing package, validation logic
needs to be available there too. Gossiper already depends on
routing and having routing call the validation logic inside
gossiper would be a circular dependency. Therefore the validation
was moved to routing.
2018-09-11 20:12:07 +02:00
Joost Jager
ac8a3af84c
routing: add channel update validation test 2018-09-11 20:10:15 +02:00
Joost Jager
47d2e1e024
routing: add option to specify explicit id for test channels 2018-09-11 20:10:15 +02:00
Joost Jager
c5c580ab54
routing: add private key map to test graph instance 2018-09-11 20:10:14 +02:00
Joost Jager
5daf75b264
routing: restructure test context creation 2018-09-11 20:10:14 +02:00
Joost Jager
2d255e3bc3
routing: use fixed time value in tests 2018-09-11 20:10:14 +02:00
maurycy
ac24b12bf2
multi: fix various typos in comments 2018-09-07 06:51:49 +02:00
Conner Fromknecht
89113654fe
routing/conf: add experimental assume valid conf 2018-09-03 20:15:18 -07:00
Conner Fromknecht
1e473b2364
routing/router: add assume chan valid 2018-09-03 20:15:12 -07:00
Olaoluwa Osuntokun
85c0dd3de0
routing/chainview: update to latest API changes 2018-08-24 03:36:25 -07:00
Wilmer Paulino
191c4f3e28
multi: enable txindex on miner harness 2018-08-24 03:36:25 -07:00
Johan T. Halseth
4b04e1afd1
routing: return ErrRejected if found in reject cache
We make sure to return an error other than ErrIgnored, as ErrIgnored is
expected to only be returned for updates where we already have the
necessary information in the database.

In case of a channel ID found in the rejectCache, there was a
possibility that we had rejected an invalid update for this channel
earlier, and when attempting to add the current update we wouldn't
distinguish the failure to add from an outdated/ignored update.
2018-08-23 07:40:40 +02:00
Johan T. Halseth
18e680f192
router: simplify edge outdated logic, return ErrOutdated if old. 2018-08-23 07:40:39 +02:00
Wilmer Paulino
6c903393e5 router+server: use first hop channel id rather than node key 2018-08-20 21:04:52 -07:00
Olaoluwa Osuntokun
518b0d2cf7
Merge pull request #1679 from cfromknecht/move-linter-v2
Makefile: move linter v2
2018-08-09 21:03:05 -07:00
Olaoluwa Osuntokun
3438baaf0c
routing: eliminate nested db transactions by using new FetchOtherNode method 2018-08-09 20:48:06 -07:00
Joost Jager
29b6bae4ed routing: convert fee limit test to table-driven 2018-08-09 20:23:40 -07:00
Joost Jager
3e7473f4f0 routing: backward searching 2018-08-09 20:23:40 -07:00
Joost Jager
e52d829168 routing: path finding test refactored 2018-08-09 20:23:40 -07:00
Joost Jager
6c918a1806 channeldb: store unknown policies in database
The commit ensures that for every channel, there will always
be two entries in the edges bucket. If the policy from one or
both ends of the channel is unknown, it is marked as such.

This allows efficient lookup of incoming edges. This is
required for backwards payment path finding.
2018-08-09 20:23:40 -07:00
Wilmer Paulino
9d2eeb6304
multi: update to latest fee estimation interface 2018-08-09 17:29:52 -07:00
Wilmer Paulino
936fcc1f16
multi: update to latest BitcoindClient interface
In this commit, we introduce a nice optimization with regards to lnd's
interaction with a bitcoind backend. Within lnd, we currently have three
different subsystems responsible for watching the chain: chainntnfs,
lnwallet, and routing/chainview. Each of these subsystems has an active
RPC and ZMQ connection to the underlying bitcoind node. This would incur
a toll on the underlying bitcoind node and would cause us to miss ZMQ
events, which are crucial to lnd. We remedy this issue by sharing the
same connection to a bitcoind node between the different clients within
lnd.
2018-08-08 20:26:30 -07:00
Conner Fromknecht
a76b9a86cd
routing/chainview/interface_test: use EdgePoint field names 2018-08-07 16:57:40 -07:00
Conner Fromknecht
9eff876f52
routing/pathfind_test: fmt whitespace 2018-08-02 18:20:49 -07:00
Conner Fromknecht
614b3b3862
routing/pathfind: fmt whitespace 2018-08-02 18:20:49 -07:00
Conner Fromknecht
1ded697e8d
multi: sort import paths with gofmt 2018-08-02 18:20:49 -07:00
Olaoluwa Osuntokun
3c60861005
routing/chainview: give neutrino backend more time to catch up for re-org test
In this commit, we modify the test to explitlcy give the neutrino
backend more time to catch up compared to the RPC backends. We do this
as a recent change has been made in the neutrino backend to wait for the
filter headers to finish syncing before proceeding with the rescan
itself. As a result, we'll need to account for this in the test and
sleep enough to give the backend a chance to catch up.
2018-07-31 21:29:03 -07:00
Olaoluwa Osuntokun
8c0a151cec
routing: update UTXO verification to use new interface, graph pruning too 2018-07-31 21:28:55 -07:00
Olaoluwa Osuntokun
fb98f59407
routing/chainview: ensure bitcoind impl meets new interface requirements 2018-07-31 21:28:55 -07:00
Olaoluwa Osuntokun
9b9029cf4b
routing/chainview: ensure btcd impl meets new interface requirements 2018-07-31 21:28:55 -07:00
Olaoluwa Osuntokun
7e6997d6d3
routing/chainview: update neutrino API due to recent changes
In this commit, we ensure that the neutrino backend meets the target
interface, and also we update the API usage for the internal neutrino
rescan struct to use the new InputWithScript struct.
2018-07-31 21:28:54 -07:00
Olaoluwa Osuntokun
ec0fb9c537
routing/chainview: modify the UpdateFilter method to take a channeldb.EdgePoint
In this commit, we update the existing UpdateFilter method to take the
new channeldb.EdgePoint struct in place of the prior wire.OutPoint. We
must do this as the caller now typically has this type due to the
preparation to enable lnd to be able to be compatible with the new
neutrino protocol.
2018-07-31 21:28:54 -07:00
Olaoluwa Osuntokun
a6c814010c
routing: exit gracefully if generateSphinxPacket is passed a nil set of hops 2018-07-30 13:41:06 -07:00
Olaoluwa Osuntokun
c903a9a711
routing: add new TestEmptyRoutesGenerateSphinxPacket test 2018-07-30 13:40:25 -07:00
Olaoluwa Osuntokun
e0baa49690
Merge pull request #1387 from wpaulino/send-disable-chan-update
multi: send a channel update with disabled flag set on channel close
2018-07-23 17:46:37 -07:00
Olaoluwa Osuntokun
ef56f76200
routing: demote channel update logging to trace 2018-07-23 17:01:39 -07:00
Olaoluwa Osuntokun
2e75499787
channeldb+routing: move adding shell nodes into db txn of AddChannelEdge
In this commit, we fix a slight race condition that can occur when we go
to add a shell node for a node announcement, but then right afterwards,
a new block arrives that causes us to prune an unconnected node. To
ensure this doesn't happen, we now add shell nodes within the same db
transaction as AddChannelEdge. This ensures that the state is fully
consistent and shell nodes will be added atomically along with the new
channel edge.

As a result of this change, we no longer need to add shell nodes within
the ChannelRouter, as the database will take care of this operation as
it should.
2018-07-22 21:02:53 -07:00
Olaoluwa Osuntokun
f8cbe34e93
routing: prune nodes from the channel graph on start up 2018-07-21 19:52:26 -07:00
Olaoluwa Osuntokun
505544e1b3
routing/chainview: update API usage for neutrino due to recent changes 2018-07-13 17:15:12 -07:00
Olaoluwa Osuntokun
6f60f139f4 multi: switch over import paths from roasbeef/* to btcsuite/* 2018-07-13 17:05:39 -07:00
Wilmer Paulino
191e111661
routing: include disabled field in topology updates 2018-07-11 17:07:02 -07:00
Olaoluwa Osuntokun
16df90fc23
Merge pull request #1504 from cfromknecht/chainview-filter-locking
[routing/chainview] improve filter locking granularity in btcd/bitcoind
2018-07-10 17:19:36 -07:00
maurycy
3915c6a569 routing: do not reacquire the lock for topology notifications 2018-07-10 17:09:52 -07:00
Olaoluwa Osuntokun
e13f96e64b
routing: ensure we create a pruneViewSnapshot for route sessions
In this commit, we fix an existing bug that could at times lead to a
panic if a user manually crafts a route via SendToRoute, and that route
results in a payment error. The fix is simple: create the map even
though it won't be used in the sessions since the user is feeding the
router manual routes.
2018-07-08 16:34:25 -07:00
Conner Fromknecht
daa28e0d0f
routing/chainview/bitcoind: improve filterMtx locking
In this commit, we modify the granularity of the locking
around the filterMtx in the bitcoind chainview, such that
we only lock once per block connected or filter update.
Currently, we acquire and release the lock for every
update to the map.

We also fix a bug that would cause us to not fully remove
all previous outpoints spent by a txn when doing manual
filter, as we previously would only remove the first output
detected.
2018-07-06 19:08:17 -07:00
Conner Fromknecht
0f6bc3538d
routing/chainview/btcd: improve filterMtx locking
In this commit, we modify the granularity of the locking
around the filterMtx in the btcd chainview, such that we
only lock once per block connected or filter update.
Currently, we acquire and release the lock for every
update to the map.

We also fix a bug that would cause us to not fully remove
all previous outpoints spent by a txn when doing manual
filter, as we previously would only remove the first output
detected.
2018-07-06 19:08:17 -07:00
Olaoluwa Osuntokun
39c1f46f85
routing: new newLogClosure to delay spew evaluation until log print time
In this commit, we update the generateSphinxPacket to use newLogClosure
to delay the spew evaluation until log print time. Before this commit,
even if we weren't on the trace logging level, the spew call would
always be evaluated.
2018-06-30 17:14:28 -05:00
Olaoluwa Osuntokun
c045defd0f
Merge pull request #1358 from joostjager/newfee
routing: routing may come up with suboptimal routes (weight function)
2018-06-28 19:18:13 -07:00
Joost Jager
b52796749f routing: new weight function
In this commit, a new weight function is introduced. This will create a
meaningful effect of time lock on route selection. Also, removes the
squaring of the fee term. This led to suboptimal routes.

Unit test added that covers the weight function and asserts that the
lowest fee route is indeed returned.
2018-06-28 14:52:39 +02:00
Joost Jager
416f368979 routing: add time lock asserts to NewRoute test
This comment extends the unit tests for NewRoute with checks
on the total time lock for a route as well as the expected time
lock values for every hop along the route.
2018-06-28 14:04:02 +02:00
Joost Jager
d4471878df routing: fix route fee calculation and channel capacity check
This commit fixes the logic inside the newRoute function to
address the following problems:

- Fee calculation for a hop does not include the fee that needs
  to be paid to the next hop.

- The incoming channel capacity "sanity" check does not include
  the fee to be paid to the current hop.
2018-06-28 13:53:45 +02:00
Olaoluwa Osuntokun
6a6c29cd00
routing: fix incorrect expiry values in spec_example.json
In this commit, we fix the incorrect expiry values in the
spec_example.json test file. Many of the time locks were incorrect which
allowed bugs within the path finding logic related to CLTV deltas to go
un-detected.
2018-06-26 17:29:42 -07:00
Olaoluwa Osuntokun
6c986864ff
routing: fix bug in newRoute, use time lock delta of prior hop, not current
In this commit, we fix an existing bug in the newRoute method. Before
this commit we would use the time lock delta of the current hop to
compute the outgoing time lock for the current hop. This is incorrect as
the time lock delta of the _outgoing_ hop should be used, as this is
what we're paying for "transit" on. This is a bug left over from when we
switched the meaning of the CLTV delta on the ChannelUpdate message
sometime last year.

The fix is simple: use the CLTV delta of the prior (later in the route)
hop.
2018-06-26 17:29:42 -07:00
Wilmer Paulino
ddf8f2cb01
routing: allow specifying a fee limit during route construction 2018-06-12 18:14:48 -07:00
Wilmer Paulino
6a6de812ba
routing: denote a payment's fee limit in millisatoshis instead 2018-06-12 18:14:47 -07:00
Sebastian Delgado
6746609ec6
Add constant and percentage-based fee limits to payments
- Extend SendRequest and QueryRoutesRequest protos
- newRoute function takes fee limit and cuts off routes that exceed it
- queryRoutes, payInvoice and sendPayment commands take the feeLimit inputs and pass them down to newRoute
- When no feeLimit is included, don't enforce any feeLimits at all (by setting feeLimit to maxValue)
2018-06-12 18:14:43 -07:00
Olaoluwa Osuntokun
eb65b0cd5a
routing: check to see if pubkey is nil before logging in sendPayment
In this commit, we now need to check to see if the pubkey is nil as if
the user specified a set of manual routes, then the dest pubkey would be
empty.
2018-06-06 20:45:56 -07:00
Olaoluwa Osuntokun
0113035f6d
routing: add new NewPaymentSessionFromRoutes function instead of interface
In this commit, we modify the recent refactoring of the mission control
sub-system to overload the existing payment session, rather than create
a brand new one. This allows us to re-use more of the existing logic, and
also feedback into mission control the failures incurred by any user
selected routes.
2018-06-06 20:45:56 -07:00
t4sk
53eb396ae9 routing/router: add SendToRoute 2018-06-06 20:45:00 -07:00
maurycy
3be08e69cf multi: 64bit aligment of atomic vars on arm/x86-32 2018-06-04 20:02:34 -07:00
Olaoluwa Osuntokun
f494433cbf
routing+server: add new QueryBandwidth method to reduce outbound failures
In this commit, we introduce a new method to the channel router's config
struct: QueryBandwidth. This method allows the channel router to query
for the up-to-date available bandwidth of a particular link. In the case
that this link emanates from/to us, then we can query the switch to see
if the link is active (if not bandwidth is zero), and return the current
best estimate for the available bandwidth of the link. If the link,
isn't one of ours, then we can thread through the total maximal
capacity of the link.

In order to implement this, the missionControl struct will now query the
switch upon creation to obtain a fresh bandwidth snapshot. We take care
to do this in a distinct db transaction in order to now introduced a
circular waiting condition between the mutexes in bolt, and the channel
state machine.

The aim of this change is to reduce the number of unnecessary failures
during HTLC payment routing as we'll now skip any links that are
inactive, or just don't have enough bandwidth for the payment. Nodes
that have several hundred channels (all of which in various states of
activity and available bandwidth) should see a nice gain from this w.r.t
payment latency.
2018-05-14 16:23:54 -07:00
Conner Fromknecht
1b1f5f197a
routing/chainview/neutrino: cache filter entries
This commit alters the neutrino chainview such that it
caches the filter entries corresponding to watched
outpoints at the moment they are added to the filter.
Previously, we would rederive each filter entry when
reconstructing the relevant filter entries, which
would lead to unnecessary work on the gc. Now, each is
created at most once, and reused across subsequent
reconstructions.
2018-05-08 20:46:46 -07:00
Conner Fromknecht
99e7ec0895
routing/validation_barrier_test: test semaphore and quit
Adds test checking the basic functionality of the barrier's
semaphore and behavior during shutdown.
2018-05-08 16:32:08 -07:00
Conner Fromknecht
995e3fa85f
routing/validation_barrier: adds ErrVBarrierShuttingDown
Adds a new error ErrVBarrierShuttingDown that is returned
from WaitForDependants if the validation barrier's quit
chan is closed. This allows any blocked goroutines to
distinguish whether the dependent task has been completed,
or if validation should be aborted entirely.
2018-05-08 16:32:08 -07:00
Conner Fromknecht
eaa8cdf916
routing/router: improve validation barrier shutdown
This commit improves the shutdown of the router's
pending validation tasks, by ensuring the pending
tasks exit early if the validation barrier
receives a shutdown request.

Currently, any goroutines blocked by WaitForDependants
will continue execution after a shutdown is signaled.
This may lead to unnexpected behavior as the relation
between updates is no longer upheld. It also has the
side effect of slowing down shutdown, since we
continue to process the remaining updates.

To remedy this, WaitForDependants now returns an error
that signals if a shutdown was requested. The blocked
goroutines can exit early upon seeing this error,
without also signaling completion of their task to
the dependent tasks, which should will now properly
wait to read the validation barrier's quit signal.
2018-05-08 16:32:08 -07:00
Kenneth Perry
f7c5a7a19e routing/chainview/neutrino: error if we get a nil cfilter back (#1176) 2018-05-03 15:21:56 -07:00
Olaoluwa Osuntokun
76b9501e04
routing: update TestSendPaymentErrorPathPruning due to recent UnknownPeer change
In this commit, we update the TestSendPaymentErrorPathPruning test to
reflect the new behavior w.r.t how we respond to UnknownPeer errors. In
this new test, we expect that we'll find alternative route in light of
us getting an UnknownPeer error "pointing" to our destination node.
2018-04-23 17:53:46 -07:00
Olaoluwa Osuntokun
bd9f1b597e
routing: prune edges instead of vertexes in response to an FailUnknownNextPeer error
In this commit we fix an lingering bug in the Mission Control logic we
execute in response to the FailUnknownNextPeer error. Historically, we
would treat this as the _next_ node not being online. As a result, we
would then prune away the vertex from the current reachable graph all
together. It was recently realized, that this would at times be a bit
_tooo_ aggressive if the channel we attempt to route over was faulty,
down, or the incoming node had connectivity issues with the outgoing
node.

In light of this realization, we'll now instead only prune the _edge_
that we attempted to route over. This ensures that we'll continue to
explore the possible edges. Additionally, this guards us against failure
modes where nodes report FailUnknownNextPeer to other nodes in an
attempt to more closely control our retry logic.

This change is a stop gap on the path to a more intelligent set of
autopilot heuristics.

Fixes #1114.
2018-04-23 17:51:02 -07:00
Wilmer Paulino
beb08b3887
routing: add test for finding paths with additional edges 2018-04-20 04:01:33 -04:00
Wilmer Paulino
4598df054e
routing: modify findPath to take into account additional edges
In this commit, we modify our path finding algorithm to take an
additional set of edges that are currently not known to us that are
used to temporarily extend our graph with during a payment session.
These edges should assist the sender of a payment in successfully
constructing a path to the destination.

These edges should usually represent private channels, as they are not
publicly advertised to the network for routing.
2018-04-20 04:01:32 -04:00
Wilmer Paulino
7965247db1
routing: add additionalEdges field to a missionControl's paymentSession
In this commit, we introduce the ability for payment sessions to store
an additional set of edges that can be used to assist a payment in
successfully reaching its destination.
2018-04-20 04:01:31 -04:00
Wilmer Paulino
5ddee85479
routing: prevent logging the public key params within the routing hints 2018-04-20 04:01:30 -04:00
Wilmer Paulino
1e5659e5f4
routing: introduce routing hints to lightning payments
In this commit, we add a new field of routing hints to payments over the
Lightning Network. These routing hints can later be used within the path
finding algorithm in order to craft a path that will reach the
destination succesfully.
2018-04-20 04:01:29 -04:00
Dimitris Tsapakidis
4009f7f874 multi: fix typos in comments 2018-04-17 19:03:27 -07:00
practicalswift
663c396235 multi: fix a-vs-an typos 2018-04-17 19:02:04 -07:00
Olaoluwa Osuntokun
a6ffe999c6
routing: prune vertex, not ege after repeated FeeInsufficientErrors
In this commit, we modify the way we handle FeeInsufficientErrors to
more aggressively route around nodes that repeatedly return the same
error to us. This will ensure we skip older nodes on the network which
are running a buggier older version of lnd. Eventually most nodes will
upgrade to this new version, making this change less needed.

We also update the existing test to properly use a multi-hop route to
ensure that we route around the offending node.
2018-04-06 14:52:02 -07:00
Olaoluwa Osuntokun
1a05b48080
routing: add new test case to ensure we route around time lock failures 2018-03-23 17:55:16 -07:00
Olaoluwa Osuntokun
62b8ddb839
routing: update TestAddEdgeUnknownVertexes due to additional node in test graph 2018-03-23 17:55:15 -07:00
Olaoluwa Osuntokun
d5721165de
routing: fix TestPathInsufficientCapacity to actually send 1 BTC 2018-03-23 17:55:15 -07:00
Olaoluwa Osuntokun
de1150b78b
routing: add new vertex and to new channels to basic_graph.json
In this commit, we add a new node to the current default test graph
that we use for our path finding tests. This new node connects roasbeef
to sophon via a new route with very high fees. With this new node and
the two channels it adds, we can properly test that we’ll route around
failures that we run into during payment routing.
2018-03-23 17:55:15 -07:00
Olaoluwa Osuntokun
1365392d2b
routing: don't traverse links that advertise a timelock delta of zero 2018-03-23 17:55:14 -07:00
Olaoluwa Osuntokun
f935bd0baf
routing: add vertex pruning for non-final CLTV related errors
In this commit, we add vertex pruning for any non-final CLTV error.
Before this commit, we assumed that any source of this error was due to
the local node setting the incorrect time lock. However, it’s been
recently noticed on main net that there’re a set of nodes that seem to
not be properly scanned to the chain. Without this patch, users aren’t
able to route successfully as atm, we’ll stop all path finding attempts
if we encounter this.
2018-03-23 17:55:14 -07:00
Olaoluwa Osuntokun
635780858e
routing: add rejected spent outputs to the reject cache 2018-03-23 15:50:27 -07:00
Olaoluwa Osuntokun
93b04b39fe
routing: account for remaining routing onion errors in SendPayment
In this commit, we address a number of edge cases that were unaccounted
for when responding to errors that can be sent back due to an HTLC
routing failure. Namely:

   * We’ll no longer stop payment attempts if we’re unable to apply a
channel update, instead, we’ll log the error, prune the channel and
continue.
    * We’ll no remember which channels were pruned due to insufficient
fee errors. If we ever get a repeat fee error from a channel, then we
prune it. This ensure that we don’t get stuck in a loop due to a node
continually advertising the same fees.
   * We also correct an error in which node we’d prune due to a
temporary or permanent node failure. Before this commit, we would prune
the next node, when we should actually be pruning the node that sent us
the error.

Finally, we also add a new test to exercise the fee insufficient error
handling and channel pruning.

Fixes #865.
2018-03-21 15:23:48 -07:00
Olaoluwa Osuntokun
363cec7012
routing: add new PayAttemptTimeout field to LightningPayment
In this commit, we add a new field to the LightningPayment struct:
PayAttemptTimeout. This new field allows the caller to control exactly
how much time should be spent attempting to route a payment to the
destination. The default value we’ll use is 60 seconds, but callers are
able to specify a diff value. Once the timeout has passed, we’ll
abandon th e payment attempt, and return an error back to the original
caller.
2018-03-21 15:23:45 -07:00
Alex
948cb09c14 routing/chainview: bitcoind back-end now requires explicit NotifyBlocks() 2018-03-14 12:16:40 -06:00
Olaoluwa Osuntokun
800eea931f
build+multi: switch from bolt to bbolt
In this commit, we switch from boltbd/bolt to coreos/bbolt as the
former is no longer being actively maintained.
2018-03-10 19:01:13 -08:00
Olaoluwa Osuntokun
aa0410c90a
routing: add new methods to check the freshness of an edge/node
In this commit, we add a set of new methods to check the freshness of
an edge/node. This will allow callers to skip expensive validation in
the case that the router already knows of an item, or knows of a
fresher version of that time.

A set of tests have been added to ensure basic correctness of these new
methods.
2018-03-06 13:34:48 -05:00
t4sk
55f751fd37 routing/router Fix FindRoutes in router test
In router_test FindRoutes is passing DefaultFinalCLTVDelta in place
where numPaths is expected. This commit passes a default numPaths for
function calls to FindRoutes so that final cltv delta are correctly
passed.
2018-02-28 19:25:19 -08:00
Olaoluwa Osuntokun
7abdd30a87
routing: correct recent type change in heap_test.go 2018-02-12 16:44:42 -08:00
Olaoluwa Osuntokun
528aa67df7
routing: weight findPath more heavily towards fees during edge relaxation
In this commit, we modify the edgeWeight function that’s used within
the findPath method to weight fees more heavily than the time lock
value at an edge. We do this in order to greedily prefer lower fees
during path finding. This is a simple stop gap in place of more complex
weighting parameters that will be investigated later.

We also modify the edge distance to use an int64 rather than a float.
Finally an additional test has been added in order to excessive this
new change. Before the commit, the test was failing as we preferred the
route with lower total time lock.
2018-02-12 16:27:38 -08:00
Olaoluwa Osuntokun
ad0f5b31f6
routing: return cached route response in FindRoutes if enough routes
In this commit, we modify the caching structure to return a set of
cached routes for a request if the number of routes requested is less
than or equal to the number of cached of routes.
2018-02-12 16:22:28 -08:00
Olaoluwa Osuntokun
8a29c01354
routing: modify FindRoutes to take a max number of routes to return 2018-02-12 16:19:52 -08:00
Olaoluwa Osuntokun
33ba60ce66
routing: modify findPaths to take an upper limit on the number of routes
In this commit, we modify the findPaths method to take the max number
of routes to return. With this change, FindRoutes can eventually itself
also take a max number of routes in order to make the function useable
again.
2018-02-12 16:18:49 -08:00
Olaoluwa Osuntokun
b54b8dd7f1
routing: reject any new announcements which were pruned as zombies 2018-02-06 20:14:34 -08:00
Olaoluwa Osuntokun
6d05cb5aae
routing: extract zombie pruning to distinct method 2018-02-06 20:14:34 -08:00
Olaoluwa Osuntokun
1f3124f48a
routing: use [33]byte instead of *btcutil.Publickey for SendToSwitch
With this change, we can avoid unnecessarily serializing a public key.
2018-02-06 20:14:33 -08:00
Olaoluwa Osuntokun
6751cd8b9f
routing: update package to account for recent channeldb API changes 2018-02-06 20:14:32 -08:00
Daniel McNally
8543497dcc multi: fixing it's/its typos in comments 2018-02-06 19:13:07 -08:00
Alex Bosworth
bbc2ecd580 routing: typo fix
"we're closed" should be "were closed"
2018-02-06 19:12:10 -08:00
practicalswift
a93736d21e multi: comprehensive typo fixes across all packages 2018-02-06 19:11:11 -08:00
Olaoluwa Osuntokun
0a4de859a2
discovery+routing: reduce number of active validation barrier jobs
In order to reduce high CPU utilization during the initial network view
sync, we slash down the total number of active in-flight jobs that can
be launched.
2018-01-28 14:55:32 -08:00
Johan T. Halseth
b07f242dc2
routing: use multimutex.Mutex instead of package internal mutex 2018-01-24 10:26:40 +01:00
Olaoluwa Osuntokun
eba6ba0760
routing: properly prune edges that return a FailPermanentChannelFailure
In this commit, we now account for a case where a node sends us a
FailPermanentChannelFailure during a payment attempt. Before this
commit, we wouldn’t properly prune the edge to avoid re-using it. We
remedy this by properly attempting to prune the edge if possible.
Future changes well send a FailPermanentChannelFailure in the case that
we ned to go on-chain for an outgoing HTLC, and cancel back the
incoming HTLC.
2018-01-22 19:19:34 -08:00
Alex
187f59556a multi: add bitcoind drivers and tests 2018-01-15 13:59:34 -08:00
Johan T. Halseth
0b8e7ff836
routing: embed FeeSchema in ChannelPolicy
This commit embeds the FeeSchema within the new
struct ChannelPolicy, which also contains the
TimeLockDelta for a channel.
2018-01-12 22:56:31 +01:00
Olaoluwa Osuntokun
d70e4bb0a0
routing: account for case where final destination send TemporaryChannelFailure
In this commit, we fix an existing bug that could cause lnd to crash if
we sent a payment, and the *destination* sent a temp channel failure
error message. When handling such a message, we’ll look in the nextHop
map to see which channel was *after* the node that sent the payment.
However, if the destination sends this error, then there’ll be no entry
in this map.

To address this case, we now add a prevHop map. If we attempt to lookup
a node in the nextHop map, and they don’t have an entry, then we’ll
consult the prevHop map.

We also update the set of tests to ensure that we’re properly setting
both the prevHop map and the nextHop map.
2018-01-10 15:15:55 -08:00
Johan T. Halseth
0571918d3d
routing: process only single ChannelEdgePolicy per channelID concurrently
This commit adds synchronization around the processing
of multiple ChannelEdgePolicy updates for the same
channel ID at the same time.

This fixes a bug that could cause the database access
HasChannelEdge to be out of date when the goroutine
came to the point where it was calling UpdateEdgePolicy.
This happened because a second goroutine would have
called UpdateEdgePolicy in the meantime.

This bug was quite benign, as if this happened at
runtime, we would eventually get the ChannelEdgePolicy
we had lost again, either from a peer sending it to
us, or if we would fail a payment since we were using
outdated information. However, it would cause some of
the tests to flake, since losing routing information
made payments we expected to go through fail if this
happened.

This is fixed by introducing a new mutex type, that
when locking and unlocking takes an additional
(id uint64) parameter, keeping an internal map
tracking what ID's are currently locked and the
count of goroutines waiting for the mutex. This
ensure we can still process updates concurrently,
only avoiding updates with the same channel ID from
being run concurrently.
2018-01-09 13:09:29 +01:00
Olaoluwa Osuntokun
98f63cdce1
routing: add new paymentSession companion struct to missionControl
In this commit, we modify the pruning semantics of the missionControl
struct. Before this commit, on each payment attempt, we would fetch a
new graph pruned view each time. This served to instantly propagate any
detected failures to all outstanding payment attempts. However, this
meant that we could at times get stuck in a retry loop if sends take a
few second, then we may prune an edge, try another, then the original
edge is now unpruned.

To remedy this, we now introduce the concept of a paymentSession. The
session will start out as a snapshot of the latest graph prune view.
Any payment failures are now reported directly to the paymentSession
rather than missionControl. The rationale for this is that
edges/vertexes pruned as result of failures will never decay for a
local payment session, only for the global prune view. With this in
place, we ensure that our set of prune view only grows for a session.

Fixes #536.
2018-01-08 19:50:24 -08:00
Matt Drollette
adf0d98194 multi: fix several typos in godoc comments 2017-12-17 18:40:05 -08:00
Olaoluwa Osuntokun
7a3116f8a9
routing: ensure we set the TotalFees value on routes properly
Before this commit, we wouldn’t properly set the TotalFees attribute.
As a result, our sorting algorithm at the end to select candidate
routes would simply maintain the time-lock order rather than also sort
by total fees. This commit fixes this issue and also allows the test
added in the prior commit to pass.
2017-12-06 18:45:52 -08:00
Olaoluwa Osuntokun
cc3b32c0d4
routing: extend TestFindRoutesFeeSorting to assert proper fee attributes 2017-12-06 18:39:56 -08:00
Olaoluwa Osuntokun
455568279b
routing: ensure that new blocks connected monotonically increase in height
This commit fixes an existing bug within the ChannelRouter. Prior to
this commit, if the chain view skipped blocks or for some reason we had
a gap in blocks delivered, then we would simply accept them. This had
the potential to cause us to miss on-chain channel closure events. To
remedy this, we won’t process any blocks whose heights aren’t
*strictly* increasing.

A longer term fix would be to have the ChainView take a block height,
and re-dispatch any notifications from that height to the current
height.
2017-12-05 17:46:21 -08:00
Olaoluwa Osuntokun
ffd6b65ec1
routing: ensure access to r.bestHeight is thread-safe 2017-12-05 17:46:20 -08:00
Olaoluwa Osuntokun
a78ff8ba04
routing: during path finding skip edges that are marked disabled
In this commit, we implement adherence of the disabled bit within a
ChannelUpdate during path finding. If a channel is marked as disabled,
then we won’t attempt to route through it. A test has been added to
exercise this new check.
2017-11-30 22:31:27 -08:00
Olaoluwa Osuntokun
b61670fe23
routing: during path finding skip edge if amount < edge.MinHTLC
In this commit, we update path finding to skip an edge if the amount
we’re trying to route through it is below the MinHTLC (in mSAT) value
for that node. We also add a new test to exercise this behavior. In
order for out test to work properly, we’ve modified the JSON to make
the edge to Goku have a higher min HTLC value.
2017-11-30 22:29:45 -08:00
Olaoluwa Osuntokun
73ccb76853
routing: update ChannelUpdate handling to recognize lnwire.ChanUpdateFlag 2017-11-30 22:24:35 -08:00
Olaoluwa Osuntokun
7757721a92
routing: don't dispatch notification if error is non nil after processing ann 2017-11-29 17:53:09 -08:00
Olaoluwa Osuntokun
5bc9f07d12
routing: validate incoming announcements in parallel 2017-11-29 16:44:14 -08:00
Olaoluwa Osuntokun
fcd5e4aa41
routing: when updating chain view filter on restart, use best height as height hint
In this commit, we modify the high value passed into UpdateFilter upon
restart. Before this commit, we would pass in the prune height, which
would cause a full rescan within the FilteredChainView if the best
height as > than the prune height. This was redundant as we would
shortly carry out a manual rescan in the method below. To fix this, we
now pass in the bestHeight, this isn’t an issue as the
syncGraphWithChain method will manually scan up to that best height.
2017-11-29 16:26:39 -08:00
Olaoluwa Osuntokun
978023ab1d
routing: ensure notifyTopologyChange is theadsafe 2017-11-29 16:24:30 -08:00
Olaoluwa Osuntokun
33ce4e5689
routing: add new ValidationBarrier to allow for safe parallel validation of announcements
In this commit, we add a new abstraction, the ValidationBarrier. This
struct will be used to allow parallel validation of announcements
within notes AuthenticatedGossiper as well as the ChannelRouter.
Naively validating the announcement in parallel would run into issues
as it would be possible for validate an update announcement, before
validating the channel announcement itself. We solve this by creating a
waiting dependance using the ValidationBarrier to ensure that the
defendant jobs wait until their parents have been full validated.
2017-11-29 16:24:20 -08:00
Olaoluwa Osuntokun
d2c8f01768
routing/chainview: remove possibility of deadlock in shutdown of blockEventQueue 2017-11-16 16:01:21 -08:00
Olaoluwa Osuntokun
f189e2395a
routing: if the graph has never been pruned, prune with current height
In this commit we ensure that if this is the first time that the
ChannelRouter is starting, then we set the pruned height+hash to the
current best height. Otherwise, it’s possible that we attempt to update
the filter with a 0 prune height, which will restart a historical
rescan unnecessarily.
2017-11-15 18:19:39 -08:00
Olaoluwa Osuntokun
2b052cc889
routing: only update the filter if we have a non-zero chain view
In this commit we ensure that we only update the filter, if we have a
non-zero chain view. Otherwise, a mini rescan may be kicked off
unnecessarily  if we don’t yet know of any channels yet in the greater
graph.
2017-11-15 18:17:51 -08:00
Laura Cressman
7408aa6c8d config+pathfind_test+notifications_test: format and fix newVertex import
Run go fmt so config file is formatted correctly. Also rename
newVertex to NewVertex in pathfind_test and notifications_test
as it is now exported from the routing package.
2017-11-15 16:50:19 -08:00
Laura Cressman
39d38da732 discovery: create deDupedAnnouncements struct in gosspier.go
For Part 1 of Issue #275. Create isolated private struct in
networkHandler goroutine that will de-duplicate
announcements added to the batch. The struct contains maps
for each of channel announcements, channel updates, and
node announcements to keep track of unique announcements.

The struct has a Reset method to reset stored announcements, an
AddMsg(lnwire.Message) method to add a new message to the current
batch, and a Batch method to return the set of de-duplicated
announcements.

Also fix a few minor typos.
2017-11-15 16:50:19 -08:00
Conner Fromknecht
e62bb0319d
routing/chainview/btcd: makes rescan logging statement print to trace 2017-11-11 15:53:42 -08:00
Conner Fromknecht
153794adb0
routing/router: sets bestHeight before filtering chain
This commit alters the behavior of the router's logic on
 startup, ensuring that the chain view is filtered using
 the router's latest prune height. Before, the chain was
 filtered using the bestHeight variable, which was
 uninitialized, benignly forcing a rescan from genesis.
 In tracking down this, we realized that we should
 actually be using the prune height, as this is
 representative of the channel view loaded from disk.
 The best height/hash are now only used during
 startup to determine if we are out of sync.
2017-11-11 15:53:38 -08:00
Olaoluwa Osuntokun
5dbf5084e7
routing: fix linter error 2017-11-06 16:49:36 -08:00
Olaoluwa Osuntokun
9294358b5b
routing: populate the chain filter before calling syncGraphWithChain
In this commit we fix an existing bug within the ChannelRouter. Before
this commit, we would sync our graph prune state, *then* update the
cain filter. This is incorrect as the blocks we manually pruned may
have included channel closing transactions. As a result, we would miss
the pruning of a set of channels, and assume that they were still
active.

In this commit, we fix this by reversing the order: we first update the
chain filter and THEN sync the channel graph.
2017-11-06 16:10:24 -08:00
Olaoluwa Osuntokun
ccf94e9457
routing: add new TestRouterChansClosedOfflinePruneGraph test
In this commit we add a new test to the set of unit tests for the
ChannelRouter: TestRouterChansClosedOfflinePruneGraph. This tests that
if channels are closed while the ChannelRouter is down, then upon
restart the channels are properly recognized as being closed.
2017-11-06 16:06:15 -08:00
Olaoluwa Osuntokun
2824f45aad
routing: update tests to adhere to new newMockChainView API 2017-11-06 16:00:57 -08:00
Olaoluwa Osuntokun
3be905109c
routing: add RestartRouter method to testCtx 2017-11-06 16:00:22 -08:00
Olaoluwa Osuntokun
cdf2f43432
routing: add Reset() method to mockChainView
In this commit, we add a Reset() method to the mockChainView struct.
With this new method tests are able to fully simulate a restart of the
ChannelRouter. This is necessary as the FilteredChainView instances are
assumed to be stateless, and don’t write their state to disk before a
restart.
2017-11-06 15:58:57 -08:00
Olaoluwa Osuntokun
fea8cbf920
routing: add quit channel to mockChainView 2017-11-06 15:57:29 -08:00
Olaoluwa Osuntokun
4f32ee94ea
routing: implement FilterBlock for mockChainView 2017-11-06 15:56:31 -08:00
Olaoluwa Osuntokun
5f915280bc
routing/chainview: if column wrapping on comments in neutrino.go 2017-11-06 15:55:21 -08:00
Alex
1c408adcab routing/chainview: update neutrino driver to comply with BIP 2017-11-02 20:04:07 -07:00
Johan T. Halseth
256db86b02
routing: make channel router aware of stale blocks
This commit make the channel router handle the case where
it wakes up on a stale branch, and also the case where a
reorg happens while it is active.
2017-11-03 00:05:32 +01:00
Johan T. Halseth
8eb994c14b
routing/chainView test: add testFilterBlockDisconnected
This commit adds a test for the FilteredChainView interfaces,
making sure they notify about disconnected/connected blocks
in the correct order during a reorg.
2017-11-03 00:05:31 +01:00
Johan T. Halseth
6d15be5b79
routing/chainview: use blockEventQueue for neutrino block events
This commit makes use of the blockEventQueue within the neutrino
implementation of FilteredChainView to ensure connected and
disconnected blocks are consumed in order by the reader.

It also specifies that neutrino is not to send disconnected blocks
notifications during rescans, making it consistent with the btcd
implementation.
2017-11-03 00:05:30 +01:00
Johan T. Halseth
082f012fcf
routing/chainview: implement staleBlocks for btcd view.
This commit moves btcd view away from using the deprecated
callbacks onBlockConnected/Disconnected, and instead use
onFilteredBlockConnected/disconnected.

This commit also implements the sending of disconnected blocks
over the staleBlocks channel. To send these blocks, the
blockEventQueue is used to ensure the ordering of blocks are
correctly kept.

It also changes the way filter updates are handled. Since we
now load the tx filter to the rpc server itself, we can call
RescanBlocks instead of manually filtering blocks. These
rescanned blocks are also added to the blockEventQueue,
ensuring the ordering is kept.
2017-11-03 00:05:30 +01:00
Johan T. Halseth
e9c2f703d9
routing/chainview: additional comments to FilteredChainView godoc 2017-11-03 00:05:29 +01:00
Johan T. Halseth
a219aca0d3
routing/chainView: add blockEventQueue data structure
blockEventQueue is an ordered queue for block events sent from a
FilteredChainView. The two types of possible block events are
connected/new blocks, and disconencted/stale blocks. The
blockEventQueue keeps the order of these events intact, while
still being non-blocking. This is important in order for the
chainView's call to onFilteredBlockConnected/Disconnected to not
get blocked, and for the consumer of the block events to always
get the events in the correct order.
2017-11-03 00:05:28 +01:00
Olaoluwa Osuntokun
57b9b78ce3
routing: modify path finding tests to accommodate for flip in fee calc
Before this commit, we would expect that structurally we don’t pay any
fee for the first hop, but do for the final hop. After the latest
commit, this is now flipped as when we say fee, we mean the fee that we
need to pay to transit a link. For the final hop, there’s no additional
distance to be traveled, so the fee is nothing.
2017-10-24 18:29:23 -07:00
Olaoluwa Osuntokun
a7fe7ae941
routing: correct miscalculation multi-hop onion payload fees
In this commit we fix an existing miscalculation in the fees that we
prescribe within the onion payloads for multi-hop routes. Before this
commit, if a route had more than 3 hops, then we would erroneously give
the second to last hop zero fees.

In this commit we correct this behavior, and also re-write the fee
calculation code fragment within newRoute for readability and clarity.
There are now only two cases: this is the last hop, and this is any
other hop. In the case of the last hop, simply send the exact amount
with no additional fee. In the case of an intermediate hop, we use the
_prior_ (closer to the destination) hop to calculate the amount of fees
we need, which allows us to compute the incoming flow. Using that
incoming flow, we then can compute the amount that the hop should
forward out.

Partially fixes #391.
2017-10-24 18:27:35 -07:00
Olaoluwa Osuntokun
176fde4ec0
routing: modify applyChannelUpdate to not error out if update is stale
In this commit we fix a slight bug within the existing SendPayment loop
which would cause the wrong error to be returned to users. Prior to
this commit, if we received an update identical to what we were already
aware of, then that error would be returned rather than the
ForwardingError that encapsulated this update.

In this commit with remedy this by properly returning the exact error.

Partially fixes #391.
2017-10-24 18:23:04 -07:00
Olaoluwa Osuntokun
25614f67f8
routing: restore in memory selfNode within the ChannelRouter
In this commit we restore the in memory ChannelRouter as we’ll no
dynamically set the ChannelRouter’s pointer within he spec path finding
test example.
2017-10-22 18:36:59 -07:00
Olaoluwa Osuntokun
76ec41103e
routing: extend missionControl.RequestRoute to take final CLTV delta 2017-10-22 18:36:58 -07:00
Olaoluwa Osuntokun
828057889e
routing/testdata: modify spec example to set bob as initial source node 2017-10-22 18:36:58 -07:00
Olaoluwa Osuntokun
6e00abf3f1
routing: update tests in router_test.go to pass in new param to FindRoutes 2017-10-22 18:36:52 -07:00
Olaoluwa Osuntokun
69a3783d55
routing: add new test case from spec to assert proper route calc 2017-10-22 18:36:52 -07:00
Olaoluwa Osuntokun
acd160a419
routing: update path finding tests to account for change in fee calculation 2017-10-22 18:36:51 -07:00
Olaoluwa Osuntokun
aee1619488
routing: add new FinalCLTVDelta attribute to SendPayment
In this commit, we’ll now optionally allow the user to pass in the CLTV
delta value specified by the recipient a payment. If the value isn’t
specified, then we’ll use the current global default for the payment.
2017-10-22 18:36:51 -07:00
Olaoluwa Osuntokun
c8f45e3a04
routing: add new param to FindRoutes for CLTV expiry for final hop
In this commit, we modify the FindRoutes method to pass in the CLTV
expiry for the final hop. If the value isn’t passed in, then we’ll use
the current global default value in place.
2017-10-22 18:36:50 -07:00
Olaoluwa Osuntokun
4b82e2ec43
routing: correct fee calculation when converting from path to route
In this commit, we correct the fee calculation when converting from a
path to route. Previously we would apply the “no fee” case at the
_first_ hop, rather than the last hop. As a result, we needed to swap
the edges during path finding, otherwise, if the incoming and outgoing
edges had different fee rates, then we would create an invalid onion
payload.

In this commit we now properly switch fee calculation into three cases:
  * a single hop route, so there’s no fee
  * we’re at the first hop in a multi hop route, and we apply the fee
for the _next_ hop
  * we’re at an intermediate hop and the fee calculation proceeds as
normal
2017-10-22 18:36:50 -07:00
Olaoluwa Osuntokun
8df69a83a7
routing: revert the swapping of in/out edges during path finding
In this commit we revert a commit which was added in the past as way to
allow the path -> route conversion code to remain the same, while
properly respecting the necessary time locks and fees. In an upcoming
change, this swap is no longer necessary as we’ll always use: the time
lock of the outgoing node and the fee of the incoming node.
2017-10-22 18:36:49 -07:00
Olaoluwa Osuntokun
23b6d84493
routing: modify newPath to take the preferred final CLTV delta of last hop
In this commit, rather than reading the final CLTV delta from the
channel graph itself (which would require _both_ edges to be advertised
in order to route over), we now instead have moved to allowing the
receiving node to choose their own final CLTV delta.
2017-10-22 18:36:48 -07:00
Olaoluwa Osuntokun
65482faa79
routing: remove the selfNode attribute from memory
In this commit, we’ve removed the selfNode attribute from memory, as
the set of new tests we’ll write, will depend on us being able to
switch the source node dynamically from the database itself.
2017-10-22 18:36:48 -07:00
Olaoluwa Osuntokun
f5881ad3df
routing: only write a single edge as edges in test data are now directional 2017-10-22 18:36:47 -07:00
Olaoluwa Osuntokun
86283b0d06
routing: make setting source node optional when parsing testdata 2017-10-22 18:36:47 -07:00
Olaoluwa Osuntokun
e70031da7b
routing: properly assign flags and write both edges when parsing test data 2017-10-22 18:36:46 -07:00
Olaoluwa Osuntokun
4ff66efa2b
routing: convert testChan stuct to take an int64 for FeeRate 2017-10-22 18:36:46 -07:00
Olaoluwa Osuntokun
fae3dacc9b
routing: modify basic_graph.json to include both directional edges 2017-10-22 18:36:45 -07:00
Olaoluwa Osuntokun
3c01d5bab9
routing: convert basic_graph.json testdata to use fixed point fee rate 2017-10-22 18:36:44 -07:00
Jim Posen
9fd77a6e40 multi: Update lnd to use new feature vector API. 2017-10-17 22:47:20 -07:00
Olaoluwa Osuntokun
b4273d1eaa
routing: abstract path finding behind mission control
In this commit, from the PoV of the SendPayment method we now delegate
all path finding+verification to missionControl. This change doesn’t
materially affect anything, it simply expands the abstraction to make
way for future features that more heavily utilize mission control.
2017-10-17 19:44:17 -07:00
Olaoluwa Osuntokun
209fb98d0f
routing: properly use vertexDecay for vertexes in missionControl 2017-10-16 20:31:32 -07:00
Olaoluwa Osuntokun
bbb34cebe0
routing: modify the TestSendPaymentRouteFailureFallback to clear missionControl between attempts
In order to maintain the original essence of the test, we need to clear
the state of missionControl with each attempt, essentially advancing
time between each payment attempt.
2017-10-16 19:07:40 -07:00
Olaoluwa Osuntokun
8ef829ed80
routing: modify SendPayment loop to be lazy, iterative, and use missionControl
In this commit we modify the SendPayment loop to optimize for
time-to-first-payment-success-or-failure. The prior logic would first
attempt to find at least 100 routes to the destination, then
iteratively prune them away as errors were encountered. In this commit,
we modify this approach to instead take a lazy approach: we first find
the current “best” path, attempt to send to that, and if an error
occurs we prune a section of the graph by reporting to missionControl,
then continue.

With this new approach, if the first known path has sufficient
capacity, and is available, then the payment speed is greatly improved
from the PoV of users. Additionally, we avoid the excessive computation
of crawling most of the graph in the k-shortest paths loop. With the
decay on missionControl, all routes will now feed information into the
central knowledge hung, allowing all payments to iteratively find out
the inactive portions of the payment graph.
2017-10-16 19:05:47 -07:00