Commit Graph

101 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
4697cfde30
routing: extend path finding to be TLV-EOB aware, allow dest TLV records
In this commit, we extend the path finding to be able to recognize when
a node needs the new TLV format, or the legacy format based on the
feature bits they expose. We also extend the `LightningPayment` struct
to allow the caller to specify an arbitrary set of TLV records which can
be used for a number of use-cases including various variants of
spontaneous payments.
2019-08-22 18:53:01 -07:00
Joost Jager
f1769c8c8c
routing: convert to node pair based
Previously mission control tracked failures on a per node, per channel basis.
This commit changes this to tracking on the level of directed node pairs. The goal
of moving to this coarser-grained level is to reduce the number of required
payment attempts without compromising payment reliability.
2019-08-13 19:21:37 +02:00
Olaoluwa Osuntokun
8c389d13f9
Merge pull request #3197 from breez/optimize_prune_zombie_channels
Optimize prune zombie channels
2019-07-18 20:56:56 -07:00
Joost Jager
34b264a3d8
routing: create PathFindingConfig for global parameters 2019-07-18 15:49:25 +02:00
Roei Erez
9781ea0082 router: Optimize pruneZombieChannels.
The current approach iterates all channels in the graph in order to
filter those in need. This approach is time consuming, several seconds
on my mobile device for ~40,000 channels, while during this time the
db is locked in a transaction.

The proposed change is to use an existing functionality that utilize the
fact that channel update are saved indexed by date. This method enables
us to go over only a small subset of the channels, only those that
were updated before the "channel expiry" time and further filter
them for our need.
The same graph that took several seconds to prune was pruned, after
the change, in several milliseconds.

In addition for testing purposes I added Initiator field to the
testChannel structure to reflect the channeldEdgePolicy direction.
2019-07-16 17:48:07 +03:00
Joost Jager
569f821142
routing/test: allow arbitrary graph source nodes 2019-07-11 10:37:22 +02:00
Joost Jager
7133f37bb8
routing: global probability based mission control
Previously every payment had its own local mission control state which
was in effect only for that payment. In this commit most of the local
state is removed and payments all tap into the global mission control
probability estimator.

Furthermore the decay time of pruned edges and nodes is extended, so
that observations about the network can better benefit future payment
processes.

Last, the probability function is transformed from a binary output to a
gradual curve, allowing for a better trade off between candidate routes.
2019-06-04 10:00:25 +02:00
Joost Jager
3349d517aa
routing: add min probability to path finding
This commit adds a new restriction to pathfinding that allows returning
only routes with a minimum success probability.
2019-06-04 10:00:23 +02:00
Joost Jager
6b70791c2d
routing: use probability source in path finding
This PR replaces the previously used edge and node ignore lists in path
finding by a probability based system. It modifies path finding so that
it not only compares routes on fee and time lock, but also takes route
success probability into account.

Allowing routes to be compared based on success probability is achieved
by introducing a 'virtual' cost of a payment attempt and using that to
translate probability into another cost factor.
2019-06-04 10:00:21 +02:00
Joost Jager
35003bd87c
routing/test: sort nodes for channel graph
This commit ensures that channel endpoints in EdgeInfo are ordered
node1 < node2 to not violate assumptions being made by dependent code.
2019-06-04 10:00:13 +02:00
Joost Jager
7a5bd29a69
lnrpc+routing: remove k shortest path finding 2019-05-07 19:54:06 +02:00
Olaoluwa Osuntokun
bab957382f
router: convert Route.ToHopPayloads() to Route.ToSphinxPath()
In this commit, we update the process that we use to generate a sphinx
packet to send our onion routed HTLC. Due to recent changes in the
`sphinx` package we use, we now need to use a new PaymentPath struct. As
a result, it no longer makes sense to split up the nodes in a route and
their per hop paylods as they're now in the same struct. All tests have
been updated accordingly.
2019-04-30 20:13:32 -07:00
Johan T. Halseth
ee257fd0eb
multi: move Route to sub-pkg routing/route 2019-04-29 14:52:33 +02:00
Wilmer Paulino
f23c3b488e
routing: prune channels with disabled bit set on both edges
In this commit, we add an additional heuristic when running with
AssumeChannelValid. Since AssumeChannelValid being present assumes that
we're not able to quickly determine whether channels are valid, we also
assume that any channels with the disabled bit set on both sides are
considered zombie. This should be relatively safe to do, since the
disabled bits are usually set when the channel is closed on-chain. In
the case that they aren't, we'll have to wait until both edges haven't
had a new update within two weeks to prune them.
2019-04-18 21:57:34 -07:00
Wilmer Paulino
78bdcbb115
routing: prune channels only if both edges are present
We do this to ensure we don't prune too aggressively, as it's possible
that we've only received the channel announcement for a channel, but not
its accompanying channel updates.
2019-04-18 21:57:31 -07:00
Wilmer Paulino
29664c9704
routing: make test channel policies optional 2019-04-18 20:47:32 -07:00
Joost Jager
3b94627c12
routing: remove route cache
This commit removes the QueryRoutes route cache. It is causing wrong
routes to be returned because not all of the request parameters are
stored.

The cache allowed high frequency QueryRoutes calls to the same
destination and with the same amount to be returned fast. This behaviour
can also be achieved by caching the request on the client side. In case
a route is invalidated because of for example a channel update,
the subsequent SendToRoute call will fail. This is a trigger to call
QueryRoutes again for a fresh route.
2019-04-13 08:10:32 +02:00
Joost Jager
6006549ed5
routing: enforce cltv limit in path finding 2019-03-19 10:36:45 +01:00
Joost Jager
2be1051fb6
zpay32: move HopHint and DefaultFinalCLTVDelta 2019-03-15 10:08:32 +01:00
Joost Jager
c62c9d64da
routing: add source parameter to query routes
This commit allows execution of QueryRoutes from any source node.
Previously this was restricted to only the self node.
2019-03-06 15:31:03 +01:00
Joost Jager
7719bc432f
routing: take Vertex types as path finding source and target nodes
Currently public keys are represented either as a 33-byte array (Vertex) or as a
btcec.PublicKey struct. The latter isn't useable as index into maps and
cannot be used easily in compares. Therefore the 33-byte array
representation is used predominantly throughout the code base.

This commit converts the argument types of source and target nodes for
path finding to Vertex. Path finding executes no crypto operations and
using Vertex simplifies the code.

Additionally, it prepares for the path finding source parameter to be
exposed over rpc in a follow up commit without requiring conversion back
and forth between Vertex and btcec.PublicKey.
2019-03-06 15:31:00 +01:00
Joost Jager
b09adc3219
lnrpc+routing: add edges and nodes restrictions to query routes
This commit allows the execution of QueryRoutes to be controlled using
lists of black-listed edges and nodes. Any path returned will not pass
through the edges and/or nodes on the list.
2019-03-06 15:30:57 +01:00
Joost Jager
4376f3e1bd
routing: allow nil maps for ignored edges and nodes
This allows removing a lot of empty map initialization code and makes
the code more readable.
2019-03-06 15:30:54 +01:00
Joost Jager
b2b28b49b1
routing: export RestrictParams and EdgeLocator 2019-03-06 15:30:52 +01:00
Joost Jager
4937304732
routing: remove redundant fee limit check in newRoute
This check was a left over from when the fee limit wasn't checked yet in
the path finding algorithm.
2019-03-06 15:30:49 +01:00
Valentine Wallace
648adaea69 routing/pathfind: ensure max htlc is considered during path finding
In this commit, we update the path finding logic to
ignore a channel if the HTLC value (including the fees
at the point) exceeds the max HTLC value (if set) of the
link.
2019-03-01 08:05:27 -08:00
Valentine Wallace
348a66ed13 routing: update test edge policy fields to include max htlc + new flags
Since the MaxHTLC field was recently added to the ChannelEdgePolicy struct,
and the Flags field was broken into ChannelFlags and MessageFlags, the
test edge policies should be updated accordingly.
2019-03-01 08:05:27 -08:00
Joost Jager
7c30a8c493
routing: add outgoing channel restriction 2019-02-08 23:05:56 +01:00
Valentine Wallace
0fd6004958
multi: partition lnwire.ChanUpdateFlag into ChannelFlags and MessageFlags
In this commit:

* we partition lnwire.ChanUpdateFlag into two (ChanUpdateChanFlags and
ChanUpdateMsgFlags), from a uint16 to a pair of uint8's

* we rename the ChannelUpdate.Flags to ChannelFlags and add an
additional MessageFlags field, which will be used to indicate the
presence of the optional field HtlcMaximumMsat within the ChannelUpdate.

* we partition ChannelEdgePolicy.Flags into message and channel flags.
This change corresponds to the partitioning of the ChannelUpdate's Flags
field into MessageFlags and ChannelFlags.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:26 +01:00
Joost Jager
b2cb760c65
routing: check for empty hops list
This commit fixes a crash that could be triggered by sending an empty
hop list to the SendToRoute rpc.
2018-12-10 11:27:07 +01:00
Joost Jager
3ed16837e4
routing: prune channels unidirectional
In this commit we introduce pruning of channel edges instead of channels.
Channel failures apply to a single direction and it is unnecessarily
restricting to prune both directions.
2018-12-05 14:39:23 +01:00
Joost Jager
8b8e82a171
routing: remove route hop maps
Hop maps were used in a test to verify the population of the hop map
itself and further only in a single function (getFailedChannelID).
Rewrote that function and removed the hop maps completely.
2018-12-05 14:39:23 +01:00
Joost Jager
42debc6c12
routing: fix node ordering in test graph
There is the general assumption that channel edge policy nodes are
ordered such that the node1 pubkey is smaller than the key of node 2. In
the test graph, this assumption didn't hold. This commit fixes the test
graph and also adds a check to prevent this from happening again.
2018-12-05 09:15:27 +01:00
Johan T. Halseth
5f8425cb03
routing/pathfind_test: check error before attempting cleanup 2018-12-04 12:20:45 +01:00
Johan T. Halseth
67b018476c
routing/pathfind test: add TestPathSourceEdgesBandwidth
This commit adds a new test that checks that the bandwidth hints are
considered correclty for local channels, and that disable flags are
ignored in this case.
2018-12-04 12:20:45 +01:00
Johan T. Halseth
0b6a19866b
routing/pathfind: pass ignore disable flags for local channels
To decouple our own path finding from the graph state, we don't consider
the disable bit when attempting to use local channels. Instead the
bandwidth hints will be zero for local inactive channels.

We alos modify the unit test to check that the disable flag is ignored
for local edges.
2018-12-04 12:20:45 +01:00
Johan T. Halseth
20b9d5d000
routing: define graphParams and restrictParams
To avoid the findPath parameter list getting out of hand, we define new
structs that wraps the mandatory and optional parameters to findPath.
2018-12-04 12:20:44 +01:00
Joost Jager
a779004a18
lnrpc+routing: fix issues with missing data in unmarshallRoute
In this commit the dependency of unmarshallRoute on edge policies being
available is removed. Edge policies may be unknown and reported as nil.
SendToRoute does not need the policies, but it does need pubkeys of the
route hops. In this commit, unmarshallRoute is modified so that it
takes the pubkeys from edgeInfo instead of channelEdgePolicy.

In addition to this, the route structure is simplified. No more connection
to the database at that point. Fees are determined based on incoming and
outgoing amounts.
2018-10-13 22:59:23 +02:00
Olaoluwa Osuntokun
d8e6085c17
multi: update to go 1.11 gofmt 2018-10-07 14:37:38 +09:00
Joost Jager
47d2e1e024
routing: add option to specify explicit id for test channels 2018-09-11 20:10:15 +02:00
Joost Jager
c5c580ab54
routing: add private key map to test graph instance 2018-09-11 20:10:14 +02:00
Joost Jager
5daf75b264
routing: restructure test context creation 2018-09-11 20:10:14 +02:00
Joost Jager
2d255e3bc3
routing: use fixed time value in tests 2018-09-11 20:10:14 +02:00
Joost Jager
29b6bae4ed routing: convert fee limit test to table-driven 2018-08-09 20:23:40 -07:00
Joost Jager
3e7473f4f0 routing: backward searching 2018-08-09 20:23:40 -07:00
Joost Jager
e52d829168 routing: path finding test refactored 2018-08-09 20:23:40 -07:00
Conner Fromknecht
9eff876f52
routing/pathfind_test: fmt whitespace 2018-08-02 18:20:49 -07:00
Conner Fromknecht
1ded697e8d
multi: sort import paths with gofmt 2018-08-02 18:20:49 -07:00
Olaoluwa Osuntokun
6f60f139f4 multi: switch over import paths from roasbeef/* to btcsuite/* 2018-07-13 17:05:39 -07:00
Olaoluwa Osuntokun
c045defd0f
Merge pull request #1358 from joostjager/newfee
routing: routing may come up with suboptimal routes (weight function)
2018-06-28 19:18:13 -07:00