Commit Graph

146 Commits

Author SHA1 Message Date
elkanatovey
5ff017664c fix typo 2020-02-27 15:22:00 +02:00
Joost Jager
06bdeb56e2
routing: add graph interface 2020-02-11 13:35:38 +01:00
Joost Jager
a8ed1b342a
routing: remove pathfinding db tx
Pathfinding is never used with an externally supplied bbolt transaction.
2020-02-10 19:55:40 +01:00
Conner Fromknecht
c7a241fc59
routing/pathfind: ignore unknown required features
This commit brings us inline with recent modifications to the spec, that
say we shouldn't pay nodes whose feature vectors signal unknown required
features, and also that we shouldn't route through nodes signaling
unknown required features.

Currently we assert that invoices don't have such features during
decoding, but now that users can specify feature vectors via the rpc
interface, it makes sense to perform this check deeper in call stack.
This will also allow us to remove the check from decoding entirely,
making decodepayreq more useful for debugging.
2020-01-08 12:25:00 -08:00
Joost Jager
c84e57a522
routing: remove unnecessary newline 2019-12-20 11:00:07 +01:00
Joost Jager
b760b25229
routing: limit routing info size during pathfinding
Also the max hop count check can be removed, because the real bound is
the payload size. By moving the check inside the search loop, we now
also backtrack when we hit the limit.
2019-12-20 11:00:05 +01:00
Joost Jager
72a6383975
routing: use absolute final expiry in pathfinding 2019-12-20 11:00:00 +01:00
Joost Jager
e546a2c42b
routing: more efficient feature validation
Also prepares for payload size tracking during pathfinding
2019-12-20 10:17:03 +01:00
Joost Jager
95ddab57fa
channeldb: add tx argument for FetchLightningNode
To allow execution within an existing tx.
2019-12-20 10:17:01 +01:00
Joost Jager
37d9ee302c
channeldb: take serialized key to fetch lightning node
This prevents inefficient key conversions in a follow up commit that
change the inner pathfinding loop.
2019-12-19 12:44:00 +01:00
Conner Fromknecht
0993256b77
routing/pathfind: set single-shot MPP if payment addr is known 2019-12-18 23:56:42 -08:00
Conner Fromknecht
7965cb08db
routing/pathfind: consolidate population of custom records 2019-12-18 23:56:23 -08:00
Conner Fromknecht
cddb71ee53
routing/pathfind: evaluate TLV support sooner
We move up the check for TLV support, since we will later use it to
determine if we can use dependent features, e.g. TLV records and payment
addresses.
2019-12-18 23:56:05 -08:00
Conner Fromknecht
e3a9603846
routing/pathfind: simplify cltv calculation 2019-12-18 23:55:47 -08:00
Conner Fromknecht
b97adf79a4
routing/pathfind: consolidate final vs non-final hop processing 2019-12-18 23:55:27 -08:00
Conner Fromknecht
495ae8ca42
routing: consolidate final hop params for newRoute
This commit creates a wrapper struct, grouping all parameters that
influence the final hop during route construction. This is a preliminary
step for passing in the receiver's invoice feature bits, which will be
used to select an appropriate payment or payload type.
2019-12-18 23:55:08 -08:00
Conner Fromknecht
71e05e05bf
routing/pathfind: set final hop features used in pathfinding
In this commit, we overwrite the final hop's features with either the
destination features or those loaded from the graph fallback. This
ensures that the same features used in pathfinding will be provided to
route construction.

In an earlier commit, we validated the final hop's transitive feature
dependencies, so we also add validation to non-final nodes.
2019-12-18 23:54:49 -08:00
Conner Fromknecht
990d55d08c
routing/pathfind: ensure final hop supports payment addrs
This commit adds an optional PaymentAddr field to the RestrictParams, so
that we can verify the final hop can support it before doing an
expensive round of pathfindig.
2019-12-18 23:54:31 -08:00
Conner Fromknecht
acb7b83ead
routing/pathfind: validate final hop feature dependencies 2019-12-18 23:54:11 -08:00
Conner Fromknecht
cfa3fe2921
routing/pathfind: fix TLV fallback for unadvertised hops
In this commit, we fix a bug that prevents us from sending custom
records to nodes that aren't in the graph. Previously we would simply
fail if we were unable to retrieve the node's features.

To remedy, we add the option of supplying the destination's feature bits
into path finding. If present, we will use them directly without
consulting the graph, resolving the original issue. Instead, we will
only consult the graph as a fallback, which will still fail if the node
doesn't exist since the TLV features won't be populated in the empty
feature vector.

Furthermore, this also permits us to provide "virtual features" into the
pathfinding logic, where we make assumptions about what the receiver
supports even if the feature vector isn't actually taken from an
invoice. This can useful in cases like keysend, where we don't have an
invoice, but we can still attempt the payment if we assume the receiver
supports TLV.
2019-12-18 23:53:51 -08:00
Joost Jager
c37289cd94
routing: pass custom records into pathfinding 2019-12-12 00:15:00 +01:00
Joost Jager
d02de70d20
multi: do not use tlv.Record outside wire format handling
This commit prepares for more manipulation of custom records. A list of
tlv.Record types is more difficult to use than the more basic
map[uint64][]byte.

Furthermore fields and variables are renamed to make them more
consistent.
2019-12-12 00:14:58 +01:00
Joost Jager
883f9e5f9a
Merge pull request #3749 from joostjager/extended-routing-failures
routing: local balance check before path finding
2019-12-04 12:17:17 +01:00
Joost Jager
9ae014edf6
routing: use self instead of source node for creating unified policy
A unified policy differs between local channels and other channels on
the network. There is more information available for local channels and
this is used in the unified policy.

Previously we used the pathfinding source pubkey to determine whether to
apply the local channel logic or not. If queryroutes is executed with a
source node that isn't the self node, this wouldn't work.
2019-12-04 09:45:09 +01:00
Joost Jager
97344af8f3
routing: local balance check 2019-12-04 09:45:07 +01:00
Joost Jager
3aaf32dc2e
routing: improve equal cost route comparison
When the (virtual) payment attempt cost is set to zero, probabilities
are no longer a factor in determining the best route. In case of routes
with equal costs, we'd just go with the first one found. This commit
refines this behavior by picking the route with the highest probability.
So even though probability doesn't affect the route cost, it is still
used as a tie breaker.
2019-12-02 14:23:57 +01:00
Joost Jager
ac2df9bbf4
routing: use simple errors for path finding 2019-11-27 15:21:08 +01:00
Joost Jager
f8e9efbf99
routing: allow route to self 2019-11-26 10:15:44 +01:00
Joost Jager
81b7798c03
routing: pop heap at the end of the loop
This prepares for routing to self.
2019-11-26 10:15:42 +01:00
Joost Jager
683282fa24
routing: check loop conditions at end
This prepares for routing to self. When checking the condition at the
start, the loop would terminate immediately because the source is equal
to the target.
2019-11-26 10:15:40 +01:00
Joost Jager
814dbea745
routing: add last hop restriction 2019-11-18 21:03:01 +01:00
Joost Jager
a347237e7a
routing: use unified policy for path finding
In this commit we change path finding to no longer consider all channels
between a pair of nodes individually. We assume that nodes forward
non-strict and when we attempt a connection between two nodes, we don't
want to try multiple channels because their policies may not be identical.
Having distinct policies for channel to the same peer is against the
recommendation in the spec, but it happens in the wild. Especially since
we recently changed the default cltv delta value.

What this commit introduces is a unified policy. This can be looked upon
as the greatest common denominator of all policies and should maximize
the probability of getting the payment forwarded.
2019-10-25 11:22:28 +02:00
Juan Pablo Civile
818c302d46 routing: use nodeWithDist instead of vertex to avoid map access
The same nodeWithDist was fetched from the map for every channel it has.
This struct is not mutated so it can be fetched before and reused.
2019-10-24 21:38:07 -03:00
Juan Pablo Civile
df70095ad0 routing: optimize path finding structures
distance map now holds the edge the current path is coming from,
removing the need for next map.

Both distance map and distanceHeap now hold pointers instead of the full
struct to reduce allocations and copies.

Both these changes reduced path finding time by ~5% and memory usage by
~2mb.
2019-10-24 21:38:07 -03:00
Juan Pablo Civile
fc36df0e60 routing: avoid unneeded map access
`processEdge` basically had 4 expensive operations: 3 map accesses and
updating the heap. This removes one of those for a small performance
gain.
2019-10-24 21:31:30 -03:00
Juan Pablo Civile
3e60a23632 routing: pre-allocate the distance map to an estimated node count
Pre-sizing these structures avoids a lot of map resizing, which causes
copies and rehashing of entries. We mostly know that the map won't
exceed that size, and it doesn't affect memory usage in any significant
way.
2019-10-24 21:31:30 -03:00
Juan Pablo Civile
2141713936 routing: avoid walking all nodes for path finding if we don't need to
Calling `ForEachNode` hits the DB, and allocates and parses every node
in the graph. Walking the channels also loads nodes from the DB, so this
meant that each node was read/parsed/allocated several times per run.

This reduces runtime by ~10ms and memory usage by ~4mb.
2019-10-24 21:31:30 -03:00
Juan Pablo Civile
5389161162 routing: make log in findPath hot path use logClosure
It generates heap allocations for it's params even if it won't end up
using them.

Reduces memory usage by 2mb
2019-10-24 21:31:30 -03:00
Wilmer Paulino
0fc401de19
routing+routerrpc: take max cltv limit into account within path finding
With the introduction of the max CLTV limit parameter, nodes are able to
reject HTLCs that exceed it. This should also be applied to path
finding, otherwise HTLCs crafted by the same node that exceed it never
left the switch. This wasn't a big deal since the previous max CLTV
limit was ~5000 blocks. Once it was lowered to 1008, the issue became
more apparent. Therefore, all of our path finding attempts now have a
restriction of said limit in in order to properly carry out HTLCs to the
network.
2019-10-11 18:04:49 -04:00
Joost Jager
9f3656efeb
routing: log performance metrics 2019-10-03 15:07:11 +02:00
Joost Jager
8c44cf4a22
channeldb: add ComputeFee function 2019-09-17 11:10:58 +02:00
Joost Jager
3d7de2ad39
multi: remove dead code 2019-09-10 17:21:59 +02:00
Joost Jager
ff0c5a0d5e
routing: process successes in mission control
This commit modifies paymentLifecycle so that it not only feeds
failures into mission control, but successes as well.
This allows for more accurate probability estimates. Previously,
the success probability for a successful pair and a pair with
no history was equal. There was no force that pushed towards
previously successful routes.
2019-08-23 09:15:41 +02:00
Olaoluwa Osuntokun
4697cfde30
routing: extend path finding to be TLV-EOB aware, allow dest TLV records
In this commit, we extend the path finding to be able to recognize when
a node needs the new TLV format, or the legacy format based on the
feature bits they expose. We also extend the `LightningPayment` struct
to allow the caller to specify an arbitrary set of TLV records which can
be used for a number of use-cases including various variants of
spontaneous payments.
2019-08-22 18:53:01 -07:00
Joost Jager
f1769c8c8c
routing: convert to node pair based
Previously mission control tracked failures on a per node, per channel basis.
This commit changes this to tracking on the level of directed node pairs. The goal
of moving to this coarser-grained level is to reduce the number of required
payment attempts without compromising payment reliability.
2019-08-13 19:21:37 +02:00
Joost Jager
34b264a3d8
routing: create PathFindingConfig for global parameters 2019-07-18 15:49:25 +02:00
nsa
7e6a34532e routing: add index map to distanceHeap
This commit adds the pubkeyIndices map to the distanceHeap to avoid
duplicate entries on the heap. This happened in the earlier iteration
of the findPath algorithm and would cause the driving loop to
evaluate already evaluated entries when there was no need.
2019-07-12 06:08:01 -04:00
nsa
61a1ab08fb routing+channeldb: findPath uses Vertex instead of LightningNode
This commit modifies the nodeWithDist struct to use a route.Vertex
instead of a *channeldb.LightningNode. This change, coupled with
the new ForEachNodeChannel function, allows the findPath Djikstra's
algorithm to cut down on database lookups since we no longer need
to call the FetchOtherNode function.
2019-07-12 06:08:01 -04:00
Joost Jager
054e42f680
routing+routerrpc: expose mission control parameters in lnd config
This commit exposes the three main parameters that influence mission
control and path finding to the user as command line or config file
flags. It allows for fine-tuning for optimal results.
2019-06-04 13:22:50 +02:00
Joost Jager
7133f37bb8
routing: global probability based mission control
Previously every payment had its own local mission control state which
was in effect only for that payment. In this commit most of the local
state is removed and payments all tap into the global mission control
probability estimator.

Furthermore the decay time of pruned edges and nodes is extended, so
that observations about the network can better benefit future payment
processes.

Last, the probability function is transformed from a binary output to a
gradual curve, allowing for a better trade off between candidate routes.
2019-06-04 10:00:25 +02:00