Commit Graph

717 Commits

Author SHA1 Message Date
Joost Jager
e8fd05e8e3
tlv+record+routing: add payload size calculation 2019-12-20 11:00:01 +01:00
Joost Jager
72a6383975
routing: use absolute final expiry in pathfinding 2019-12-20 11:00:00 +01:00
Joost Jager
e546a2c42b
routing: more efficient feature validation
Also prepares for payload size tracking during pathfinding
2019-12-20 10:17:03 +01:00
Joost Jager
95ddab57fa
channeldb: add tx argument for FetchLightningNode
To allow execution within an existing tx.
2019-12-20 10:17:01 +01:00
Joost Jager
a3a1b488a6
routing: fix gap in result interpretation
Fixes an off by one error that skipped assigning a success pair. Added a
test that previously failed because of this bug.
2019-12-20 09:14:15 +01:00
Conner Fromknecht
ff443a389b
routing/pathfind_test: constructed sorted edge policies
This commit fixes a potential bug in our test harness, by ensuring that
the constructed node policies are configured _after_ sorting. Currently
the node pubkeys are sorted, but additional parameters (max htlc,
disabled, etc) are applied using the unsorted policies.

Most of the constructors used today use the symmetric channel
constructor, so this shouldn't cause an issue with the majority of our
tests. We recently introduced an asymmetric channel constructor for
which this could have been an issue, however, no known issues were
discovered.

Lastly, we remove the direction from the configuration altogether, and
derive it purely from the final sorting of the pubkeys.
2019-12-19 03:47:24 -08:00
Joost Jager
37d9ee302c
channeldb: take serialized key to fetch lightning node
This prevents inefficient key conversions in a follow up commit that
change the inner pathfinding loop.
2019-12-19 12:44:00 +01:00
Conner Fromknecht
f868bc128b
rpcserver+routerrpc: thread features + payment addr to SendPayment 2019-12-18 23:56:59 -08:00
Conner Fromknecht
0993256b77
routing/pathfind: set single-shot MPP if payment addr is known 2019-12-18 23:56:42 -08:00
Conner Fromknecht
7965cb08db
routing/pathfind: consolidate population of custom records 2019-12-18 23:56:23 -08:00
Conner Fromknecht
cddb71ee53
routing/pathfind: evaluate TLV support sooner
We move up the check for TLV support, since we will later use it to
determine if we can use dependent features, e.g. TLV records and payment
addresses.
2019-12-18 23:56:05 -08:00
Conner Fromknecht
e3a9603846
routing/pathfind: simplify cltv calculation 2019-12-18 23:55:47 -08:00
Conner Fromknecht
b97adf79a4
routing/pathfind: consolidate final vs non-final hop processing 2019-12-18 23:55:27 -08:00
Conner Fromknecht
495ae8ca42
routing: consolidate final hop params for newRoute
This commit creates a wrapper struct, grouping all parameters that
influence the final hop during route construction. This is a preliminary
step for passing in the receiver's invoice feature bits, which will be
used to select an appropriate payment or payload type.
2019-12-18 23:55:08 -08:00
Conner Fromknecht
71e05e05bf
routing/pathfind: set final hop features used in pathfinding
In this commit, we overwrite the final hop's features with either the
destination features or those loaded from the graph fallback. This
ensures that the same features used in pathfinding will be provided to
route construction.

In an earlier commit, we validated the final hop's transitive feature
dependencies, so we also add validation to non-final nodes.
2019-12-18 23:54:49 -08:00
Conner Fromknecht
990d55d08c
routing/pathfind: ensure final hop supports payment addrs
This commit adds an optional PaymentAddr field to the RestrictParams, so
that we can verify the final hop can support it before doing an
expensive round of pathfindig.
2019-12-18 23:54:31 -08:00
Conner Fromknecht
acb7b83ead
routing/pathfind: validate final hop feature dependencies 2019-12-18 23:54:11 -08:00
Conner Fromknecht
cfa3fe2921
routing/pathfind: fix TLV fallback for unadvertised hops
In this commit, we fix a bug that prevents us from sending custom
records to nodes that aren't in the graph. Previously we would simply
fail if we were unable to retrieve the node's features.

To remedy, we add the option of supplying the destination's feature bits
into path finding. If present, we will use them directly without
consulting the graph, resolving the original issue. Instead, we will
only consult the graph as a fallback, which will still fail if the node
doesn't exist since the TLV features won't be populated in the empty
feature vector.

Furthermore, this also permits us to provide "virtual features" into the
pathfinding logic, where we make assumptions about what the receiver
supports even if the feature vector isn't actually taken from an
invoice. This can useful in cases like keysend, where we don't have an
invoice, but we can still attempt the payment if we assume the receiver
supports TLV.
2019-12-18 23:53:51 -08:00
Conner Fromknecht
bd66c0d34e
routing/pathfind_test: allow custom node features
This commit allows custom node features to be populated in specific test
instances. For consistency, we auto-populate an empty feature vector for
nodes that have nil feature vectors before writing them to the database.
2019-12-18 23:53:32 -08:00
Conner Fromknecht
618810394c
routing/pathfind_test: add asymmetric chan consructor
Later this will be used to populate distinct feature vectors for either
end of the channel.
2019-12-18 23:53:18 -08:00
Joost Jager
d85f378aa2
routing: move zero hops check 2019-12-16 14:53:47 +01:00
Joost Jager
31b2c22cf5
routing: check for invalid routes 2019-12-16 14:53:45 +01:00
Joost Jager
b8abae6a7e
routing: rename route variable to prevent clash with package 2019-12-14 08:47:53 +01:00
Joost Jager
75aa4e7061
routing: require tlv capability for custom record payments
Previously if a payment was sent with custom records attached, path
finding wouldn't perform a check whether the final node was capable of
receiving custom records in a tlv payload.
2019-12-12 00:15:02 +01:00
Joost Jager
c37289cd94
routing: pass custom records into pathfinding 2019-12-12 00:15:00 +01:00
Joost Jager
d02de70d20
multi: do not use tlv.Record outside wire format handling
This commit prepares for more manipulation of custom records. A list of
tlv.Record types is more difficult to use than the more basic
map[uint64][]byte.

Furthermore fields and variables are renamed to make them more
consistent.
2019-12-12 00:14:58 +01:00
Joost Jager
b6eb3a69ba
routing+lnrpc+lncli: allow setting min htlc policy 2019-12-11 00:17:08 +01:00
Joost Jager
74c2df658e
localchans: log policy update error 2019-12-11 00:17:06 +01:00
Joost Jager
f289dbd826
routing: define subsystem constant 2019-12-11 00:17:04 +01:00
Joost Jager
ddb98fcc41
multi: distinguish between htlc in and out constraints 2019-12-11 00:16:57 +01:00
Joost Jager
883f9e5f9a
Merge pull request #3749 from joostjager/extended-routing-failures
routing: local balance check before path finding
2019-12-04 12:17:17 +01:00
Joost Jager
9ae014edf6
routing: use self instead of source node for creating unified policy
A unified policy differs between local channels and other channels on
the network. There is more information available for local channels and
this is used in the unified policy.

Previously we used the pathfinding source pubkey to determine whether to
apply the local channel logic or not. If queryroutes is executed with a
source node that isn't the self node, this wouldn't work.
2019-12-04 09:45:09 +01:00
Joost Jager
97344af8f3
routing: local balance check 2019-12-04 09:45:07 +01:00
Joost Jager
3aaf32dc2e
routing: improve equal cost route comparison
When the (virtual) payment attempt cost is set to zero, probabilities
are no longer a factor in determining the best route. In case of routes
with equal costs, we'd just go with the first one found. This commit
refines this behavior by picking the route with the highest probability.
So even though probability doesn't affect the route cost, it is still
used as a tie breaker.
2019-12-02 14:23:57 +01:00
Joost Jager
c15b0316a2
routing: use FailureReasonError for internal path finding errors 2019-11-27 15:21:12 +01:00
Joost Jager
b5f1bde604
routing: add error var for pre-built route tried 2019-11-27 15:21:10 +01:00
Joost Jager
ac2df9bbf4
routing: use simple errors for path finding 2019-11-27 15:21:08 +01:00
Joost Jager
6774b5a007
routing: remove unused error codes 2019-11-27 15:21:06 +01:00
Joost Jager
7e4f9dd70a
routing/test: simplify context instantiation 2019-11-26 21:40:30 +01:00
Joost Jager
f8e9efbf99
routing: allow route to self 2019-11-26 10:15:44 +01:00
Joost Jager
81b7798c03
routing: pop heap at the end of the loop
This prepares for routing to self.
2019-11-26 10:15:42 +01:00
Joost Jager
683282fa24
routing: check loop conditions at end
This prepares for routing to self. When checking the condition at the
start, the loop would terminate immediately because the source is equal
to the target.
2019-11-26 10:15:40 +01:00
Joost Jager
2b332893b7
routing/test: add test focusing on not having a cycle 2019-11-26 10:15:38 +01:00
Joost Jager
cdf1aa5bc1
routing: track amt ranges in mc 2019-11-22 11:17:28 +01:00
Joost Jager
62f8cca75b
routing+routerrpc: also expose amt for success results in mc 2019-11-22 11:17:26 +01:00
Olaoluwa Osuntokun
6f9fcfaccc
Merge pull request #3499 from cfromknecht/mpp-payments-rpc
channeldb+rpcserver: expose legacy payments as multi-path payments
2019-11-21 18:38:53 -08:00
Johan T. Halseth
8755a35860
Merge pull request #3667 from cfromknecht/log-fixes
fundingmanager+routing log touch ups
2019-11-20 15:57:05 +01:00
Conner Fromknecht
063f24f2ed
channeldb+routing: expose HTLCs in payment subscriptions
This commit modifies the FetchPayment method to return MPPayment structs
converted from the legacy on-disk format. This allows us to attach the
HTLCs to the events given to clients subscribing to the outcome of an
HTLC.

This commit also bubbles up to the routerrpc/router_server, by
populating HTLCAttempts in the response and extracting the legacy route
field from the HTLCAttempts.
2019-11-19 20:43:38 -08:00
Conner Fromknecht
01cecb1f27
multi: rename to FailureReasonPaymentDetails
Prior name is too long XD
2019-11-19 20:42:21 -08:00
Conner Fromknecht
9e019407fb
routing/control_tower: fix PaymentResult godocs 2019-11-19 20:40:44 -08:00
Conner Fromknecht
fbd599a2cb
multi: rename Payment.PaymentPreimage to Payment.Preimage
Preliminary step to exposing PaymentPreimage() as a method that can be
shared between legacy payments and mutli-path payments.
2019-11-19 20:40:29 -08:00
Joost Jager
a00a360018
Merge pull request #3739 from joostjager/incoming-restriction
routing: last hop restriction
2019-11-19 08:38:33 +01:00
Joost Jager
f28941c7e4
routing+routerrpc+lncli: enable last hop restriction for payments 2019-11-18 21:03:03 +01:00
Joost Jager
814dbea745
routing: add last hop restriction 2019-11-18 21:03:01 +01:00
Wilmer Paulino
29f12411cd
Merge pull request #3732 from Crypt-iQ/router_race_1115
routing: fix race condition when resuming payments
2019-11-18 11:06:18 -08:00
Joost Jager
0f6af2ed54
routing/test: fix outgoing restriction test
Previously both paths were equal cost, so it could also be a
coincedence that the path with the outgoing restriction would be chosen.
2019-11-18 12:09:43 +01:00
Joost Jager
5fde7977f4
routing/test: create path finding test context 2019-11-18 10:55:43 +01:00
nsa
929167b9af
routing: fix race condition when resuming payments 2019-11-15 14:44:05 -05:00
Joost Jager
a364d2cd82
routing: log route when payment attempt fails 2019-11-12 14:13:31 +01:00
Joost Jager
85dcaff0d9
routing/route: report amount per hop in Route.String()
Makes log output slightly more descriptive to aid debugging of routing
problems.
2019-11-12 14:13:27 +01:00
Joost Jager
b222b6e625
Merge pull request #3685 from cfromknecht/flat-features
flat features
2019-11-09 11:43:45 +01:00
Conner Fromknecht
5e27b5022c
multi: remove LocalFeatures and GlobalFeatures 2019-11-08 05:32:00 -08:00
Joost Jager
dc0399af51
routing: use distinct probability estimation for local channels
Previously we used the a priori probability also for our own untried
channels. This led to local channels that had seen a success already
being prioritized over untried local channels. In some cases, depending
on the configured payment attempt cost, this could lead to the payment
taking a two hop route while a direct payment was also possible.
2019-11-07 11:26:52 +01:00
Joost Jager
5a80c3459f
routing: create prob estimation func taking external node prob 2019-11-07 11:25:14 +01:00
Olaoluwa Osuntokun
863d795c94
Merge pull request #3616 from Crypt-iQ/router_race_1019
ticker: add tickerMtx to prevent data race
2019-11-05 19:47:18 -08:00
Conner Fromknecht
0035e8b99d
routing/router: fix double hex encoding of target pubkey 2019-11-05 15:04:24 -08:00
Conner Fromknecht
6d971e5113
routing/route/route: add optional MPP field to Hop
This commit also modifies the Router serialization to persist the MPP
struct when present, and properly restore it when loading from disk.
2019-11-04 14:11:28 -08:00
nsa
b392dd1b95
routing: move statTicker to prevent race condition 2019-11-02 13:55:41 -04:00
Olaoluwa Osuntokun
acd8a6e302
Merge pull request #3470 from cfromknecht/invalid-onion-payload
htlcswitch+lnwire: invalid onion payload
2019-11-01 18:58:00 -07:00
Joost Jager
fcf81ed8ff
Merge pull request #3556 from joostjager/query-mc-prob
routerrpc: add queryprob rpc
2019-10-31 08:03:29 +01:00
Conner Fromknecht
d08e8ddd61
routing/result_interpretation: process InvalidOnionPayload
An InvalidOnionPayload implies that the onion was successfully received
by the reporting node, but that they were unable to extract the
contents. Since we assume our own behavior is correct, this mostly
likely poins to an error in the reporter's implementation or that we
sent an unknown required type. Therefore we only penalize that single
hop, and consider the failure terminal if the receiver reported it.
2019-10-30 21:20:08 -07:00
Conner Fromknecht
2f8021d8db
routing/result_interpretation: fix off-by-one for incoming failure
Previously we would not mark a success for the first hop if the fail
source index was 2. We also add a test to assert this behavior.
2019-10-30 21:18:52 -07:00
Joost Jager
a8f077a841
Merge pull request #3558 from joostjager/non-strict-pathfinding
routing: non-strict path finding
2019-10-30 08:42:28 +01:00
Lars Lehtonen
6254c53a4a
routing: fix use of T.Fatal() in test goroutine 2019-10-29 13:21:53 -07:00
Joost Jager
729c3a6bd6
routing: use unified policy for build route 2019-10-29 19:17:58 +01:00
Joost Jager
fb57255b5c
routerrpc: add mc probability query rpc
Probabilities are no longer returned for querymc calls. To still provide
some insight into the mission control internals, this commit adds a new
rpc that calculates a success probability estimate for a specific node
pair and amount.
2019-10-29 12:32:19 +01:00
Joost Jager
3d8adaef95
routing: embed TimedPairResult in snapshot data 2019-10-29 12:32:15 +01:00
Joost Jager
57911faa98
routing: export TimedPairResult
Prepares for this data structure being accessed in mission control rpc
implementations.
2019-10-29 12:32:13 +01:00
Joost Jager
912a8201f9
routing: remove embedded struct from timedPairResult
This prepares for decoupling the result interpretation of a single
payment attempt from the information stored in mission control memory
on the history of a node pair. A planned follow-up where we store both
the last success and last failure requires this decoupling.
2019-10-29 12:32:11 +01:00
Joost Jager
a347237e7a
routing: use unified policy for path finding
In this commit we change path finding to no longer consider all channels
between a pair of nodes individually. We assume that nodes forward
non-strict and when we attempt a connection between two nodes, we don't
want to try multiple channels because their policies may not be identical.
Having distinct policies for channel to the same peer is against the
recommendation in the spec, but it happens in the wild. Especially since
we recently changed the default cltv delta value.

What this commit introduces is a unified policy. This can be looked upon
as the greatest common denominator of all policies and should maximize
the probability of getting the payment forwarded.
2019-10-25 11:22:28 +02:00
Joost Jager
6b391d04d0
routing/test: mark function as test helper 2019-10-25 11:22:26 +02:00
Juan Pablo Civile
818c302d46 routing: use nodeWithDist instead of vertex to avoid map access
The same nodeWithDist was fetched from the map for every channel it has.
This struct is not mutated so it can be fetched before and reused.
2019-10-24 21:38:07 -03:00
Juan Pablo Civile
df70095ad0 routing: optimize path finding structures
distance map now holds the edge the current path is coming from,
removing the need for next map.

Both distance map and distanceHeap now hold pointers instead of the full
struct to reduce allocations and copies.

Both these changes reduced path finding time by ~5% and memory usage by
~2mb.
2019-10-24 21:38:07 -03:00
Juan Pablo Civile
fc36df0e60 routing: avoid unneeded map access
`processEdge` basically had 4 expensive operations: 3 map accesses and
updating the heap. This removes one of those for a small performance
gain.
2019-10-24 21:31:30 -03:00
Juan Pablo Civile
3e60a23632 routing: pre-allocate the distance map to an estimated node count
Pre-sizing these structures avoids a lot of map resizing, which causes
copies and rehashing of entries. We mostly know that the map won't
exceed that size, and it doesn't affect memory usage in any significant
way.
2019-10-24 21:31:30 -03:00
Juan Pablo Civile
2141713936 routing: avoid walking all nodes for path finding if we don't need to
Calling `ForEachNode` hits the DB, and allocates and parses every node
in the graph. Walking the channels also loads nodes from the DB, so this
meant that each node was read/parsed/allocated several times per run.

This reduces runtime by ~10ms and memory usage by ~4mb.
2019-10-24 21:31:30 -03:00
Juan Pablo Civile
5389161162 routing: make log in findPath hot path use logClosure
It generates heap allocations for it's params even if it won't end up
using them.

Reduces memory usage by 2mb
2019-10-24 21:31:30 -03:00
Joost Jager
1fac41deed
routing+routerrpc: improve prob. estimation for untried connections
This commit changes mission control to partially base the estimated
probability for untried connections on historical results obtained in
previous payment attempts. This incentivizes routing nodes to keep all
of their channels in good shape.
2019-10-22 15:52:38 +02:00
Joost Jager
fab13900e2
routing+routerrpc: remove probability estimates from mc state snapshot
Probability estimates are amount dependent. Previously we assumed an
amount, but that starts to make less sense when we make probability more
dependent on amounts in the future.
2019-10-22 15:52:36 +02:00
Joost Jager
559d3c0910
routing: also fail pairs for node-level failures
This commit modifies the interpretation of node-level failures.
Previously only the failing node was marked. With this commit, also the
incoming and outgoing connections involved in the route are marked as
failed.

The change prepares for the removal of node-level failures in mission
control probability estimation.
2019-10-22 15:52:34 +02:00
Joost Jager
843c87a203
routing: create mc getWeight method 2019-10-22 15:52:32 +02:00
Joost Jager
a3f7dbc633
routing: change representation of pair results in mc
This commit changes the in-memory structure of the mission control
state. It prepares for calculation of a node probability. For this we
need to be able to efficiently look up the last results for all channels
of a node.
2019-10-22 15:52:30 +02:00
Joost Jager
aefbee78d6
routing: use pairResult constructors
To make it explicit whether a failure or a success result is
instantiated.
2019-10-22 15:52:28 +02:00
Joost Jager
37ef46bc48
routing/test: extract mission control test parameters 2019-10-22 15:52:26 +02:00
Johan T. Halseth
610a93e2ff
routing/chainview: check GetCFilter error
Since the filter will never be nil in non-error cases now, remove the
nil-check.
2019-10-15 10:31:25 +02:00
Wilmer Paulino
0fc401de19
routing+routerrpc: take max cltv limit into account within path finding
With the introduction of the max CLTV limit parameter, nodes are able to
reject HTLCs that exceed it. This should also be applied to path
finding, otherwise HTLCs crafted by the same node that exceed it never
left the switch. This wasn't a big deal since the previous max CLTV
limit was ~5000 blocks. Once it was lowered to 1008, the issue became
more apparent. Therefore, all of our path finding attempts now have a
restriction of said limit in in order to properly carry out HTLCs to the
network.
2019-10-11 18:04:49 -04:00
Olaoluwa Osuntokun
5ef1f85998
Merge pull request #3578 from wpaulino/no-freelist-sync-option
config: add sync freelist cli flag
2019-10-04 15:16:30 -07:00
Olaoluwa Osuntokun
c03fb8aca0
Merge pull request #3561 from Roasbeef/chan-validation-cleanup
chanvalidate: create new channel validation package
2019-10-04 12:32:07 -07:00
Wilmer Paulino
194a9dea81
multi: support sync freelist option within btcwallet 2019-10-04 12:19:40 -04:00
Olaoluwa Osuntokun
6ebada112f
routing: update 3d party channel verification to use new chanvalidate package
In the process of moving to use the new package, we no longer need to
fetch the outpoint directly, and instead only need to pass the funding
transaction into the new verification logic.
2019-10-03 16:23:17 -07:00
Wilmer Paulino
5ed1084130
Merge pull request #3572 from joostjager/canceled-spelling
multi: fix canceled spelling
2019-10-03 16:50:06 -04:00
Joost Jager
b58dbb2d70
multi: fix canceled spelling 2019-10-03 17:27:36 +02:00
Joost Jager
9f3656efeb
routing: log performance metrics 2019-10-03 15:07:11 +02:00
Olaoluwa Osuntokun
18f88cbd8d
Merge pull request #3440 from joostjager/buildroute
routing: add build route functionality
2019-09-24 20:24:24 -07:00
Joost Jager
299821152a
routing+routerrpc: add BuildRoute function 2019-09-24 10:00:44 +02:00
Valentine Wallace
9a52cb6dab
multi: update internals to support updating max htlc
In this commit, we update the router and link to support users
updating the max HTLC policy for their channels. By updating these internal
systems before updating the RPC server and lncli, we protect users from
being shown an option that doesn't actually work.
2019-09-23 13:07:10 +02:00
Joost Jager
c80feeb4b3
routing+discovery: extract local channel manager
The policy update logic that resided part in the gossiper and
part in the rpc server is extracted into its own object.

This prepares for additional validation logic to be added for policy
updates that would otherwise make the gossiper heavier.

It is also a small first step towards separation of our own channel data
from the rest of the graph.
2019-09-23 13:07:08 +02:00
Joost Jager
fd63ed13ff
routing/route: add vertex constructor for hex string 2019-09-17 11:11:00 +02:00
Joost Jager
8c44cf4a22
channeldb: add ComputeFee function 2019-09-17 11:10:58 +02:00
Joost Jager
f60e4b1e14
lnwire+htlcswitch: report height for invalid payment details failure
Extends the invalid payment details failure with the new accept height
field. This allows sender to distinguish between a genuine invalid
details situation and a delay caused by intermediate nodes.
2019-09-16 10:10:20 +02:00
Lars Lehtonen
3b29ecb921 routing: Fix dropped errors in tests.
lnd: Fix dropped errors in tests.

contractcourt: Fix dropped errors in tests.

htlcswitch: Fix dropped errors in tests.

invoices: Fix dropped error in tests.

lnwallet: Fix dropped errors in tests.

macaroons: Fix dropped error in tests.
2019-09-13 06:28:05 +00:00
Joost Jager
3d7de2ad39
multi: remove dead code 2019-09-10 17:21:59 +02:00
Conner Fromknecht
947cc50c13
routing/route/route: prevent modification of hop.TLVRecords
Currently the underlying array backing the hop's TLVRecords is modified
when combining custom records with the primitive forwarding info. This
commit uses a fresh slice to prevent modifications from mutating the
hop itself.
2019-09-04 14:14:34 -07:00
Conner Fromknecht
278e10a2fd
routing/route/route: omit next_hop_id for final hop
BOLT04 says to omit this field for final hops, but must be present on
all other hops.
2019-09-04 12:05:10 -07:00
Conner Fromknecht
dc0f429271
htlcswitch+routing: use TLV hop constructors 2019-09-04 12:05:09 -07:00
Joost Jager
ff0c5a0d5e
routing: process successes in mission control
This commit modifies paymentLifecycle so that it not only feeds
failures into mission control, but successes as well.
This allows for more accurate probability estimates. Previously,
the success probability for a successful pair and a pair with
no history was equal. There was no force that pushed towards
previously successful routes.
2019-08-23 09:15:41 +02:00
Joost Jager
2594abfba1
routing/test: remove unused parameter in mission control test 2019-08-23 09:15:39 +02:00
Olaoluwa Osuntokun
4697cfde30
routing: extend path finding to be TLV-EOB aware, allow dest TLV records
In this commit, we extend the path finding to be able to recognize when
a node needs the new TLV format, or the legacy format based on the
feature bits they expose. We also extend the `LightningPayment` struct
to allow the caller to specify an arbitrary set of TLV records which can
be used for a number of use-cases including various variants of
spontaneous payments.
2019-08-22 18:53:01 -07:00
Olaoluwa Osuntokun
5b4c8ac232
routing/route+tlv: add new TLV-EOB awareness to Hop+Route
In this commit, we extend the Hop struct to carry an arbitrary set of
TLV values, and add a new field that allows us to distinguish between
the modern and legacy TLV payload.

We add a new `PackPayload` method that will be used to encode the
combined required routing TLV fields along any set of TLV fields that
were specified as part of path finding.

Finally, the `ToSphinxPath` has been extended to be able to recognize if
a hop needs the modern, or legacy payload.
2019-08-22 18:53:00 -07:00
Joost Jager
d9ec158412
routing: stricter payment result interpretation
This commit overhauls the interpretation of failed payments. It changes
the interpretation rules so that we always apply the strongest possible
set of penalties, without making assumptions that would hurt good nodes.

Main changes are:

- Apply different rule sets for intermediate and final nodes. Both types
of nodes have different sets of failures that we expect. Penalize nodes
that send unexpected failure messages.

- Distinguish between direct payments and multi-hop payments. For direct
payments, we can infer more about the performance of our peer because we
trust ourselves.

- In many cases it is impossible for the sender to determine which of
the two nodes in a pair is responsible for the failure. In this
situation, we now penalize bidirectionally. This does not hurt the good
node of the pair, because only its connection to a bad node is
penalized.

- Previously we always penalized the outgoing connection of the
reporting node. This is incorrect for policy related failures. For
policy related failures, it could also be that the reporting node
received a wrongly crafted htlc from its predecessor. By penalizing the
incoming channel, we surely hit the responsible node.

- FailExpiryTooSoon is a failure that could have been caused by any node
up to the reporting node by delaying forwarding of the htlc. We don't
know which node is responsible, therefore we now penalize all node pairs
in the route.
2019-08-17 10:24:09 +02:00
Joost Jager
e7a457f1ce
routing: query bandwidth hints before each payment attempt
Previously the bandwidth hints were only queried once per payment. This
did not allow for concurrent payments changing channel balances.
2019-08-17 10:24:07 +02:00
Joost Jager
e135cf7326
routing: penalize all node pairs for unknown outcomes
When an undecryptable failure comes back for a payment attempt, we
previously only penalized our own outgoing connection. However,
any node could have caused this failure. It is therefore better to
penalize all node connections along the route. Then at least we know for
sure that we will hit the responsible node.
2019-08-17 10:24:03 +02:00
Joost Jager
c39d7a29cd
routing/test: do not test local channel mission control
This commit updates existing tests to not rely on mission control for
pruning of local channels. Information about local channels should
already be up to date before path finding starts. If not, the problem
should be fixed where bandwidth hints are set up.
2019-08-17 10:24:01 +02:00
Joost Jager
45dacd0df1
routing: isolate failure interpretation from mission control
This commit moves the payment outcome interpretation logic into a
separate file. Also, mission control isn't updated directly anymore, but
results are stored in an interpretedResult struct. This allows the
mission control state to be locked for a minimum amount of time and
makes it easier to unit test the result interpretation.
2019-08-17 10:23:59 +02:00
Joost Jager
e7af6a077a
routing: convert to nillable failure reason
This commit converts several functions from returning a bool and a
failure reason to a nillable failure reason as return parameter. This
will take away confusion about the interpretation of the two separate
values.
2019-08-17 10:23:57 +02:00
Joost Jager
f1769c8c8c
routing: convert to node pair based
Previously mission control tracked failures on a per node, per channel basis.
This commit changes this to tracking on the level of directed node pairs. The goal
of moving to this coarser-grained level is to reduce the number of required
payment attempts without compromising payment reliability.
2019-08-13 19:21:37 +02:00
Joost Jager
a332990d2c
routing/route: add vertex constructor from bytes 2019-08-13 18:45:08 +02:00
Joost Jager
b610f417d2
routing: extended DirectedNodePair functionality 2019-08-13 18:45:06 +02:00
Joost Jager
2b4debf42b
routing/test: remove unused methods from mock 2019-08-13 18:45:02 +02:00
Joost Jager
aabd68ebcd
multi: rename FailUnknownPaymentHash to FailIncorrectDetails
Align naming better with the lightning spec. Not the full name of the
failure (FailIncorrectOrUnknownPaymentDetails) is used, because this
would cause too many long lines in the code.
2019-08-08 11:04:17 +02:00
Joost Jager
7e7b620355
routing: persist mission control data 2019-07-31 08:44:00 +02:00
Joost Jager
9e26f0d079
routing: define payment result
This commit groups together all payment result data. It is a preparation
for historical payment results being retrieved from the database.
2019-07-29 09:38:36 +02:00
Joost Jager
1507ff6590
routing: extract timestamp from applying payment result
This commit prepares for timestamps being supplied from the database for
processing of historical payment results.
2019-07-29 09:38:34 +02:00
Joost Jager
b4a7665bae
routing: provide payment id to mission control 2019-07-29 09:38:32 +02:00
Joost Jager
334b6a3bfe
routing: move unreadable failure handling into mission control 2019-07-29 09:38:30 +02:00
Joost Jager
934ea8e78d
routing: move failure interpretation into mission control 2019-07-29 09:38:26 +02:00
nsa
7762d55e80 itest+routing: updating tests to account for BlockPadding 2019-07-23 21:03:30 -04:00
nsa
3dde7bc4e2 routing: adding the BlockPadding value to sendpayment
This commit adds the BlockPadding value (currently 3) to sendpayment
calls so that if some blocks are mined while the htlc is in-flight, the
exit hop won't reject it.
2019-07-23 21:03:11 -04:00
Olaoluwa Osuntokun
8c389d13f9
Merge pull request #3197 from breez/optimize_prune_zombie_channels
Optimize prune zombie channels
2019-07-18 20:56:56 -07:00
Roei Erez
da9edc876a router: only prune disabled channels when AssumeChannelValid=true. 2019-07-18 17:05:20 +03:00
Joost Jager
34b264a3d8
routing: create PathFindingConfig for global parameters 2019-07-18 15:49:25 +02:00
Olaoluwa Osuntokun
92e14af72a
Merge pull request #3298 from joostjager/globalize-mc
routing: globalize mc
2019-07-16 17:39:36 -07:00
Roei Erez
9781ea0082 router: Optimize pruneZombieChannels.
The current approach iterates all channels in the graph in order to
filter those in need. This approach is time consuming, several seconds
on my mobile device for ~40,000 channels, while during this time the
db is locked in a transaction.

The proposed change is to use an existing functionality that utilize the
fact that channel update are saved indexed by date. This method enables
us to go over only a small subset of the channels, only those that
were updated before the "channel expiry" time and further filter
them for our need.
The same graph that took several seconds to prune was pruned, after
the change, in several milliseconds.

In addition for testing purposes I added Initiator field to the
testChannel structure to reflect the channeldEdgePolicy direction.
2019-07-16 17:48:07 +03:00
Joost Jager
8055bcf2e0
routing: report failures to mission control directly
As there is no more state kept in the payment session, failure reporting
can go straight to mission control.
2019-07-13 22:38:25 +02:00
Joost Jager
dc13da5abb
routing: move second chance logic into mission control
If nodes return a channel policy related failure, they may get a second
chance. Our graph may not be up to date. Previously this logic was
contained in the payment session.

This commit moves that into global mission control and thereby removes
the last mission control state that was kept on the payment level.

Because mission control is not aware of the relation between payment
attempts and payments, the second chance logic is no longer based
tracking second chances given per payment.

Instead a time based approach is used. If a node reports a policy
failure that prevents forwarding to its peer, it will get a second
chance. But it will get it only if the previous second chance was
long enough ago.

Also those second chances are no longer dependent on whether an
associated channel update is valid. It will get the second chance
regardless, to prevent creating a dependency between mission control and
the graph. This would interfer with (future) replay of history, because
the graph may not be the same anymore at that point.
2019-07-13 22:38:23 +02:00
Joost Jager
d31efddf1b
routing: move generateBandwidthHints 2019-07-13 22:38:21 +02:00
Joost Jager
37e2751695
routing+routerrpc: isolate payment session source from mission control 2019-07-13 22:38:19 +02:00
Joost Jager
cf3dd3fb94
routing/test: create mission control test context 2019-07-13 22:38:17 +02:00
nsa
7e6a34532e routing: add index map to distanceHeap
This commit adds the pubkeyIndices map to the distanceHeap to avoid
duplicate entries on the heap. This happened in the earlier iteration
of the findPath algorithm and would cause the driving loop to
evaluate already evaluated entries when there was no need.
2019-07-12 06:08:01 -04:00
nsa
61a1ab08fb routing+channeldb: findPath uses Vertex instead of LightningNode
This commit modifies the nodeWithDist struct to use a route.Vertex
instead of a *channeldb.LightningNode. This change, coupled with
the new ForEachNodeChannel function, allows the findPath Djikstra's
algorithm to cut down on database lookups since we no longer need
to call the FetchOtherNode function.
2019-07-12 06:08:01 -04:00