Commit Graph

807 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
7ab5906093
Merge pull request #5245 from bhandras/kvdb_module
kvdb: make kvdb a top level submodule to allow dependency in other projects
2021-05-13 15:59:20 -07:00
eugene
a70d0bef34
multi: validate payment params at RPC layer
With this patch, we'll fail out earlier in the cycle in case of
some wonky parameters, and not leave zombie payments in the router
which currently are not cleaned up.
2021-05-13 17:17:52 -04:00
Conner Fromknecht
8c404ade18
routing: allow splitting for AMP or MPP 2021-05-10 22:02:14 -07:00
Oliver Gugger
6c37cae639
lntest+routing: update best height after graph pruning
It seems #5246 introduced a subtle bug that lead to the error "out of
order block: expecting height=1, got height=XXX" some times during
startup. Apparently it can happen that during pruning of the graph tip
some blocks can come in before we start our chain view and the new block
subscription. By querying the chain backend for the best height before
syncing with the graph we ensure that we never miss a block.
2021-05-10 16:46:28 +02:00
Oliver Gugger
9f0228b21c
router: add SyncedHeight() method
The router subsystem has its own goroutine that receives chain updates
and then does its (quite time consuming) work on each new block. To make
it possible to find out what block the router currently is synced to, we
export its internal best height through a new method.
2021-05-10 16:46:27 +02:00
Andras Banki-Horvath
14c851c8fc
kvdb: move channeldb/kvdb to top level 2021-05-07 14:18:56 +02:00
Olaoluwa Osuntokun
4685341dcb
Merge pull request #5227 from Roasbeef/spent-channs-zombie-index
routing: add chans rejected due to failed chain validation to zombie index
2021-05-05 18:32:13 -07:00
Olaoluwa Osuntokun
9d9f420da5
Merge pull request #5246 from joostjager/router-best-height
routing: no backend call to fetch best block
2021-05-04 19:27:37 -07:00
Olaoluwa Osuntokun
e0ce384f02
routing: add new error for failed funding tx validation
In this commit we add a new error for when we fail to validate the
funding transaction (invalid script, etc) and mark it as a zombie like
the other failed validation cases.
2021-05-03 19:06:59 -07:00
Olaoluwa Osuntokun
92c47983cb
routing: add chans rejected due to failed chain validation to zombie index
In this commit, we start to add any channels that fail the normal chain
validation to the zombie index. With this change, we'll ensure that we
won't continue to re-process the same set of spent channels over and
over again.

Fixes #5191.
2021-05-03 19:06:55 -07:00
Olaoluwa Osuntokun
897a19d9df
routing: add new error for invalid funding tx rejection 2021-05-03 19:06:52 -07:00
Olaoluwa Osuntokun
0dc6f8058d
routing: add new error for spent channel UTXOs 2021-05-03 19:06:50 -07:00
Wilmer Paulino
eb4413154d
routing: return err on validation failure of network update
This ensures the waiting receiving channel always receives an error to
prevent a deadlock when processing a network update that fails due to
the validation barrier.

On commit d5aedbcbd9db510c974c9f7be5ab177ad6546994:

1000 @ 0x43a285 0x44a38f 0xc42e86 0xc80fda 0xc8682d 0xc976c9 0x46fce1
github.com/lightningnetwork/lnd/routing.(*ChannelRouter).AddNode+0x245                      github.com/lightningnetwork/lnd/routing/router.go:2218
github.com/lightningnetwork/lnd/discovery.(*AuthenticatedGossiper).addNode+0x3b9            github.com/lightningnetwork/lnd/discovery/gossiper.go:1510
github.com/lightningnetwork/lnd/discovery.(*AuthenticatedGossiper).processNetworkAnnouncement+0x574c github.com/lightningnetwork/lnd/discovery/gossiper.go:1554
github.com/lightningnetwork/lnd/discovery.(*AuthenticatedGossiper).networkHandler.func1+0x24github.com/lightningnetwork/lnd/discovery/gossiper.go:1043
2021-04-29 15:52:24 -07:00
Elle Mouton
6ad5781bf1 routing: add block cache to CfFilteredChainView
This commit adds the block cache to the CfFilteredChainView struct
and wraps its GetBlock function so that block cache mutex map is used
when the call to neutrino's GetBlock function is called.
2021-04-28 09:46:11 +02:00
Elle Mouton
f470946379 routing: add block cache to BtcdFilteredChainView
This commit makes the block cache available to BtcdFilteredChainView and
wraps its GetBlock method so that the block cache is used.
2021-04-28 09:46:11 +02:00
Elle Mouton
0193669ed8 routing: add block cache to BitcoindFilteredChainView
This commit adds the block cache to the BitcoindFilteredChainView struct
and wraps its GetBlock function so that block cache is used.
2021-04-28 09:46:11 +02:00
Joost Jager
cdddaaca75 routing: no backend call to fetch best block 2021-04-27 19:05:55 +02:00
Johan T. Halseth
c4fc72d573
routerrpc+routing: set AMP options for payments specified as AMP in SendPayment 2021-04-27 09:47:23 +02:00
Johan T. Halseth
56a2c65477
router: use SetID as identifier for SendToRoute AMP-payments 2021-04-27 09:47:23 +02:00
Johan T. Halseth
f07c9d002c
routing: use Identifier in place of PaymentHash
Since we want to support AMP payment using a different unique payment
identifier (AMP payments don't go to one specific hash), we change the
nomenclature to be Identifier instead of PaymentHash.
2021-04-27 09:47:23 +02:00
Johan T. Halseth
5531b812e3
routing: use AMP shard tracker
We'll use the AMP-specific ShardTracker for payments having non-nil
AMPOptions.
2021-04-27 09:47:23 +02:00
Johan T. Halseth
e1399fb1ec
routing/router: use attempt's unique hash if set on restart 2021-04-27 09:45:13 +02:00
Johan T. Halseth
06f045fca3
channedb/mp_payment: add Hash to individual HTLCs
For AMP payments the hash used for each HTLC will differ, and we will
need to retrive it after a restart. We therefore persist it with each
attempt.
2021-04-27 09:44:19 +02:00
Johan T. Halseth
41ae3530a3
routing/payment_lifecycle: use ShardTracker to track shards
We'll let the payment's lifecycle register each shard it's sending with
the ShardTracker, canceling failed shards. This will be the foundation
for correct AMP derivation for each shard we'll send.
2021-04-27 09:43:40 +02:00
Johan T. Halseth
6474b253d6
routing/shards: add ShardTracker interface
We'll use this to keep track of the outstanding shards and which
preimages we are using for each. For now this is a simple map from
attempt ID to hash, but later we'll hide the AMP child derivation behind
this interface.
2021-04-27 08:27:33 +02:00
Johan T. Halseth
a9f19b100b
router+switch: rename paymentID->attemptID
To distinguish the attempt's unique ID from the overall payment
identifier, we name it attemptID everywhere, and note that the
paymentHash argument won't be the actual payment hash for AMP payments.
2021-04-27 08:27:33 +02:00
Johan T. Halseth
7795353e9f
channeldb: return full payment for inflight payments
We might as well return all info, and we'll need the individual HTLCs
in later commits.
2021-04-27 08:27:32 +02:00
carla
b43ddfdb11
routing: label payment lifecycle loop 2021-04-23 08:51:07 +02:00
carla
a63640c488
routing: account for payment terminal errors
If we have processed a terminal state while we're pathfinding
for another shard, the payment loop should not error out on
ErrPaymentTerminal. Instead, it would wait for our shards to
complete then cleanly exit.
2021-04-23 08:46:22 +02:00
carla
58d95be4dd
multi: change RegisterAttempt error checking order
Move our more generic terminal check forward so that we only
need to handle a single class of expected errors. This change
is mirrored in our mock, and our reproducing tests are updated
to assert that this move catches both classes of errors we get.
2021-04-23 08:39:45 +02:00
carla
12136a97a9
routing/test: add test for stuck payment with in-flight htlcs
Add an additional stuck-payment case, where our payment gets
a terminal error while it has other htlcs in-flight, and a
shard fails with ErrTerminalPayment. This payment also falls in
our class of expected errors, but is not currently handled. The
mock is updated accordingly, using the same ordering as in our
real RegisterAttempt implementation.
2021-04-23 08:39:45 +02:00
carla
80451afe48
routing/test: add test to demonstrate stuck payment, single shard
This commit adds a test which demonstrates that payments can
get stuck if we receive a payment failure while we're pathfinding
for another shard, then try to dispatch a shard after we've
recorded a permanent failure. It also updates our mock to
only consider payments with no in-flight htlcs as in-flight,
to more closely represent our actual RegisterAttempt.
2021-04-23 08:39:44 +02:00
carla
125980afb7
routing/test: block on pathfinding in tests
This commit adds a step to our payment lifecycle test to add
control over when we find a path for our payment, This is
required for testing race conditions around pathfinding
completing and payment failures being reported.
2021-04-23 08:39:43 +02:00
carla
a68155545c
routing/test: use half shard for single success case
Update our single shard success case to use a route which
splits the payment amount in half. This change still tests
the case where reveal of the preimage counts as a success,
even if we don't have the full amount. This change is made
to cut down on potential races in this test case. While we
are waiting for collectResultAsync to report a success, the
payment lifecycle will continue trying to dispatch shards.
In the case where we send 1/4 of the payment amount, we
send 1 or 2 more shards, depending on how long collectAsync
takes. Reducing this test to send 1/2 of the payment amount
means that we will always only try one more shard before
waiting for our shard.
2021-04-23 08:39:42 +02:00
carla
198d567cb2
routing/test: assert error value for payment failures 2021-04-23 08:39:41 +02:00
carla
e0c52e4473
routing/test: close payment result channel on shutdown, mimicking switch
This commit updates our mock to more closely follow the behavior of the
switch for mocked calls to GetPaymentResult. As it stands, our tests
send a test-created error from the switch when we want to mock shutdown.
In reality, the switch will close its result channel, so we update this
test to follow that behavior. This matters for the commit that follows,
because we start checking the error our payments return. If we have an
error from the switch, our tests will fail with an error that we do
not encounter in practice.
2021-04-23 08:39:40 +02:00
carla
9a78e9da73
routing/tests: move lock acquisition after state driving channels
In our payment lifecycle tests, we have two goroutines that
compete for the lock in our mock control tower: the resumePayment
loop which tries to call RegisterAttempt, and the collectResult
handler which is launched in a goroutine by collectResultAsync
and is responsible for various settle/fail calls.

The order that the lock is acquired by these goroutines is
arbitrary, and can lead to flakes in our tests if the step
that we do not intend to execute first gets the lock (eg,
we want to fail a payment in collectResult, but RegisterAttempt
gets there first). This commit moves contention for this lock
after our mock's various "state driving" channels, so that the
lock will be acquired in the order that the test intends it.
2021-04-23 08:39:39 +02:00
carla
b2d941ebfb
routing/test: remove test channel buffers
Now that we run each test individually, we don't need to buffer
our mock's channels anymore. This helps to tighten our test loop,
which currently can move on from a step before it's actually
been processed by the mock. This removal ensures that our payment
loop processes each of the test's steps before moving on to the
next once.
2021-04-23 08:39:38 +02:00
carla
806c4cbd57
routing/test: run each test case individually, add names
Update our payment lifecycle test to run each test case with
a fresh router. This prevents test cases from interacting with
each other. Names are also added for easy debugging.
2021-04-23 08:39:37 +02:00
carla
cb927e89b0
routing/test: add check that sendpayment completes
As is, we don't check that our SendPayment call in
TestRouterPaymentStateMachine completes. This makes it easier
to create malformed tests that just run through steps but leave
the SendPayment call hanging. This commit adds a check that we
have completed our payment to help catch tests like this. We
also remove an unused quit channel.
2021-04-23 08:39:36 +02:00
Olaoluwa Osuntokun
7b589e5811
routing: add strict zombie pruning as a config level param
In this commit, we add strict zombie pruning as a config level param.
This allow us to add the option for those that want a tighter graph, and
not change the default composition of the channel graph for most users
over night.

In addition, we expand the test case slightly by testing that the self
node won't be pruned, but also that if there's a node with only a single
known stale edge, then both variants will prune that edge.
2021-04-21 13:56:27 -05:00
Olaoluwa Osuntokun
916059da48
routing: update chan pruning test w/ new zombie logic 2021-04-21 13:56:24 -05:00
Conner Fromknecht
e3a8b3b0c4
routing/router: prune zombies when either end is stale 2021-04-21 13:56:21 -05:00
yyforyongyu
37aa49c7aa
routing: fix pathfind edge features being nil 2021-04-13 15:53:18 +08:00
Wilmer Paulino
82fe5d9bba
build: update btcwallet dependency introducing pruned bitcoind support
This is achieved by some recent work within the BitcoindClient enabling
it to retrieve pruned blocks from its server's peers.
2021-04-06 14:55:14 -07:00
Olaoluwa Osuntokun
ca96e66b43
Merge pull request #5116 from joostjager/mc-deadlock
routing: fix mission control deadlock
2021-04-05 20:02:57 -07:00
Wilmer Paulino
a620ce3682
build: update btcd and btcwallet dependencies 2021-04-05 15:41:04 -07:00
Olaoluwa Osuntokun
a329c80612
Merge pull request #5133 from wpaulino/routing-validation-cancel-deps
discovery+routing: cancel dependent jobs if parent validation fails
2021-04-01 18:32:58 -07:00
Johan T. Halseth
1231c90a19
routing: avoid open DB transaction if no zombies to prune
We add a simple length check to the channels to be pruned to avoid
opening the DB if there is nothing to be done.
2021-03-30 11:04:13 +02:00
Johan T. Halseth
a0f3624303
routing: delay initial zombie prune by 30 sec
Since zombie pruning can be very slow on some devices (e.g. mobile) it
would stall lnd startup. Since it is not essential for pruning to be
finished for lnd to be functional, we instead delay the initial prune by
30 seconds.

Note that we could also wait for the graphPruneInterval to tick, but
since this is by default 2 hours, it is unlikely that a mobile app will
ever be open that long.
2021-03-30 11:04:13 +02:00