Now that the HTLC second-level transactions are going through the
sweeper instead of the nursery, there are a few things we must account
for.
1. The sweeper sweeps the CSV locked HTLC output one block earlier than
the nursery.
2. The sweeper aggregates several HTLC second levels into one
transaction. This also means it is not enough to check txids of the
transactions spent by the final sweep, but we must use the actual
outpoint to distinguish.
In case of anchor channel types, we mine one less block before we expect
the second level sweep to appear in the mempool, since the sweeper
sweeps one block earlier than the nursery.
Since the tests set a quite high fee rate before the node goes to chain,
the HTLCs wouldn't be economical to sweep at this fee rate.
Pre sweeper handling of the second-level transactions this was not a
problem, since the fees were set when the second-levels were created,
before the fee estimate was increased.
To avoid running into the "server is still starting" error when trying
to close a channel, we first wait for the error to disappear before we
try closing the actual channel.
This commit replaces most of the hard coded 10, 15, 20 and 30 second
timeouts with the default timeout. This should allow darwin users to
successfully run the parallel itests locally as well.
In high CPU usage scenarios such as our parallel itests, it seems that
some goroutines just don't get any CPU time before our test timeouts
expire. By polling 10 times less frequently, we hope to reduce the
overall number of goroutines that are spawned because of the RPC
requests within the polling code.
This ensures that the nodes will properly be shutdown even if one fails
to start or any of them fail to connect. Previously the shutdown is
defered only in the event that the setup was successful.
Certain checks were implemented with Errorf, which only logs the
failure. This results in the test harness panicking further down. We go
further ahead and convert all calls in this file to use require.
With the new btcd version we can specify our own listen address
generator function for any btcd nodes. This should reduce flakiness as
the previous way of getting a free port was based on just picking a
random number which lead to conflicts.
We also double the default values for connection retries and timeouts,
effectively waiting up to 4 seconds in total now.
To avoid the "Error outside of test" log and to properly terminate the
test if a sub test fails, we need to correctly invoke them using the
RunTestCase method.
In this commit, we extend the `BuildRoute` method and RPC on the router
sub-server to accept a raw payment address which will be included as
part of an MPP payload for the finla hop. This change actually also
allows users to craft their own MPP paths using BuildRoute+SendToRoute.
Our primary goal however, was to fix some broken itests since we now
require the payAddr to be present for ALL payments other than key send
payments.
Currently trying to run etcd tests on darwin will cause the timeouts to
improperly select timeouts_darwin.go which are stricter than
timeouts_etcd.go. We fix this by always defaulting to timeouts_etcd.go
no matter the platform, and only falling back to timeouts_darwin.go if
the kvdb_etcd tag is not present.
In some tests we moved channeld.db to a temp location in order to
"time travel". This commit extends the existing semantics by moving all
files, including embedded etcd db too besides the channeld.db file.
Previously, the verbose output of listsweeps would fail if we did not
find some sweeps in our wallet's listtransactions output. This could be
the case for sweeps that were rbf-ed, so the endpoint would fail. This
commit also updates the listsweeps endpoint to always check against the
wallet, so that we do not return these discarded sweeps that never
confirmed.
This commit adds the icase name to the log filename, to make it simpler
to find problematic tests. Additionally after this commit we'll restart
Alice and Bob (the base harness nodes) before each icase to start with a
clean state.
To make sure the test that takes the longest overall time is always
started first, independent of the number of test tranches we run, we
move it to the beginning of the list. Because that test involves a lot
of waiting, it allows us to play around with the number of tranches more
efficiently.
Updating the fee of the mock estimator _after_ starting carol turned out
to be flaky and could lead to the new fee not being picked up in time
for the force close. That lead to carol not cpfp'ing the force closed
transaction.
To allow running multiple test tranches in parallel, we need a way to
make sure the TCP ports don't collide. We'll work with offsets for the
ports, using a different offset for each tranche.
This fixes tests that were surfaced as flaky. Usually these tests had
an off-by-one error when mining blocks while waiting for a
CSV-encumbered output.
As a preparation to fix an issue with the mempool wait, we clean up the
multi-hop itests a bit. We fix the formatting, use the require library
for assertions consistently and simplify some of the wait predicates.
Commonly used code is also extracted into functions.
In some cases the router isn't yet fully aware of all newly opened
channels. We need to give it some time to process the updates. Therefore
we add a wait for sending payments to give it a few more changes to
catch up.
This fixes itest flakes that happen when a payment is attempted that
ends up causing a channel closure.
completePaymentRequests() attempts to monitor the open channels after a
payment is attempted in order to identify that payment was actually
dispatched to a remote node before returning.
However, when the payment actually causes a channel closure (for
example, because the receiver sent an incorrect preimage) this logic
fails in that the channel will no longer the found in the list of open
channels. This could cause a flake when there was enough time for the
channel to close before performing the check.
One example of such a flaky test is failing_link.
This fixes the issue by also checking whether the total number of
channels was reduced, which indicates (assuming itest operations are
being executed serially) that one of the attempted payments affected at
least one channel.
This ensures the Carol and Dave nodes created in the single_hop series
of tests are synced to the latest chain tip generated by the miner
before creating the route that will eventually be used for payments.
This prevents a flake during CI tests where slow processing of blocks by
Carol could cause the generated route to have a final CLTV delta too
short for Dave to accept.
While at it, we switch the test to use the default CLTV delta provided
by QueryRoutes which is now the global default CLTV.
We create a new build flag for running the bitcoind tests without the
txindex enabled. We don't want this to be the default so we use a
negated build flag.
Now that we have all functions that we need to complete the whole
PSBT channel funding flow, we change the itest to use Dave's wallet
to fund the channel from Carol to Dave through a PSBT.
We fix all linter issues except for the 'lostcontext' and 'unparam' ones
as those are too numerous and would increase the diff even more.
Therefore we silence them in the itest directory for now.
Because the linter is still not build tag aware, we also have to silence
the unused and deadcode sub linters to not get false positives.
In this commit, we modify our build tag set up to allow the main test
files to be buildable w/o the current rpctest tag. We do this so that
those of us that use extensions which will compile live files like
vim-go can once again fix compile errors as we go in our editors.
In order to do this, we now make an external `testsCases` variable, and
have two variants: one that's empty (no build tag), and one that's fully
populated with all our tests (build tag active). As a result, the main
file will now always build regardless of if the build tag is active or
not, but we'll only actually execute tests if the `testCases` variable
has been populated.
As sample run w/ the tag off:
```
=== RUN TestLightningNetworkDaemon
--- PASS: TestLightningNetworkDaemon (0.00s)
PASS
ok github.com/lightningnetwork/lnd/lntest/itest 0.051s
```
For unconfirmed commit tx anchors, supply the sweeper with cpfp info and
a confirmation target fee estimate.
The sweeper will try to pay for the parent commit tx as long as the
current fee estimate exceeds the pre-signed commit tx fee rate.
As we already create two channels in our PSBT funding flow itest we can
easily just submit the final transaction for the second channel in the
raw wire format to test this new functionality.
- let users specify their MAXIMUM WUMBO with new config option which sets the maximum channel size lnd will accept
- current implementation is a simple check by the fundingManager rather than anything to do with the ChannelAcceptor
- Add test cases which verify that maximum channel limit is respected for wumbo/non-wumbo channels
- use --maxchansize 0 value to distinguish set/unset config. If user sets max value to 0 it will not do anything as 0 is currently used to indicate to the funding manager that the limit should not be enforced. This seems justifiable since --maxchansize=0 doesn't seem to make sense at first glance.
- add integration test case to ensure that config parsing and valiation is proper. I simplified the funding managers check electing to rely on config.go to correctly parse and set up either i) non wumbo default limit of 0.16 BTC OR ii) wumbo default soft limit of 10 BTC
Addresses: https://github.com/lightningnetwork/lnd/issues/4557
Previously the chainbackend connected to the miner using a permanent
connection, which would retry the connection when it’s disconnected.
It would leave multiple connections between the chainbackend and the
miner, causing difficulties in debugging. Currently, there are two
occasions when disconnection happens. One happens when running the open
channel reorg test, the miner is connected/disconnected multiple times
on purpose. The other happens when the chainbackend receives a
MSG_WITNESS_TX type from the miner, which would be considered as an
invalid type and disconnects the miner. With the latter one being fixed
in btcd, the chainbackend will still be disconnected from the miner if
it reaches the ban score by requesting too many notfound messages in a
short time which is why the `--nobanning` flag is added.
To fix the compiler of some IDEs complaining about types and functions
it cannot find, we rename all files that contain tests back to lnd_xxx_test.go to make
sure they are compiled correctly.
This commit moves all localized instances of mock implementations of
the Signer interface to the lntest/mock package. This allows us to
remove a lot of code and have it housed under a single interface in
many cases.
We change the external funding test to now test two more things: First
that we can open multiple externally funded channels without needing to
lift the default --maxpendingchannels setting. Then we test that we can
use the safer pending_funding_shim_only flag of the AbandonChannel RPC
to get rid of the never confirming external channels.
As a preparation to test accepting multiple externally funded channels
at the same time, we extract the deriveFundingShim function from the
external funding integration test.
Follow up labelling of external transactions with labels for the
transaction types we create within lnd. Since these labels will live
a life of string matching, a version number and rigid format is added
so that string matching is less painful. We start out with channel ID,
where available, and a transaction "type". External labels, added in a
previous PR, are not updated to this new versioned label because they
are not lnd-initiated transactions. Label matching can check this case,
then check for a version number.
This reduces the flakiness of the CPFP test by asserting the wallet has
seen the unspent output before attempting to perform the walletkit's
BumpFee method.
Previously the attempt to bump the fee of the target transaction could
be made before the wallet had had a chance to fully process the
transaction, causing a flaky error.
This switches a few call sites that used a different timeout when
openening channels to the correct openChannelTimeout, which better deal
with flakes in the CI.
