Commit Graph

505 Commits

Author SHA1 Message Date
Oliver Gugger
af0f39f4a6
itest: fix log whitelist check
Because a previous PR changed the location of the itest log files, we
also need to update the script that looks for non-whitelisted errors.
2020-12-09 08:06:16 +01:00
Conner Fromknecht
2f38a7ce9c
itest: increase recovery payment amount so htlc isn't dust 2020-12-08 14:55:40 -08:00
Oliver Gugger
9de659ea4f
itest: use require.Eventually where tick speed matters
To not waste a lot of time with the adjusted longer tick interval, we
use a short interval with require.Eventually where we can save some
time.
2020-12-08 21:40:37 +01:00
Oliver Gugger
a436618e43
lntest: fix linter errors for changed code 2020-12-08 21:40:36 +01:00
Oliver Gugger
809d238364
itest: wait for server to start when restoring
To avoid running into the "server is still starting" error when trying
to close a channel, we first wait for the error to disappear before we
try closing the actual channel.
2020-12-08 21:40:31 +01:00
Oliver Gugger
cdcbc0376d
lntest: replace hard coded timeouts
This commit replaces most of the hard coded 10, 15, 20 and 30 second
timeouts with the default timeout. This should allow darwin users to
successfully run the parallel itests locally as well.
2020-12-08 21:37:12 +01:00
Oliver Gugger
3823315820
lntest: go easy on goroutines when polling
In high CPU usage scenarios such as our parallel itests, it seems that
some goroutines just don't get any CPU time before our test timeouts
expire. By polling 10 times less frequently, we hope to reduce the
overall number of goroutines that are spawned because of the RPC
requests within the polling code.
2020-12-08 21:37:11 +01:00
Oliver Gugger
4d2a12e552
itest: fix incorrect error message 2020-12-08 21:37:10 +01:00
Conner Fromknecht
123c3a2530
itest: defer shutdown of nodes in main test method
This ensures that the nodes will properly be shutdown even if one fails
to start or any of them fail to connect. Previously the shutdown is
defered only in the event that the setup was successful.
2020-12-03 23:06:32 -08:00
Conner Fromknecht
374725842e
itest: use fresh alice and bob for fwd interceptor tests 2020-12-03 23:06:32 -08:00
Conner Fromknecht
92b820a6d1
itest: move to require in forward_interceptor_test
Certain checks were implemented with Errorf, which only logs the
failure. This results in the test harness panicking further down. We go
further ahead and convert all calls in this file to use require.
2020-12-03 23:06:29 -08:00
Oliver Gugger
36756c012d
lntest: use nextAvailablePort for miner and btcd backend
With the new btcd version we can specify our own listen address
generator function for any btcd nodes. This should reduce flakiness as
the previous way of getting a free port was based on just picking a
random number which lead to conflicts.
We also double the default values for connection retries and timeouts,
effectively waiting up to 4 seconds in total now.
2020-12-03 11:30:23 +01:00
Oliver Gugger
2edc3cf98f
multi: update to latest btcd version
With this commit we update btcd to the latest version which allows us to
specify the btcd binary we want to use when spinning up a new node.
2020-12-03 11:30:22 +01:00
Oliver Gugger
0a4962a18e
itest: run sub itests correctly
To avoid the "Error outside of test" log and to properly terminate the
test if a sub test fails, we need to correctly invoke them using the
RunTestCase method.
2020-12-01 22:24:03 +01:00
Johan T. Halseth
8ec4697fe3
Merge pull request #4810 from Roasbeef/gossip-throttle-config
multi: add new config option to toggle gossip rate limiting
2020-12-01 13:32:52 +01:00
Olaoluwa Osuntokun
94f8311667
Merge pull request #4782 from cfromknecht/anchor-wtserver
watchtower: anchor channel support
2020-11-30 17:49:32 -08:00
Olaoluwa Osuntokun
447c9f2c0b
lntest: always turn off gossip throttling for nodes created in itests 2020-11-30 16:39:06 -08:00
Olaoluwa Osuntokun
7e298f1434
Merge pull request #3367 from cfromknecht/batched-graph-updates
Batched graph updates
2020-11-25 18:40:40 -08:00
Conner Fromknecht
82a238317c
lncfg+itest: expose configurable batch-commit-interval
This will permit a greater degree of tuning or customization depending
on various hardware/environmental factors.
2020-11-25 16:45:25 -08:00
Olaoluwa Osuntokun
4e079d1d20
lntest/itest: fix SendToRoute in UpdateChanPolicy test
It needs to include the MPP payload  now.
2020-11-25 16:32:15 -08:00
Olaoluwa Osuntokun
7fdf46ea4e
lntest/itest: update SendToRoute tests to include payment addr 2020-11-25 16:32:09 -08:00
Olaoluwa Osuntokun
d996607470
routing+lnrpc: extend BuildRoute to accept raw payAddr
In this commit, we extend the `BuildRoute` method and RPC on the router
sub-server to accept a raw payment address which will be included as
part of an MPP payload for the finla hop. This change actually also
allows users to craft their own MPP paths using BuildRoute+SendToRoute.
Our primary goal however, was to fix some broken itests since we now
require the payAddr to be present for ALL payments other than key send
payments.
2020-11-25 16:32:02 -08:00
Conner Fromknecht
b28bbc3223
wtclinet+itest: expose anchor client via rpc + add itest 2020-11-25 16:16:47 -08:00
Conner Fromknecht
f86c26878a
lntest/timeouts: fix darwin+kvdb_etcd build
Currently trying to run etcd tests on darwin will cause the timeouts to
improperly select timeouts_darwin.go which are stricter than
timeouts_etcd.go. We fix this by always defaulting to timeouts_etcd.go
no matter the platform, and only falling back to timeouts_darwin.go if
the kvdb_etcd tag is not present.
2020-11-18 15:28:39 -08:00
Conner Fromknecht
b70b6d9a6e
lntest/timeouts: remove spaces from build predicates
Otherwise this breaks various conditional compilation targets and the
linter.
2020-11-18 15:16:25 -08:00
Andras Banki-Horvath
5f3b800630
lint: fix error reported by the linter 2020-11-18 15:34:46 +01:00
Andras Banki-Horvath
efedb5547c
itest: timeouts for itets with etcd 2020-11-18 15:34:45 +01:00
Andras Banki-Horvath
369ae5e372
itest: move all test db files when using both etcd and bbolt
In some tests we moved channeld.db to a temp location in order to
"time travel". This commit extends the existing semantics by moving all
files, including embedded etcd db too besides the channeld.db file.
2020-11-18 15:34:45 +01:00
Andras Banki-Horvath
98342433ab
itest: add etcd flag to control if new lnd nodes should run on etcd 2020-11-18 15:34:44 +01:00
carla
887ff442db
multi: fix verbose output for listsweeps and check all against wallet
Previously, the verbose output of listsweeps would fail if we did not
find some sweeps in our wallet's listtransactions output. This could be
the case for sweeps that were rbf-ed, so the endpoint would fail. This
commit also updates the listsweeps endpoint to always check against the
wallet, so that we do not return these discarded sweeps that never
confirmed.
2020-11-13 13:59:52 +02:00
Andras Banki-Horvath
38e8184926
itest: update error log whitelist 2020-11-12 09:17:53 +01:00
Andras Banki-Horvath
9eac0dd3c9
itest: fix crash in parallel macaroon tests 2020-11-12 09:17:52 +01:00
Andras Banki-Horvath
979c8497b2
itest: add icase to node log filename + restart nodes before each icase
This commit adds the icase name to the log filename, to make it simpler
to find problematic tests. Additionally after this commit we'll restart
Alice and Bob (the base harness nodes) before each icase to start with a
clean state.
2020-11-12 09:17:52 +01:00
Andras Banki-Horvath
d6c2ee1bbf
itest: use require.NoError in TestLightningNetworkDaemon 2020-11-12 09:17:47 +01:00
Olaoluwa Osuntokun
99b0913562
Merge pull request #3672 from breez/invoice-hints
Allow user specify routing hints in private invoice.
2020-11-11 19:57:49 -08:00
Oliver Gugger
47e8882480
make+itest: allow the same test to be run in parallel 2020-11-10 13:57:31 +01:00
Oliver Gugger
24adf475ce
lnd_test: add integration tests for stateless init 2020-11-07 11:24:35 +01:00
Roei Erez
4463a4ce17 invoicerpc: Allow user specify routing hints. 2020-11-06 17:54:20 +02:00
Oliver Gugger
a606f462bc
itest: move longest test to beginning
To make sure the test that takes the longest overall time is always
started first, independent of the number of test tranches we run, we
move it to the beginning of the list. Because that test involves a lot
of waiting, it allows us to play around with the number of tranches more
efficiently.
2020-11-04 11:03:37 +01:00
Oliver Gugger
e6c47294fb
itest: fix typo and formatting 2020-11-04 11:03:36 +01:00
Oliver Gugger
ea4bb5dc5c
itest: fix chanbackup restore flake
Updating the fee of the mock estimator _after_ starting carol turned out
to be flaky and could lead to the new fee not being picked up in time
for the force close. That lead to carol not cpfp'ing the force closed
transaction.
2020-11-04 11:03:34 +01:00
Oliver Gugger
ca7564e4b4
itest: add flags for lnd executable 2020-11-04 11:03:31 +01:00
Oliver Gugger
f358a4474d
lntest: add log dir flag 2020-11-04 11:03:30 +01:00
Oliver Gugger
9bbf134237
itest: split tests into dynamic tranches 2020-11-04 11:03:29 +01:00
Oliver Gugger
891eb0d63d
lntest: use nextAvailablePort for bitcoind 2020-11-04 11:03:27 +01:00
Oliver Gugger
5c04449dfd
lntest: use nextAvailablePort for fee service 2020-11-04 11:03:26 +01:00
Oliver Gugger
a50d337e42
lntest: lower initial port, add ApplyPortOffset function
To allow running multiple test tranches in parallel, we need a way to
make sure the TCP ports don't collide. We'll work with offsets for the
ports, using a different offset for each tranche.
2020-11-04 11:03:25 +01:00
Eugene
225390ab5b
Merge pull request #4487 from Crypt-iQ/cc_pkg_0723
chainreg: new chainreg package
2020-10-30 06:07:26 -04:00
Eugene
bebe258e2d chainregistry+lnd: moving chainregistry+chainparams to new package 2020-10-29 15:24:14 -04:00
Matheus Degiovani
835c73632b
itest: fix off-by-one mining tests
This fixes tests that were surfaced as flaky.  Usually these tests had
an off-by-one error when mining blocks while waiting for a
CSV-encumbered output.
2020-10-28 14:48:15 -03:00
Oliver Gugger
098598ec74
itest: sort error whitelist 2020-10-28 14:35:11 +01:00
Oliver Gugger
fd684a8ffa
itest: add new errors to whitelist 2020-10-28 14:35:10 +01:00
Oliver Gugger
34439fbc2a
itest: cleanup multi-hop tests
As a preparation to fix an issue with the mempool wait, we clean up the
multi-hop itests a bit. We fix the formatting, use the require library
for assertions consistently and simplify some of the wait predicates.
Commonly used code is also extracted into functions.
2020-10-28 13:38:04 +01:00
Oliver Gugger
d1b46211d8
itest: replace WaitForTxBroadcast with WaitForTxInMempool 2020-10-28 13:29:18 +01:00
Oliver Gugger
1714394add
itest: fix send payment flake
In some cases the router isn't yet fully aware of all newly opened
channels. We need to give it some time to process the updates. Therefore
we add a wait for sending payments to give it a few more changes to
catch up.
2020-10-28 13:22:48 +01:00
Oliver Gugger
e6b0141b10
itest: fix typos, formatting 2020-10-28 13:22:48 +01:00
Matheus Degiovani
16a4687718
itest: Fix flakes when payments cause chan closure
This fixes itest flakes that happen when a payment is attempted that
ends up causing a channel closure.

completePaymentRequests() attempts to monitor the open channels after a
payment is attempted in order to identify that payment was actually
dispatched to a remote node before returning.

However, when the payment actually causes a channel closure (for
example, because the receiver sent an incorrect preimage) this logic
fails in that the channel will no longer the found in the list of open
channels. This could cause a flake when there was enough time for the
channel to close before performing the check.

One example of such a flaky test is failing_link.

This fixes the issue by also checking whether the total number of
channels was reduced, which indicates (assuming itest operations are
being executed serially) that one of the attempted payments affected at
least one channel.
2020-10-27 08:07:49 -03:00
Matheus Degiovani
473f730eec
itest: Sync nodes to chain during singleHop tests
This ensures the Carol and Dave nodes created in the single_hop series
of tests are synced to the latest chain tip generated by the miner
before creating the route that will eventually be used for payments.

This prevents a flake during CI tests where slow processing of blocks by
Carol could cause the generated route to have a final CLTV delta too
short for Dave to accept.

While at it, we switch the test to use the default CLTV delta provided
by QueryRoutes which is now the global default CLTV.
2020-10-26 16:29:39 -03:00
Oliver Gugger
3e3618ae9a
lntest: add build flag for disabling txindex on bitcoind
We create a new build flag for running the bitcoind tests without the
txindex enabled. We don't want this to be the default so we use a
negated build flag.
2020-10-09 13:35:04 +02:00
Oliver Gugger
b7fc7c3dae
lntest: use extraArgs, fix linter issues 2020-10-09 13:35:02 +02:00
Oliver Gugger
43c2031fa8
lntest: extract common bitcoind code into own file
To prepare for running multiple tests on bitcoind with different options
each time, we extract the common code into its own file.
2020-10-09 13:35:00 +02:00
Oliver Gugger
eb74011d5a
itest: add error to whitelist 2020-10-08 20:39:11 +02:00
Oliver Gugger
eb280fd248
itest: use new PSBT functions for funding flow
Now that we have all functions that we need to complete the whole
PSBT channel funding flow, we change the itest to use Dave's wallet
to fund the channel from Carol to Dave through a PSBT.
2020-10-03 10:34:56 +02:00
Oliver Gugger
f114fb3c8d
itest: use require library
We rewrite the test to use the require library to make it a
bit more condensed.
2020-10-03 10:34:56 +02:00
Oliver Gugger
f947576f33
lnwallet+mock: add new PSBT methods 2020-10-03 10:34:39 +02:00
Johan T. Halseth
ef0bca7a8e
Merge pull request #4660 from yyforyongyu/add-temp-miner-logs
itest: save temp miner's logs
2020-10-02 12:43:22 +02:00
Johan T. Halseth
1a73bc7d74
Merge pull request #4653 from LN-Zap/feat/spend-unconfirmed
Ability to spend unconfirmed coins when making onchain transactions
2020-10-02 08:40:45 +02:00
yyforyongyu
310f87e271
itest: save temp miner's logs 2020-10-02 00:04:47 +08:00
Tom Kirkpatrick
73a5f325b6
lnrpc: ability to spend unconfirmed coins 2020-10-01 13:27:03 +01:00
yyforyongyu
089e7557be
itest: update log error whitelist 2020-09-29 05:55:36 +08:00
Conner Fromknecht
68ddba7cb7
Merge pull request #4512 from yyforyongyu/channel-balance-verbose
display verbose result in ChannelBalance
2020-09-25 13:31:01 -07:00
Johan T. Halseth
74e62e5c5b
Merge pull request #4589 from yyforyongyu/itest-proper-cleanup
Itest: proper connect/disconnect miner
2020-09-25 10:31:56 +02:00
yyforyongyu
86779fb6a5
itest: fix code lint 2020-09-24 20:51:58 +08:00
yyforyongyu
48f6c47b17
itest: test all ChannelBalance fields 2020-09-24 20:33:23 +08:00
yyforyongyu
1b200c9930
itest: explicit checking ChannelBalance response 2020-09-24 20:30:51 +08:00
Oliver Gugger
9f7d8dd92e
lntest: make compilable without subserver build tags
To make it possible to compile the itests together with the other tests,
we don't want to use anything from the optional subservers.
2020-09-21 21:16:32 +02:00
Olaoluwa Osuntokun
719e32830d
lntest: fix most linter warnings, silence rest
We fix all linter issues except for the 'lostcontext' and 'unparam' ones
as those are too numerous and would increase the diff even more.
Therefore we silence them in the itest directory for now.
Because the linter is still not build tag aware, we also have to silence
the unused and deadcode sub linters to not get false positives.
2020-09-21 21:16:31 +02:00
Olaoluwa Osuntokun
c769247198
lntest: allow the main test files to be buildable w/o the rpctest build tag
In this commit, we modify our build tag set up to allow the main test
files to be buildable w/o the current rpctest tag. We do this so that
those of us that use extensions which will compile live files like
vim-go can once again fix compile errors as we go in our editors.

In order to do this, we now make an external `testsCases` variable, and
have two variants: one that's empty (no build tag), and one that's fully
populated with all our tests (build tag active). As a result, the main
file will now always build regardless of if the build tag is active or
not, but we'll only actually execute tests if the `testCases` variable
has been populated.

As sample run w/ the tag off:
```
=== RUN   TestLightningNetworkDaemon
--- PASS: TestLightningNetworkDaemon (0.00s)
PASS
ok  	github.com/lightningnetwork/lnd/lntest/itest	0.051s
```
2020-09-21 21:16:31 +02:00
Olaoluwa Osuntokun
acd615aca4
lntest: default to btcd as default test harness backend
Otherwise, if we remove the build tags, then there's no default backend,
and compilation will fail.
2020-09-21 21:16:31 +02:00
Olaoluwa Osuntokun
95634186c2
lntest/itest: move harness to new file
This in prep for a bigger move in the next commit.
2020-09-21 21:16:30 +02:00
Joost Jager
6ae05c6bac
Merge pull request #4592 from joostjager/sweep-deadline
cnct+sweep: cpfp-aware anchor sweeping
2020-09-18 20:15:03 +02:00
Joost Jager
29602e88e8
cnct: cpfp-sweep anchors
For unconfirmed commit tx anchors, supply the sweeper with cpfp info and
a confirmation target fee estimate.

The sweeper will try to pay for the parent commit tx as long as the
current fee estimate exceeds the pre-signed commit tx fee rate.
2020-09-17 12:30:41 +02:00
yyforyongyu
d2d71476bd
lntest: init SignerClient and test DeriveSharedKey 2020-09-17 14:52:36 +08:00
Wilmer Paulino
a5c5304c09
Merge pull request #4452 from yyforyongyu/add-connection-timeout
lnrpc+tor: add network connection timeout
2020-09-16 12:28:29 -07:00
Joost Jager
2ebfb64b9b
Merge pull request #4606 from joostjager/cpfp-sweep-prep
cnct+sweep+itest: preparations for cpfp-aware sweeper
2020-09-16 11:48:39 +02:00
Joost Jager
b6ebf3f27d
lntest: use web fee estimator in itests 2020-09-16 08:17:34 +02:00
yyforyongyu
e8d9643f3a
itest: test for connection timeout 2020-09-16 11:50:59 +08:00
Olaoluwa Osuntokun
fa342a1230
Merge pull request #4567 from calvinrzachman/max-wumbo
add new max channel size config option
2020-09-15 15:31:47 -07:00
Oliver Gugger
107e135df9
itest: test PSBT funding flow with raw final TX
As we already create two channels in our PSBT funding flow itest we can
easily just submit the final transaction for the second channel in the
raw wire format to test this new functionality.
2020-09-15 08:30:23 +02:00
Calvin Zachman
f5fb64e552 add new max channel size config option
- let users specify their MAXIMUM WUMBO with new config option which sets the maximum channel size lnd will accept
- current implementation is a simple check by the fundingManager rather than anything to do with the ChannelAcceptor
- Add test cases which verify that maximum channel limit is respected for wumbo/non-wumbo channels
- use --maxchansize 0 value to distinguish set/unset config. If user sets max value to 0 it will not do anything as 0 is currently used to indicate to the funding manager that the limit should not be enforced. This seems justifiable since --maxchansize=0 doesn't seem to make sense at first glance.
- add integration test case to ensure that config parsing and valiation is proper. I simplified the funding managers check electing to rely on config.go to correctly parse and set up either i) non wumbo default limit of 0.16 BTC OR ii) wumbo default soft limit of 10 BTC

Addresses: https://github.com/lightningnetwork/lnd/issues/4557
2020-09-14 21:16:32 -05:00
yyforyongyu
567fa9ed10
itest: disable node retrying to connect to miner
Previously the chainbackend connected to the miner using a permanent
connection, which would retry the connection when it’s disconnected.
It would leave multiple connections between the chainbackend and the
miner, causing difficulties in debugging. Currently, there are two
occasions when disconnection happens. One happens when running the open
channel reorg test, the miner is connected/disconnected multiple times
on purpose. The other happens when the chainbackend receives a
MSG_WITNESS_TX type from the miner, which would be considered as an
invalid type and disconnects the miner. With the latter one being fixed
in btcd, the chainbackend will still be disconnected from the miner if
it reaches the ban score by requesting too many notfound messages in a
short time which is why the `--nobanning` flag is added.
2020-09-12 20:09:54 +08:00
yyforyongyu
724f6e0969
itest: require no error when cleaning up chainbackends 2020-09-12 20:09:53 +08:00
yyforyongyu
933d84273a
itest: require no error when tearing down miners 2020-09-12 20:09:52 +08:00
Conner Fromknecht
3c606bd9c0
Merge pull request #4542 from Crypt-iQ/mock_signer_cleanup_0817
multi: remove mock.go
2020-09-11 13:16:17 -04:00
Oliver Gugger
295bd44fea
itest: test custom permissions 2020-09-04 09:50:41 +02:00
Oliver Gugger
decd2d975c
itest: parallelize test steps 2020-09-04 09:50:40 +02:00
Oliver Gugger
9862ee7cd6
itest: extract connection setup 2020-09-04 09:50:40 +02:00
Oliver Gugger
17276e9a7f
itest: use only one timeout context
All these operations should be very fast and can be done in just one
timeout context.
2020-09-04 09:50:39 +02:00
Oliver Gugger
a929f56781
itest: use require for macaroon tests 2020-09-04 09:50:39 +02:00
Oliver Gugger
906011f278
itest: add _test file name suffix to tests
To fix the compiler of some IDEs complaining about types and functions
it cannot find, we rename all files that contain tests back to lnd_xxx_test.go to make
sure they are compiled correctly.
2020-09-04 09:50:39 +02:00