This change makes the test more sensitive to bugs than a route
with nodes that all enforce the same fee policy.
In addition to that, the fee has been increased to a level
at which potential problems with improper fee calculation
become detectable (and not disappear in rounding).
With the recent bug fixes in the peer connection, it's no longer the
case that just disconnecting a certain peer causes it to no longer
connect to the other. As a result, we now shutdown Alice to ensure no
reconnection occurs. We'll then later restart alice when we restart
dave.
This commit adds the helper method shutdownAndAssert, and makes us shut
down nodes created temporarily during the test cases using defers
instead of at the end of the test. This makes it easier to remember to
shut them down, as it is done right after they are started.
This commit add 3 integration tests for send-to-route RPC call.
One test ensures that single-hop payments are processed. Another test
checks that payments through a multi-hop route are processed. Lastly,
there is a test to check error propagation while sending payments via fake
predefined route.
Before the previous commit, we assumed the HTLC's timeout transaction
would be the only transaction in the mempool. In reality, after mining
some blocks for the HTLC to expire and waiting for the timeout
transaction to arrive in the mempool, at times we would instead detect
the funding output's sweeping transaction and proceed the test with this
assumption, leading to the case where we would have to mine extra blocks
to include the HTLC sweeping transaction. This has been resolved in the
previous commit, so this fix is no longer needed.
This reverts commit e54f1ea4dbe59b2e53a94774995ae1711746c2f8.
In this commit, we address an existing flake that would be triggered
when testing HTLC timeouts. After force closing a channel and generating
enough blocks to expire an HTLC, we would wait for a transaction to
arrive in the mempool and assumed it was the timeout transaction.
Instead, we'd detect the funding output sweep transaction and attempt to
proceed with the test with the incorrect assumption of the timeout
transaction being broadcast.
This commit adds an integration test that checks that in case a channel
counterparty tries to settle an HTLC with the wrong preimage, the
channel is failed and force closed.
In this commit, we rewrite the node announcement integration test to no
longer depend on a sleep interval. Instead, we use graph topology
updates in order to be notified exactly when we receive the node
announcement.
In this commit, we address a timeout issue on our Travis builds within
the update channel policy integration test. Before asserting the policy
for the channel between Alice and Bob, we need to make sure we receive
the updates responsible for modifying this policy first. At times, the
update wasn't received in time before checking the policy, which led to
us checking the old policy. We fix this by explicitly making sure we
wait for the updates first.
This commit attempts to fix a flake we have seen, where a routing hints
for an inactive channel was unexpectedly added to the invoice. This
happened because of a race between shutting down the endpoint of this
channel (Eve) and creating the invoice. This commit fixes this by moving
the call to AddInvoice with corresponding route hint check inside a
WaitPredicate.
In this commit, we fix an existing flake within the set of revocation
integration tests. Right after Bob's restart, we attempt to force close
the channel. However, it may be the case that the chain arbitrator
hasn't yet been created. As a result, the request to force close the
channel will fail. We easily fix this by wrapping the force close
attempt in a WaitPredicate.
This commit is similar to a recent commit which attempts to account for
internal block races by mining a second block if the initial assertion
for HTLC state fails. This can happen again if by the time that the
sweeping transaction is broadcast, it doesn't make it into the next
block mine.
In this commit, we modify the
testMultHopRemoteForceCloseOnChainHtlcTimeout test slightly to attempt
to account for a block race between the arrival of a message betwen the
contract resolver and the utxo nursery. If this message arrives "late"
(relative to the speed with which we mine blocks in test), then it'll be
detected as such by the utxo nursery. However, since we attempt to mine
a precise number of blocks, if this happens, then we'll never actually
mine that extra block to trigger a broadcast of the sweep transaction.
This commit removes the mining of a block within the integration tests,
that would lead to a flake during runs. The reason was that the sweeoing
transaction was actually broadcast already at this point, and this block
was mined with the assumption that it wasn't. However, test would pass
more often than not because the UTXO nursery would be working on
graduating new heights, whcih is a bit slow, so in most cases the block
would be mined before it had broadcast the sweeping transaction.
By removing this block we ensure that the sweeping transaction will be
found for sure in the waitForTxInMempool call below.
This commit modifies the integration tests to work with the recent
changes to the ChannelArbitrator, where it will only act on commitments
that has been confirmed. Main changes involving when to look for
transactions in the mempool and in blocks, and using the new RPC for
getting channels in the "waiting close" phase when they are waiting for
the commitment to confirm.
In this commit, we fix an existing flake in the integration tests. If it
was the case that Alice didn't yet know of all the channels, then the
payment attempt below would fail at times, depending on other timing
factors in the test. We fix this flake by waiting for Alice to learn of
all channels before we proceed to the actual testing logic.
In this commit, we extend the closeChannelAndAssert testing utility
function to ensure that the channel is no longer marked as "pending
close" in the database. With this change, we hop to catch a recently
reported issue wherein users report that a co-op close channel has been
fully confirmed, yet it still pops up in the `pendingchannels` command.
In this commit, we update the testUpdateChannelPolicy to exercise the
recent set of changes within the switch. If one applies this test to a
fresh branch (without those new changes) it should fail. This is due to
the fact that before, Bob would attempt to apply the constraints of the
incoming link (which we updated) instead of the outgoing link. With the
recent set of changes, the test now properly passes.
In this commit, we extend the testRevokedCloseRetributionRemoteHodl so
that the final broadcast revoked transaction has incoming *and* outgoing
HTLC's. As is, this test fails as there's a lingering bug in the way we
generate htlc resolutions. A follow up commit will remedy this issue.
In this commit, we modify the sphinx replay test to actually force close
that final channel. We do this, as otherwise, we'll now reject the co-op
close attempt as the channel still has active HTLCs.
This commit is a follow up to the prior commit which fixed a rounding
error bug in lnwallet. For uniformity, we also fix other occurrences in
the breach arbiter, as well as the integration tests.
This commit removes two unnecessary disconnect/reconnect attempts
from the switch itest. Both occur after the primary test has been
completed, and were being executed before doing a final sanity check
that the path is still usable. Nothing about the test behavior should
change.
This commits removes many of the timeout-based consistency
checks, which tend to be less dependable on Travis. They
have been replaced with stronger WaitPredicates, and the new
WaitInvariant where appropriate. This should give provide a
much greater degree of reliability in our highly-variant
testing environments.
// 1. Carol --> Dave --> Alice --> Bob forward payment
// 2. Carol --- Dave X Alice --- Bob disconnect intermediaries
// 3. Carol --- Dave X Alice <-- Bob settle last hop
// 4. Carol --- Dave X X shutdown Bob, restart Alice
// 5. Carol <-- Dave <-- Alice X expect settle to propagate
Since a ChannelPoint's funding txid can now be get/set as raw bytes or
a string, we first need to check what type it's currently set to before
accessing it.
In this commit, we fix an existing flake on Travis related to the new
set of on-chain HTLC tests. In this timing flake, Bob would broadcast
his sweeping transaction, but *mid block mining*. As a result, the
output would never be properly swept, needing an additional block to be
mined. We’ll now wait for both Bob’s sweeping transaction, and Carol’s
sweep transaction to be confirmed before we attempt our assertions.
In this commit, we fix an existing bug in the implementation of the
resolution of the htlcOutgoingContestResolver. Before this commit, we
would _always_ watch the claim outpoint. However, if this is on the
remote party’s commitment transaction, then we would end up watching
the wrong output. We’ll now properly detect this by modifying which
output we watch, based on if we have a second level transaction or not.
In this commit, we add 6 new integration tests to test the various
actions that may need to be performed when either side goes on-chain to
fully resolve HTLC’s. Many of the tests are mirrors of each other as
they test sweeping/resolving HTLC’s from both commitment transactions.
In this commit, we modify the testChannelForceClosure integration test
to account for the fact that now the HTLCs are only sent to the nursery
once they’ve fully timed out. Additionally, we now send Carol HTLCs
that she doesn’t know the preimage to, so she doesn’t attempt to sweep
them before we can actually time them out.
This commit adds a new integration test, that checks that
policy/fee updates get propagated properly in the network,
such that the other nodes learn about the changes.
This commit extracts the launching of a goroutine subscribing
to and forwarding graph topology notifications into its own
utility method, such that it can be used in other tests as
well.
This commit adds a line of text including a test case's
name to Alice's and Bob's logfiles during integration
tests, making it easier to seek for the place in the log
where the specific tests start.
This commit adds a new test, that in a small network
of 4 nodes, tests that a private channel can be used
for routing payments by the endpoints of the channel,
while the existence of the channel is not known to
the rest of the network.
In this commit we rename the lnrpc.PendingChannelRequest and
lnrpc.PendingChannelResponse to
lnrpc.PendingChannelsRequest/lnrpc.PendingChannelsResponse. We do this
as we strive to ensure that the naming scheme across the RPC interface
is consistent.
This commit adds a small cancellation check to the
invoice subscription integration test. Calls to
Fatalf were observed on travis after the test had
ended stemming from the subscriber's goroutine,
which could happen if the subscription is late.
It also extends the timeout on the test from 5 to
10 seconds.
There is a sleep after channels are opened to ensure the channel is
removed from the set of pending reservations. We can avoid this sleep
and get better guarantees of the channel being opened by deleting the
reservation just before updating channel state to active instead of
after.
This helps catch issues in the RPC tests faster by putting a timeout
on all SendPayment requests. Otherwise, if a payment stalled, the test
would run until the test suite timeout, 10 minutes. Also simplifies
some code with a helper function and using SendPaymentSync where
possible.
If an lnd node encounters a fatal process errors, we now log the error
asynchronously, instead of calling Fatalf on the test instance. This
is because calling Fatalf in a separate goroutine does not actually
end the test immediately as expected, but rather just kills the
goroutine. This ensures that we see server errors on all nodes before
the test process exits.
In this commit, we increase the initial amount sent in the multi-hop
error integration test. We must do this now as the Bandwidth() function
now takes into account the total fee paid within the commitment
transaction. This caused an earlier send to fail instead of the once
following send.
To fix this, we simply send less in the initial test case.
This commit removes the testChannelReestablishment integration test as
it is currently incomplete. In order to properly test this, we require
some infrastructure that allows us to kill the connection at will once
a message is sent across the wire.
In this commit BOLT№2 retranmission logic for the channel link have
been added. Now if channel link have been initialised with the
'SyncState' field than it will send the lnwire.ChannelReestablish
message and will be waiting for receiving the same message from remote
side. Exchange of this message allow both sides understand which
updates they should exchange with each other in order sync their
states.
This commit adds a test for an lnd instance experiecing a reorg,
making sure a channel that is in the channel graph (using the
describegraph rpc call) will no longer be after the funding tx
gets reorged out.
In this commit, we modify the assertNumOpenChannelsPending to poll
every 200ms rather than just a single attempt. The goal of this commit
is to reduce the number of flakes on travis caused by slow instances.
In this commit, we extend the testMultiHopPayments test case to add an
additional node to the multi-hop payment route: Dave.
Related to #391 as it uses the provided code fragment to reproduce the
bug. With this test in place, we’ll now be able to notice any
regressions in this area.
In this commit we modify all the test that involve sending payments to
use the payreq returned rather than manually populating the
payhash+dest fields in the SendRequest proto argument to SendPayment.
This is required as if we don’t use the payreq returned, then the
receiving node will reject the payment as it’ll use the global final
CLTV delta value, rather than the value (within the pay req) that the
receiver is expecting.
In order to maintain the original essence of the test, we need to clear
the state of missionControl with each attempt, essentially advancing
time between each payment attempt.
This commit fixes a bug related to swallowing an error that should go
to the switch in the case of an insufficient balance error when
attempting to add a new HTLC to the channel state machine. In this
case, an error would never be returned back to the client/switch, and
the internal processing within the channelLink would loop forever,
attempting to add an HTLC that can’t be added due to insufficient
balance to state machine itself.
We fix this issue by only treating the lnwallet.ErrMaxHTLCNumber as the
only error that prompts adding an HTLC to the overflow queue rather
than sending the error directly back to the switch.
This commit alters the existing retribution
post breach conf test case with the intention
of testing the correct response in event that
the remote party broadcasts a prior state
while HTLCs have been extended. This serves
as a preliminary integration for an expansion
of the breach arbiter integration tests.
The primary change involves using the new
htlchodl mode for debugging, which causes the
remote peer to ignore any intent to settle
incoming HTLCs. The result is that any
payments sent to the remote party are held in
limbo, allowing us to test for these
conditions more accurately.
Currently the test case only tests that the
justice transaction is mined. After we have
fully integrated the breach arbiter to sweep
2nd layer HTLCs, this test will be altered
to check for spends from the appropriate
inputs.
This commit adds a breached contract retribution storage layer using
boltdb to the breach arbiter. The breach arbiter now stores retribution
state on disk between detecting a contract breach, broadcasting a
justice transaction that sweeps the channel, and finally witnessing the
justice transaction confirm on the blockchain. It is critical that such
state is persisted on disk, so that if our node restarts at any point
during the retribution procedure, we can recover and continue from the
persisted state.
This commit modifies the HTLC integration tests to be mindful of the
max payment size. Rather than sending the payment in one large batch,
we instead now send it in chunks of the max payment size.
This commit prevents the case where btcd stops before lnd is fully
started, thus making lnd_test hang on trying to stop lnd using
`StopDaemon`. The underlying issue is that while lnd is trying to
start the server and subscribe to block notifications from btcd,
btcd stops, and lnd continues to attempt to reconnect before it
ever starts the interrupt handler. This reversal avoids that issue
by making sure lnd is stopped before btcd.
In the "multiple channel creation" test, in some cases the
responder (Bob) was not yet considering the first channel
to be opened (activeReservation still not deleted in
fundingManager) when Alice tried to open the second channel.
This would cause the test to fail. This commit adds a small
sleep before the creation of the second channel, to give
Bob some time to finish the opening process.
This commit updates the integration tests to reflect the reality after
removing code that would always attempt to increment the current update
timestamp by one for each channel announcement. Without connecting
directly to carol, it isn’t guaranteed that Alice will receive that
announcement as Bob would have already processed one for Carol when
their channel was created.
Make the fundingmanager send an updated node announcement
each time it opens a new channel. This is to make sure
our node announcement is propagated in the network, since
peers will ignore our node announcements if we haven't
opened any channels yet.
Within the network, it's important that when an HTLC forwarding failure
occurs, the recipient is notified in a timely manner in order to ensure
that errors are graceful and not unknown. For that reason with
accordance to BOLT №4 onion failure obfuscation have been added.
This commit fixes a minor bug in the goroutine that’s launched to check
the sync status of a particular node. Previously, the goroutine could
end up infinitely stuck on a send as once the chain has been detected
as synced, it didn't exit.
We fix this now by ensure that the goroutine always terminates after
the initial notification to the caller. Additionally, we not ensure
that both the internal and exterior goroutine are both reading off of
the peer’s quit channel.
This commit modifies the travis build script, and our local test script
to ensure that the race condition builds are conducted in a parallel
build. After this commit two travis builds will be kicked off for each
push/commit: one that runs the race condition tests in isolation, and
another that runs the integration tests then the coverage tests.
In order to do the above cleanly, the integration tests are now guarded
behind a build flag. In order to run the integration tests, one now
needs to specify the `-tags rpctest` flag when running the `go test`
command.
In this commit additional test have been added which tests the ability
of Alice and Bob asynchroniously exchange the payment between each
other. This scenario will be higly frequent in the payment between
payment providers.
