Earlier the channel arbitrator would fail to recognize channels pending
close that were in the breached state. This lead to the state machine
not progressing correctly, and in some cases crashing since we would
attempt to force close an already closed channel.
A test TestChannelArbitratorForceCloseBreachedChannel is added to
exercise one of these scenarios.
Earlier we would not react to breaches, as these are handled by other
subsystems. Now we advances our state machine in case of breach, such
that we'll gracefully exit, and won't have leftover state in case of a
restart.
A simple test TestChannelArbitratorBreachClose to exercise this behavior
is added.
This commit fixes the 'unable to find incoming resolution' error that
occured when trying to resolve incoming htlcs below the dust limit that
are not actually present on the commitment tx.
Previously the invoice registry wasn't aware of replayed htlcs. This was
dealt with by keeping the invoice accept/settle logic idempotent, so
that a replay wouldn't have an effect.
This mechanism has two limitations:
1. No accurate tracking of the total amount paid to an invoice. The total
amount couldn't just be increased with every htlc received, because it
could be a replay which would lead to counting the htlc amount multiple
times. Therefore the total amount was set to the amount of the first
htlc that was received, even though there may have been multiple htlcs
paying to the invoice.
2. Impossible to check htlc expiry consistently for hodl invoices. When
an htlc is new, its expiry needs to be checked against the invoice cltv
delta. But for a replay, that check must be skipped. The htlc was
accepted in time, the invoice was moved to the accepted state and a
replay some blocks later shouldn't lead to that htlc being cancelled.
Because the invoice registry couldn't recognize replays, it stopped
checking htlc expiry heights when the invoice reached the accepted
state. This prevents hold htlcs from being cancelled after a restart.
But unfortunately this also caused additional htlcs to be accepted on an
already accepted invoice without their expiry being checked.
In this commit, the invoice registry starts to persistently track htlcs
so that replays can be recognized. For replays, an htlc resolution
action is returned early. This fixes both limitations mentioned above.
Currently the invoice registry cannot tell apart the htlcs that pay to
an invoice. Because htlcs may also be replayed on startup, it isn't
possible to determine the total amount paid to an invoice.
This commit is a first step towards fixing that. It reports the circuit
keys of htlcs to the invoice registry, which forms the basis for
accurate invoice accounting.
In this commit, we make a series of changes to ensure that we'll be able
to properly survive restarts if we crash right after we call
MarkChannelClosed. In order to ensure we can survive restarts, we'll now
long the confirmed CommitSet to disk right before we close the channel.
Upon restart, we'll read these from disk so we can pick up where we left
over.
Additionally, we also will now consult the legacy chain actions if it
turns out that the channel has been closed, but we don't have a
confCommitSet written to disk. This will only be the case for nodes that
had pending close channels before this commitment.
Since we no longer have up to date chain actions on disk, we'll use the
HTLC sets in memory which contain the necessary information we need to
in order to obtain the HTLC amounts.
In this commit, we change the behavior of the channel arb to no longer
write chain actions to disk. Instead, using the new CommitSet struct,
we'll replay our set of prior actions based on what actually got into
the chain. As a result, we no longer need to write the chain actions at
all, instead they're reconstructed at run time to determine decisions,
and before any commitments are broadcast in order to determine if we
need to go to chain at all.
In this commit, we add a new `checkLocalChainActions` method. This
method differs from the existing `checkChainActions` method in that it's
only concerned with actions we should take on chain for our local state
based on the local _and_ remote state. This change ensures that we'll
now to go to chain order to cancel an HTLC that was on the remote
party's commitment transaction, but not our own.
In this commit, we fix a lingering TOOD statement in the channel arb.
Before this commitment, we would simply wipe our our local HTLC set of
the HTLC set that was on the remote commitment transaction on force
close. This was incorrect as if our commitment transaction had an HTLC
that the remote commitment didn't, then we would fail to cancel that
back, and cause both channels to time out on chain.
In order to remedy this, we introduce a new `HtlcSetKey` struct to track
all 3 possible in-flight set of HTLCs: ours, theirs, and their pending.
We also we start to tack on additional data to all the unilateral close
messages we send to subscribers. This new data is the CommitSet, or the
set of valid commitments at channel closure time. This new information
will be used by the channel arb in an upcoming commit to ensure it will
cancel back HTLCs in the case of split commitment state.
Finally, we start to thread through an optional *CommitSet to the
advanceState method. This additional information will give the channel
arb addition information it needs to ensure it properly cancels back
HTLCs that are about to time out or may time out depending on which
commitment is played.
Within the htlcswitch pakage, we modify the `SignNextCommitment` method
to return the new set of pending HTLCs for the remote party's commitment
transaction and `ReceiveRevocation` to return the latest set of
commitment transactions on the remote party's commitment as well. This
is a preparatory change which is part of a larger change to address a
lingering TODO in the cnct.
Additionally, rather than just send of the set of HTLCs after the we
revoke, we'll also send of the set of HTLCs after the remote party
revokes, and we create a pending commitment state for it.
This commit isolates preimages of forwarded htlcs from invoice
preimages. The reason to do this is to prevent the incoming contest
resolver from settling exit hop htlcs for which the invoice isn't marked
as settled.
One of the first things the incoming contest resolver does is checking
if the preimage is available and if it is, convert itself into a success
resolver.
This behaviour makes it unnecessary to already determine earlier in the
process whether an incoming contest or a success resolver is needed.
By having all incoming htlcs go through the incoming contest resolver,
the number of execution paths is reduced and it becomes easier to
ascertain that the implemented logic is correct.
The only functional change in this commit is that a forwarded htlc for
which is the preimage is known, is no longer settled when the htlc is
already expired. Previously a success resolver would be instantiated
directly, skipping the expiry height check.
This created a risk that the success resolver would never finish,
because an expired htlc could already have been swept by the remote
party and there is no detection of this remote spend in the success
resolver currently.
With the new change, the general direction that an expired htlc
shouldn't be settled and instead given up on is implemented more
consistently.
This commit prepares for fixing edges cases related to hodl
invoice on-chain resolution.
This commit adds logging of the reason to go to chain for a channel.
This can help users to find out the reason why a channels forced closed.
To get all go to chain reasons, an optimization to break early is
removed. This optimization was not significant, because the normal flow
already examined all htlcs. In the exceptional case where we need to go
to chain, it does not weigh up against logging all go to chain reasons.
This commits exposes the various parameters around going to chain and
accepting htlcs in a clear way.
In addition to this, it reverts those parameters to what they were
before the merge of commit d1076271456bdab1625ea6b52b93ca3e1bd9aed9.
The multiplier doesn't make sense because funds may be equally at risk
by failing to broadcast to chain regardless of whether the HTLC is a
redeem or a timeout.
In this commit, we modify the WitnessCache's
AddPreimage method to accept a variadic number
of preimages. This enables callers to batch
preimage writes in performance critical areas
of the codebase, e.g. the htlcswitch.
Additionally, we lift the computation of the
witnesses' keys outside of the db transaction.
This saves us from having to do hashing inside
and blocking other callers, and limits extraneous
blocking at the call site.
Previously the arbitrator wasn't advanced to the final stage after
the last contract resolved.
Also channel arbitrator now does not ignore a log error anymore
unresolved contracts cannot be retrieved.
Previously, contract resolvers that needed to publish a second level tx,
did not have access to the original htlc amount.
This commit reconstructs this amount from data that is already persisted
in arbitrator log.
Co-authored-by: Joost Jager <joost.jager@gmail.com>
In this commit, we prevent the ChainArbitrator from sending a force
close request for a channel if it has previously already sent one. We do
this to prevent blocking the caller of ForceCloseContract.
At ChannelArbitrator startup we now check the database close status of
the channel. If we detect that the channel is closed, but our state
machine hasn't advanced to reflect that (possibly because of a shutdown
before the state transition was finished), we manually trigger the state
transition to recover.
This commit moves the responsibility for closing local and remote force
closes in the database from the chain watcher to the channel arbitrator.
We do this because we previously would close the channel in the
database, before sending the event to the channel arbitrator. This could
lead to a situation where the channel was marked closed, but the channel
arbitrator didn't receive the event before shutdown. As we don't listen
for chain events for channels that are closed, those channels would be
stuck in the pending close state forever, as the channel arbitrator
state machine wouldn't progress.
We fix this by letting the ChannelArbitrator close the channel in the
database. After the contract resolutions are logged (in the state
callback before transitioning to StateContractClosed) we mark the
channel closed in the database. This way we make sure that it is marked
closed only if the resolutions have been successfully persisted.
This commit removes the state callback, and instead logs the contract
resolutions directly after receiving the unilateral close event. The
resolutions won't change so there's not really necessary to wait to log
them, and this greatly simplifies the code.
In this commit, we alter cooperative channel closures to also use
MarkChannelResolved in order to unify the logic for the different types
of channel closures.
