In this commit, we remove the per channel `sigPool` within the
`lnwallet.LightningChannel` struct. With this change, we ensure that as
the number of channels grows, the number of gouroutines idling in the
sigPool stays constant. It's the case that currently on the daemon, most
channels are likely inactive, with only a hand full actually
consistently carrying out channel updates. As a result, this change
should reduce the amount of idle CPU usage, as we have less active
goroutines in select loops.
In order to make this change, the `SigPool` itself has been publicly
exported such that outside callers can make a `SigPool` and pass it into
newly created channels. Since the sig pool now lives outside the
channel, we were also able to do away with the Stop() method on the
channel all together.
Finally, the server is the sub-system that is currently responsible for
managing the `SigPool` within lnd.
In this commit, we prevent the ChainArbitrator from sending a force
close request for a channel if it has previously already sent one. We do
this to prevent blocking the caller of ForceCloseContract.
We pool the database for the channel commit point with an exponential
backoff. This is meant to handle the case where we are in process of
handling a channel sync, and the case where we detect a channel close
and must wait for the peer to come online to start channel sync before
we can proceed.
In this commit, we modify the newly introduced UtxoSweeper.CreateSweepTx
to accept the confirmation target as a param of the method rather than a
struct level variable. We do this as this allows each caller to decide
at sweep time, what the fee rate should be, rather than using a global
value that is meant to work in all scenarios. For example, anytime
we're sweeping an output with a CLTV lock that's has a dependant
transaction we need to sweep/cancel, we may require a higher fee rate
than a regular force close with a CSV output.
This commit restructures the initialization procedure
for chain watchers such that they can proceed in parallel.
This is primarily to help nodes running with the neutrino
backend, which otherwise forces a serial rescan for each
active channel to check for spentness.
Doing so allows the rescans to take advantage of batch
scheduling in registering for the spend notifications,
ensuring that only one or two passes are made, as opposed
to one for each channel.
Lastly, this commit ensures that the chain arb is properly
shutdown if any of it's chain watchers or channel arbs
fails to start, so as to cancel their goroutines before
exiting.
At ChannelArbitrator startup we now check the database close status of
the channel. If we detect that the channel is closed, but our state
machine hasn't advanced to reflect that (possibly because of a shutdown
before the state transition was finished), we manually trigger the state
transition to recover.
This commit moves the responsibility for closing local and remote force
closes in the database from the chain watcher to the channel arbitrator.
We do this because we previously would close the channel in the
database, before sending the event to the channel arbitrator. This could
lead to a situation where the channel was marked closed, but the channel
arbitrator didn't receive the event before shutdown. As we don't listen
for chain events for channels that are closed, those channels would be
stuck in the pending close state forever, as the channel arbitrator
state machine wouldn't progress.
We fix this by letting the ChannelArbitrator close the channel in the
database. After the contract resolutions are logged (in the state
callback before transitioning to StateContractClosed) we mark the
channel closed in the database. This way we make sure that it is marked
closed only if the resolutions have been successfully persisted.
This commit removes the state callback, and instead logs the contract
resolutions directly after receiving the unilateral close event. The
resolutions won't change so there's not really necessary to wait to log
them, and this greatly simplifies the code.
This commit attempts to fix an inconsistency in when we consider an HTLC
to expire. When we first launched the resolver we would compare the
current block height against the expiry, while for new incoming blocks
we would compare against expiry-1.
This lead to a flake during integration tests, during a call to
RestartNode after _exactly_ enough blocks for the HTLC to expire. In
some cases the resolver would see the new blocks and consider the HTLC
to be expired (because of the -1), while in some cases resolver would
shut down before seeing the new blocks, and upon restart wouldn't act on
the new height because we did not compare against -1.
This commit fixes this by doing the same comparison in both cases.
Due to a recent change within the codebase to return estimated fee rates
in sat/kw, this commit ensures that we use this fee rate properly by
calculing a transaction's fees using its weight. This includes all of
the different transactions that are created within lnd (funding, sweeps,
etc.). On-chain transactions still rely on a sat/vbyte fee rate since it's
required by btcwallet.
This commit fixes a potential race condition during
shutdown, that could allow the chain arb's
activeWatchers or activeChannels map to be modified
while ranging over their contents. We fix this by
copying the contents into new maps with the mutex
held, before releasing the mutex and shutting down
each watcher or channel arbitrator.
This commit makes the chainwatcher attempt to dispatch a remote close
when it detects a remote state with a state number higher than our
known remote state. This can mean that we lost some state, and we check
the database for (hopefully) a data loss commit point retrieved during
channel sync with the remote peer. If this commit point is found in the
database we use it to try to recover our funds from the commitment.
This commit fixes a potential race condition within the
IncomingContestResolver, that could cause us to miss a
preimage that was delivered in time.
Currently we query the db for the preimage, and then
subscribe for notifications. This permits the following
ordering of events:
- query for preimage, returns nothing
- preimage is added and delivered to subscribers
- subscribe to preimages
- preimage never comes through!!
We fix this by reordering to subscribe for preimages and
then query just in case it already exists. The effect is
that the query will always return a valid read of the
preimages that are currently queued for delivery.
In this commit, we alter cooperative channel closures to also use
MarkChannelResolved in order to unify the logic for the different types
of channel closures.
We no longer have to mark the channel as fully closed in the database,
as it is done directly in the chainWatcher. Instead, we stop the watcher
and delete it from the set of active watchers.
This commit makes the dispatchCooperativeClose method mark the channel
fully closed directly, without registering for confirmation
notifications first. We can do this as recent changes to the
contractcourt changed the definition of a closed channel in the database
to have had its closing tx confirmed, and we only dispatch the
cooperative close once the transaction has 1 confirmation.
We also rename the markChanClosed method to notifyChanClosed, to more
clearly indicate that the ChainArbitrator no longer has to mark the
channel fully closed in the database.
In this commit, we attempt to fix a bug that's possible within the
Start() method of the ChainArbiter. We pass the channel pointer directly
into the newActiveChannelArbitrator function causing it to close over
the loop variable. We later use the channel point directly to send
messages to other sub-systems. It's possible that we actually have the
shadowed loop variable and will send an incorrect message. Defensively,
we now re-bind the loop variable in order to ensure we point to the
proper channel.
This commit adds a test for the case where the ChannelArbitrator fails
to broadcast its commitment during a force close because of
ErrDoubleSpend. We test that in this case it will still wait for a
commitment getting confirmed in-chain, then resolve.
This commit adds the new function closure option ContractBreach to the
ChainArbitrator config, a closure that is again used by the ChainWatcher
to reliably handoff a breach event to the breachArbiter.
This commit changes how the ChainWatcher notifies the breachArbiter
about a channel breach. Instead of assuming the breachArbiter is among
the clients subscibing to channel events, it will call a new method
contractBreach(), and assume the breachArbiter has reliably gotten the
breach info when this method returns with a non-nil error.
Since the breachArbiter was the only sybsystem having a sync chain
subsciption, we also remove the (now) unused syncDispatch option.
This move the log message "channel marked pending-closed" to the point
where the channel actually has been marked pending closed, instead of
before the database transaction has been done.
This commit removes the for loop in the closeObserver, as it wasn't
serving any purpose. After receiving a spend notification we would
return, breaking out of the loop. When getting a quit signal we would
also return, making the loop only do one iteration in any case.
This commit makes clients subscribing to channel events that are marked
"sync dispatch" _not_ being deleted from the list of clients when they
call Cancel(). Instead a go routine will be launched that will send an
error on every read of the ProcessACK channel.
This fixes a race in handing off the breach info while lnd was shutting
down. The breach arbiter could end up being shut down (and calling
Cancel()) before while the ChainWatcher was in the process of
dispatching a breach. Since the breach arbiter no longer was among the
registered clients at this point, the ChainWatcher would assume the
breach was handed off successfully, and mark the channel as pending
closed. When lnd now was restarted, the breach arbiter would not know
about the breach, and the ChainWatcher wouldn't attempt to re-dispatch,
as it was already marked as pending closed.
This commit adds MVP unit tests for the following scenarios in the
ChannelArbitrator:
1) A cooperative close is confirmed.
2) A remote force close is confirmed.
3) A local force close is requested and confirmed.
4) A local force close is requested, but a remote force close gets
confirmed.
This commit changes the ChainWatcher to only send a chain event in case
the various spends are _confirmed_ on-chain, not only seen on the
network.
A consequence of this is that we now give the ChainWatcher the
responsibility of marking the channel closed when the closing tx is
confirmed, instead of the ChannelArbitrator.
This commit changes the channel arbitrator state machine to only care
about commitment transactions that are being confirmed on-chain
according to the chain_watcher. This is meant to handles the cases where
we would broadcast our commitment, expecting it to get confirmed, but
instead a competing transaction was confirmed.
This commit readies the ChannelArbitrator state machine for the change
that will make the ChainWatcher only notify on confirmed commitments.
The state machine has gotten a new state, StateCommitmentBroadcasted,
which we'll transition to after we have broadcasted our own commitment.
From this state we'll go to the StateContractClosed state regardless of
which commitment the ChainWatcher notifies about, unifying the contract
resolution betweee the local and remote force close.
