A height hint not being set would cause lnd to scan for the
confirmation/spend of a txid/outpoint/address from genesis.
The number of confirmations not being set within a confirmation request
would cause the internal TxNotifier to deadlock when dispatching
updates.
Since GetUtxo is a potentially long running call, we would see
RegisterSpendNtfn block in some cases, in particular on starting the
chain watcher, causing lnd to hang on startup.
This commit makes the call to GetUtxo in a go routine, letting
RegisterSpendNtfn return immediately when the notification events are
created.
This commit fixes a bug that would cause the
notifier not to commit spend hints for items
that are not found. This is done by calling
UpdateSpendDetails with a nil detail, permitting
the notifier to begin updating the spend hints
with new blocks that arrive at tip. The change
is designed to mimic the behavior for historical
confirmation dispatch.
The symptom of this bug is needing to do many
long rescans on startup, even if new blocks
arrive after the rescan had completed. With
this change, nodes will have to do the scans
once more before their hints will be properly
updated. Restarts from then on should not
have this behavior.
In this commit, we update the build to point to the latest version of
neutrino and btcwallet. The latest version of neutrino includes a number
of bug fixes, and new features like reliably transaction broadcast. The
latest version of btcwallet contains a number of bug fixes related to
properly remove invalid transactions from its database.
In this commit, we modify the main interface-level tests for neutrino to
skip the block catch up re-org test. We do this, as the test itself is
very unstable and only passes occasionally by the grace of RNGesus.
What's needed to ensure it passes reliably is to extend neutrino with a
cache of the last N blocks disconnected. Sample output of the affected
test when run:
```
--- SKIP: TestInterfaces/neutrino_test_catch_up_on_missed_blocks_w/_reorged_best_block (0.00s)
interface_test.go:1574: skipping re-org test for neutrino
```
In this commit, we modify GetTestTxidAndScript to generate new P2PKH
scripts. This is needed to properly test confirmations and spends of
unique scripts on-chain within the set of interface-level test cases.
In this commit, we extend the NeutrinoNotifier to support registering
scripts for spends notifications. Once the script has been detected as
spent within the chain, a spend notification will be dispatched through
the Spend channel of the SpendEvent returned upon registration.
For scripts that have been spent in the past, the rescan logic has been
modified to match on the script rather than the outpoint. A concurrent
queue for relevant transactions has been added to proxy notifications
from the underlying rescan to the txNotifier. This is needed for
scripts, as we cannot perform a historical rescan for scripts through
`GetUtxo`, like we do with outpoints.
For scripts that are unspent, a filter update is sent to the underlying
rescan to ensure that we match and dispatch on the script when
processing new blocks.
Along the way, we also address an issue where we'd miss detecting that
an outpoint/script has been spent in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always request the backend to notify us of the spend
once it detects it at tip, regardless of whether a historical rescan was
detected or not.
In this commit, we extend the BitcoindNotifier to support registering
scripts for spends notifications. Once the script has been detected as
spent within the chain, a spend notification will be dispatched through
the Spend channel of the SpendEvent returned upon registration.
For scripts that have been spent in the past, the rescan logic has been
modified to match on the script rather than the outpoint. This is done
by re-deriving the script of the output a transaction input is spending
and checking whether it matches ours.
For scripts that are unspent, a request to the backend will be sent to
alert the BitcoindNotifier of when the script was spent by a
transaction. To make this request we encode the script as an address, as
this is what the backend uses to detect the spend. The transaction will
then be proxied through the txUpdates concurrent queue, which will hand
it off to the underlying txNotifier and dispatch spend notifications to
the relevant clients.
Along the way, we also address an issue where we'd miss detecting that
an outpoint/script has been spent in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always request the backend to notify us of the spend
once it detects it at tip, regardless of whether a historical rescan was
detected or not.
In this commit, we extend the BtcdNotifier to support registering
scripts for spends notifications. Once the script has been detected as
spent within the chain, a spend notification will be dispatched through
the Spend channel of the SpendEvent returned upon registration.
For scripts that have been spent in the past, the rescan logic has been
modified to match on the script rather than the outpoint. This is done
by encoding the script as an address.
For scripts that are unspent, a request to the backend will be sent to
alert the BtcdNotifier of when the script was spent by a transaction. To
make this request we encode the script as an address, as this is what
the backend uses to detect the spend. The transaction will then be
proxied through the txUpdates concurrent queue, which will hand it off
to the underlying txNotifier and dispatch spend notifications to the
relevant clients.
Along the way, we also address an issue where we'd miss detecting that
an outpoint/script has been spent in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always request the backend to notify us of the spend
once it detects it at tip, regardless of whether a historical rescan was
detected or not.
In this commit, we extend the NeutrinoNotifier to support registering
scripts for confirmation notifications. Once the script has been
detected as confirmed within the chain, a confirmation notification will
be dispatched to through the Confirmed channel of the ConfirmationEvent
returned upon registration.
For scripts that have confirmed in the past, the `historicalConfDetails`
method has been modified to determine whether the script has been
confirmed by locating the script in an output of a confirmed
transaction.
For scripts that have yet to confirm, a filter update is sent to the
underlying rescan to ensure that we match and dispatch on the script
when processing new blocks.
Along the way, we also address an issue where we'd miss detecting that a
transaction/script has confirmed in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always update our filters to detect the confirmation
at tip, regardless of whether a historical rescan was detected or not.
In this commit, we extend the BitcoindNotifier to support registering
scripts for confirmation notifications. Once the script has been
detected as confirmed within the chain, a confirmation notification will
be dispatched to through the Confirmed channel of the ConfirmationEvent
returned upon registration.
For scripts that have confirmed in the past, the `historicalConfDetails`
method has been modified to skip the txindex and go straight to scanning
the chain manually if confirmation request is for a script. When
scanning the chain, we'll determine whether the script has been
confirmed by locating the script in an output of a confirmed
transaction.
For scripts that have yet to confirm, they will be properly tracked
within the TxNotifier.
In this commit, we extend the BtcdNotifier to support registering
scripts for confirmation notifications. Once the script has been
detected as confirmed within the chain, a confirmation notification will
be dispatched to through the Confirmed channel of the ConfirmationEvent
returned upon registration.
For scripts that have confirmed in the past, the `historicalConfDetails`
method has been modified to skip the txindex and go straight to scanning
the chain manually if confirmation request is for a script. When
scanning the chain, we'll determine whether the script has been
confirmed by locating the script in an output of a confirmed
transaction.
For scripts that have yet to confirm, they will be properly tracked
within the TxNotifier.
In this commit, we add the current chain parameters to the
BitcoindNotifier. This will be used in a future commit in order to
convert outputs scripts into addresses. This is needed since the
bitcoind backend uses these addresses to detect whether the script
encoded within it was spent by a transaction in the chain.
In this commit, we add the current chain parameters to the BtcdNotifier.
This will be used in a future commit in order to convert outputs scripts
into addresses. This is needed since the btcd backend uses these
addresses to detect whether the script encoded within it was spent by a
transaction in the chain.
In this commit, we refactor the TxNotifier's ProcessRelevantSpendTx to
also detect script spends. This can easily be done as the transaction
filtering logic was refactored into its own method in a previous commit.
In this commit, we modify the TxNotifier's ConnectTip method to also
detect whether a registered script has been confirmed or spent on-chain
by a transaction in the connected block. Once detected, notifications
for them will be queued up for dispatch as with txids/outpoints.
We've also refactored the ConnectTip method into smaller and reusable
methods, which will serve useful later.
In this commit, we refactor the HeightHintCache and its underlying
interfaces to be able to manipulate hints for ConfRequests and
SpendRequests. By doing so, we'll be able to manipulate hints for
scripts if the request includes either a zero hash or a zero outpoint.
These structs allow us to request notifications for either
txids/outpoints or output scripts. We'll be using these in later commits
to index confirmation and spend requests within the TxNotifier.
In this commit, we modify all existing historical rescans for
ChainNotifier backends to scan backwards rather than forwards. If we
know that a transaction has been confirmed, or outpoint spent, the it's
likely that the event has recently transpired assuming we've been
offline for a short period of time. Therefore, if we scan backwards
rather than forwards, then we can save potentially hundreds or thousands
of block fetches if the event recently happened close to the tip of the
chain.
We bound this search at the genesis block, to ensure we don't underflow
the uint32 used throughout the package in the main loop.
In this commit, we alter the different ChainNotifier implementations to
dispatch confirmation and spend notifications after blocks. We do this
to ensure the external consistency of our registered clients.
In this commit, we modify the TxNotifier's ConnectTip method to no
longer dispatch notifications to any clients who had a request fulfilled
within the height connected. Instead, it will queue the notifications
for dispatch and we add a new method NotifyHeight, which will actually
dispatch them. We do this to allow the users of the TxNotifier to be
more flexible when dispatching notifications.
In this commit, we modify the notifier to handle filter updates
synchronously. We do this to prevent race conditions between new block
notifications and filter updates. Otherwise, it's possible for a new
block to come in that should match our filter, but doesn't due to the
filter being updated after.
We also modify their order so that the filter is updated first. We do
this so we can immediately start watching for the event at tip while the
rescan is ongoing.
In this commit, we'll now commit the current height of the TxNotifier as
the height hint for an outpoint/transaction after a rescan has been
completed and has determined that the outpoint is unspent/transaction is
unconfirmed. We do this to prevent another potentially long rescan if
the daemon is restarted before a new block comes in (which is when the
hints will be updated again).
In this commit, we modify the logic within RegisterSpendNtfn for the
NeutrinoNotifier to account for the recent changes made to the
TxNotifier. Since it is now able to handle spend notification
registration and dispatch, we can bypass all the current logic within
the NeutrinoNotifier and interact directly with the TxNotifier instead.
The most notable change is that now we'll only attempt a historical
rescan if the TxNotifier tells us so.
In this commit, we modify the logic within RegisterSpendNtfn for the
BtcdNotifier to account for the recent changes made to the TxNotifier.
Since it is now able to handle spend notification registration and
dispatch, we can bypass all the current logic within the
BtcdNotifier and interact directly with the TxNotifier instead.
The most notable change is that now we'll only attempt a historical
rescan if the TxNotifier tells us so.
In this commit, we modify the logic within RegisterSpendNtfn for the
BitcoindNotifier to account for the recent changes made to the
TxNotifier. Since it is now able to handle spend notification
registration and dispatch, we can bypass all the current logic within
the BitcoindNotifier and interact directly with the TxNotifier instead.
The most notable changes include the following:
1. We'll only attempt a historical rescan if the TxNotifier tells us
so.
2. We'll dispatch the historical rescan within the main goroutine to
prevent WaitGroup panics, due to the asynchronous nature of the
notifier.
In this commit, we address an issue w.r.t. updating the confirm hints
for transactions and spend hints for outpoints on chain updates.
Previously, upon a block being disconnected, we'd attempt to commit a
new height hint for all outstanding confirmation notifications. This is
not correct because we'll end up modifying the height hint for things
that have confirmed at a previous height than the one being
disconnected. This would cause issues on restart when attempting a
historical dispatch, as we would start scanning at a height above which
the transaction actually confirmed in.
This has been addressed by only updating the hints for outstanding
notifications that are still unconfirmed/unspent and for notifications
that were confirmed/spent within the block being connected/disconnected.
In this commit, we introduce support to the TxNotifier to detect
spending transactions of registered outpoints being reorged out of the
chain. In the event that a reorg does occur, we'll consume the Spend
notification if it hasn't been consumed yet, and dispatch a Reorg
notification instead.
In this commit, we add support to allow the TxNotifier to properly
determine whether a new block extending the chain contains a transaction
that spends a registered outpoint. In the event that it does, spend
notifications will be dispatched to all active registered clients for
such outpoint.
In this commit, we introduce the registration logic for spend
notifications to the TxNotifier. Most of this logic was taken from the
different existing ChainNotifier implementations, however, it features
some useful additions in order to make the ChainNotifier a bit more robust.
Some of these additions include the following:
1. RegisterSpend will now return a HistoricalSpendDispatch struct,
which includes the details for successfully determining if an outpoint
was spent in the past. A HistoricalSpendDispatch will only be returned
upon the first registration of an outpoint. This is done as,
previously, if multiple clients registered for the same outpoint, then
multiple historical rescans would also be dispatched, incurring a toll
on the backend itself.
2. UpdateSpendDetails will now be used to determine when a historical
rescan has completed, no matter if a spending transaction was found or
not. This is needed in order to responsibly update the spend hints for
outpoints at tip, otherwise we'd attempt to update them even though we
haven't yet determined if they have been spent or not. This will
dispatch notifications to all currently registered clients for the
same outpoint. In the event that another client registers later on,
then the spending details are cached in memory in order to prevent
further historical rescans.
In this commit, we introduce the required fields for the TxNotifier to
properly carry its duties in notifying its registered clients about the
spend of an outpoint. These are not yet used, but will be throughout the
some of the following commits.
In this commit, we add a new channel within the SpendEvent struct that
will be sent upon whenever the spending transaction of the registered
outpoint gets reorged out of the chain. This will pave the road for
successfully handling a funding transaction getting reorged out of the
chain among other things.
In this commit, we modify our height hint cache to no longer start a
database transaction if no outpoints/txids are provided to update the
height hints for.
In this commit, we modify the set of tests that start the different
backend notifiers with UnsafeStart to stop them within the tests
themselves. This prevents us from running into a panic when attempting
to run the package-level tests with a filter (using test.run).
Removes details field from conf notifications, in favor
of using the details on the confSet. We also bound the
requested conf depth to the reorg saftey limit, as the
behavior of state tracking within the notifier is
undefined otherwise.
In this commit, we address a small bug where it's possible to deliver a
confirmation notification with stale confirmation details upon
registration. This can happen if a transaction has confirmed but was
reorged out of the chain later on, and a subsequent notification is
registered.
In this commit, we'll attempt to consume a reorg notification for a
transaction that was previously reorged out of the chain upon block
inclusion to ensure that it is not lingering due to a client not
handling it the first time.
In this commit, we mark the rescan status for a transaction as complete
if we happen to detect it has confirmed within a new block that extends
the chain. We do this as otherwise, it's possible for us to not
immediately dispatch the notification upon a subsequent registration due
to the rescan state machine.
This commit ensures that a confSet's details
are assigned in the confNotifications index
after discovering the transaction at tip. The
recent changes allow a later notification to
be dispatched on registration if an earlier one
has already discovered the confirmation details.
Before this change, it was observed that a later
registration would attempt an immediate delivery,
but fail to do so because the confset's details
were nil. This commit remedies that dispatch path,
allowing the integration tests to pass again.
