Commit Graph

263 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
28eb8474f4
chainntnfs: modify all historical rescans to scan backwards
In this commit, we modify all existing historical rescans for
ChainNotifier backends to scan backwards rather than forwards. If we
know that a transaction has been confirmed, or outpoint spent, the it's
likely that the event has recently transpired assuming we've been
offline for a short period of time. Therefore, if we scan backwards
rather than forwards, then we can save potentially hundreds or thousands
of block fetches if the event recently happened close to the tip of the
chain.

We bound this search at the genesis block, to ensure we don't underflow
the uint32 used throughout the package in the main loop.
2018-11-24 15:50:12 -06:00
Wilmer Paulino
e402a4e146
chainntnfs: dispatch conf/spend notifications after blocks
In this commit, we alter the different ChainNotifier implementations to
dispatch confirmation and spend notifications after blocks. We do this
to ensure the external consistency of our registered clients.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
770e005943
chainntnfs/txnotifer: prevent dispatching notifications within ConnectTip
In this commit, we modify the TxNotifier's ConnectTip method to no
longer dispatch notifications to any clients who had a request fulfilled
within the height connected. Instead, it will queue the notifications
for dispatch and we add a new method NotifyHeight, which will actually
dispatch them. We do this to allow the users of the TxNotifier to be
more flexible when dispatching notifications.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
9a025867d0
Revert "chainntnfs/btcdnotify: disable height hint cache in testing"
This reverts commit 98e7c968d4bf5900a5d7be7f557eab8f623633c0.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
5127f2aa82
Revert "chainntnfs/bitcoindnotify: disable height hints in testing"
This reverts commit ab28db5b0dc780a09b7474e187cefff809ac2714.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
35bfb79099
Revert "chainntnfs/interface_test: run tests w/ disabled cache"
This reverts commit 12761a4f435a033e8425dbcb3b407bb094fcae9f.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
88ac985c95
Revert "chainntnfs/height_hint_cache_test: add tests for disabled cache"
This reverts commit 45a2c9aca8f10f341848845e1f51261bd09fcb68.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
716c20b18d
Revert "chainntnfs/height_hint_cache: add disable flag to hint cache"
This reverts commit 7df9ae026665de75175853e486143ab80122ce9d.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
e6b1a27cd7
chainntnfs/neutrinonotify: make filter update synchronous
In this commit, we modify the notifier to handle filter updates
synchronously. We do this to prevent race conditions between new block
notifications and filter updates. Otherwise, it's possible for a new
block to come in that should match our filter, but doesn't due to the
filter being updated after.

We also modify their order so that the filter is updated first. We do
this so we can immediately start watching for the event at tip while the
rescan is ongoing.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
60a1d73e08
chainntnfs/txnotifier: commit height hint after rescan is complete
In this commit, we'll now commit the current height of the TxNotifier as
the height hint for an outpoint/transaction after a rescan has been
completed and has determined that the outpoint is unspent/transaction is
unconfirmed. We do this to prevent another potentially long rescan if
the daemon is restarted before a new block comes in (which is when the
hints will be updated again).
2018-10-30 17:59:31 -07:00
Wilmer Paulino
6458868ba0
chainntnfs/interface_test: add spend reorg test 2018-10-30 17:59:31 -07:00
Wilmer Paulino
deca4cfe44
chainntnfs/neutrinonotify: remove old spend notification handling logic
In this commit, we remove the old spend notification logic within the
NeutrinoNotifier as it's been phased out by the TxNotifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
bfd11a251e
chainntnfs/neutrinonotify: handle spend notification registration w/ TxNotifier
In this commit, we modify the logic within RegisterSpendNtfn for the
NeutrinoNotifier to account for the recent changes made to the
TxNotifier. Since it is now able to handle spend notification
registration and dispatch, we can bypass all the current logic within
the NeutrinoNotifier and interact directly with the TxNotifier instead.

The most notable change is that now we'll only attempt a historical
rescan if the TxNotifier tells us so.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
74139c9a3f
chainntnfs/btcdnotify: remove old spend notification handling logic
In this commit, we remove the old spend notification logic within the
BtcdNotifier as it's been phased out by the TxNotifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
88edd320d5
chainntnfs/btcdnotify: handle spend notification registration w/ TxNotifier
In this commit, we modify the logic within RegisterSpendNtfn for the
BtcdNotifier to account for the recent changes made to the TxNotifier.
Since it is now able to handle spend notification registration and
dispatch, we can bypass all the current logic within the
BtcdNotifier and interact directly with the TxNotifier instead.

The most notable change is that now we'll only attempt a historical
rescan if the TxNotifier tells us so.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
0927f35dc1
chainntnfs/bitcoindnotify: remove old spend notification handling logic
In this commit, we remove the old spend notification logic within the
BitcoindNotifier as it's been phased out by the TxNotifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
180dffd154
chainntnfs/bitcoindnotify: handle spend notification registration w/ TxNotifier
In this commit, we modify the logic within RegisterSpendNtfn for the
BitcoindNotifier to account for the recent changes made to the
TxNotifier. Since it is now able to handle spend notification
registration and dispatch, we can bypass all the current logic within
the BitcoindNotifier and interact directly with the TxNotifier instead.

The most notable changes include the following:

  1. We'll only attempt a historical rescan if the TxNotifier tells us
  so.

  2. We'll dispatch the historical rescan within the main goroutine to
  prevent WaitGroup panics, due to the asynchronous nature of the
  notifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
1fe3d59836
chainntnfs/txnotifier_test: extend tests to handle spend notifications 2018-10-30 17:59:31 -07:00
Wilmer Paulino
2935392f51
chainntnfs/txnotifier: correctly update confirm/spend hints on chain updates
In this commit, we address an issue w.r.t. updating the confirm hints
for transactions and spend hints for outpoints on chain updates.
Previously, upon a block being disconnected, we'd attempt to commit a
new height hint for all outstanding confirmation notifications. This is
not correct because we'll end up modifying the height hint for things
that have confirmed at a previous height than the one being
disconnected. This would cause issues on restart when attempting a
historical dispatch, as we would start scanning at a height above which
the transaction actually confirmed in.

This has been addressed by only updating the hints for outstanding
notifications that are still unconfirmed/unspent and for notifications
that were confirmed/spent within the block being connected/disconnected.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
f27e73fcb8
chainntnfs/txnotifier: detect reorgs for spending transactions of registered outpoints
In this commit, we introduce support to the TxNotifier to detect
spending transactions of registered outpoints being reorged out of the
chain. In the event that a reorg does occur, we'll consume the Spend
notification if it hasn't been consumed yet, and dispatch a Reorg
notification instead.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
e6b2755a57
chainntnfs/txnotifier: watch for spends at tip
In this commit, we add support to allow the TxNotifier to properly
determine whether a new block extending the chain contains a transaction
that spends a registered outpoint. In the event that it does, spend
notifications will be dispatched to all active registered clients for
such outpoint.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
4d7fa9ecc4
chainntnfs/txnotifier: allow registration of spend notifications
In this commit, we introduce the registration logic for spend
notifications to the TxNotifier. Most of this logic was taken from the
different existing ChainNotifier implementations, however, it features
some useful additions in order to make the ChainNotifier a bit more robust.

Some of these additions include the following:

  1. RegisterSpend will now return a HistoricalSpendDispatch struct,
  which includes the details for successfully determining if an outpoint
  was spent in the past. A HistoricalSpendDispatch will only be returned
  upon the first registration of an outpoint. This is done as,
  previously, if multiple clients registered for the same outpoint, then
  multiple historical rescans would also be dispatched, incurring a toll
  on the backend itself.

  2. UpdateSpendDetails will now be used to determine when a historical
  rescan has completed, no matter if a spending transaction was found or
  not. This is needed in order to responsibly update the spend hints for
  outpoints at tip, otherwise we'd attempt to update them even though we
  haven't yet determined if they have been spent or not. This will
  dispatch notifications to all currently registered clients for the
  same outpoint. In the event that another client registers later on,
  then the spending details are cached in memory in order to prevent
  further historical rescans.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
fc7a33b64f
chainntnfs/txnotifier: add fields/structs to track spend notifications
In this commit, we introduce the required fields for the TxNotifier to
properly carry its duties in notifying its registered clients about the
spend of an outpoint. These are not yet used, but will be throughout the
some of the following commits.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
87123d5e23
chainntnfs: extend SpendEvent with reorg channel
In this commit, we add a new channel within the SpendEvent struct that
will be sent upon whenever the spending transaction of the registered
outpoint gets reorged out of the chain. This will pave the road for
successfully handling a funding transaction getting reorged out of the
chain among other things.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
f4128c9afb
chainntnfs/txnotifier: move rescanState & confNtfnSet decl to top 2018-10-30 17:59:31 -07:00
Wilmer Paulino
405e8f0fad
chainntnfs/txnotifier: rename hintCache -> confirmHintCache 2018-10-30 17:59:31 -07:00
Wilmer Paulino
f65401b439
chainntnfs/txnotifier: rename Register -> RegisterConf 2018-10-30 17:59:31 -07:00
Wilmer Paulino
82f6fd7a91
chainntnfs: rename TxConfNotifier -> TxNotifier 2018-10-30 17:59:31 -07:00
Wilmer Paulino
f8789e9db0
chainntnfs: rename txconfnotifier.go -> txnotifier.go 2018-10-30 17:59:31 -07:00
Wilmer Paulino
f4cf1073d4
chainntnfs/height_hint_cache: prevent db transactions with no updates
In this commit, we modify our height hint cache to no longer start a
database transaction if no outpoints/txids are provided to update the
height hints for.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
39d86d5731
chainntnfs/interface_test: stop UnsafeStart notifiers within test
In this commit, we modify the set of tests that start the different
backend notifiers with UnsafeStart to stop them within the tests
themselves. This prevents us from running into a panic when attempting
to run the package-level tests with a filter (using test.run).
2018-10-30 17:59:31 -07:00
Conner Fromknecht
ba28ec3be0
chainntnfs/neutrinonotify/neutrino: fix debug logs 2018-10-26 18:32:32 -07:00
Conner Fromknecht
e03c818aa7
chainntnfs/txconfnotifier: remove ntfn details, bound conf depth
Removes details field from conf notifications, in favor
of using the details on the confSet. We also bound the
requested conf depth to the reorg saftey limit, as the
behavior of state tracking within the notifier is
undefined otherwise.
2018-10-26 18:32:31 -07:00
Wilmer Paulino
589dc96d88
chainntnfs/tx_notifier: extract conf reorg dispatch into method 2018-10-26 18:32:31 -07:00
Wilmer Paulino
5ae8243d0d
chainntnfs/tx_notifier: remove cached conf details on reorg
In this commit, we address a small bug where it's possible to deliver a
confirmation notification with stale confirmation details upon
registration. This can happen if a transaction has confirmed but was
reorged out of the chain later on, and a subsequent notification is
registered.
2018-10-26 18:32:31 -07:00
Wilmer Paulino
b28145b69e
chainntnfs/tx_notifier: consume reorg notification for transactions on block inclusion
In this commit, we'll attempt to consume a reorg notification for a
transaction that was previously reorged out of the chain upon block
inclusion to ensure that it is not lingering due to a client not
handling it the first time.
2018-10-26 18:32:31 -07:00
Wilmer Paulino
a4dee14b20
chainntnfs/tx_notifier: mark rescan as complete for transactions confirmed at tip
In this commit, we mark the rescan status for a transaction as complete
if we happen to detect it has confirmed within a new block that extends
the chain. We do this as otherwise, it's possible for us to not
immediately dispatch the notification upon a subsequent registration due
to the rescan state machine.
2018-10-26 18:32:31 -07:00
Conner Fromknecht
eee5311557
chainntnfs/txconnotifier: add debug logs, log errs/warnings, godocs 2018-10-26 18:32:31 -07:00
Conner Fromknecht
2dcb86bced
chainntnfs/txconfnotifier: set confset details at tip
This commit ensures that a confSet's details
are assigned in the confNotifications index
after discovering the transaction at tip. The
recent changes allow a later notification to
be dispatched on registration if an earlier one
has already discovered the confirmation details.

Before this change, it was observed that a later
registration would attempt an immediate delivery,
but fail to do so because the confset's details
were nil. This commit remedies that dispatch path,
allowing the integration tests to pass again.
2018-10-26 18:32:30 -07:00
Conner Fromknecht
df9bb56068
chainntnfs/neutrino: use HistoricalConfDispatch in ntfn registry 2018-10-26 18:32:30 -07:00
Conner Fromknecht
6cd0f867ad
chainntnfs/btcd: use HistoricalConfDispatch in ntfn registry 2018-10-26 18:32:30 -07:00
Conner Fromknecht
a4c9f62c6b
chainntnfs/bitcoind: use HistoricalConfDispatch in ntfn registry 2018-10-26 18:32:30 -07:00
Conner Fromknecht
11c231d814
chainntnfs/txconfnotifier_test: remove clientID argument...
to UpdateConfDetails
2018-10-26 18:32:30 -07:00
Conner Fromknecht
32e7368e1e
chainntnfs/txconfnotifier: remove clientID from UpdateConfDetails signature 2018-10-26 18:32:30 -07:00
Conner Fromknecht
9ae6d43916
chainntnfs/txconfnotifier: split out ntfn dispatch into helper 2018-10-26 18:32:30 -07:00
Conner Fromknecht
217b1fc0ef
chainntnfs/txconfnotifier: return HistoricalConfDispatch from Register 2018-10-26 18:32:29 -07:00
Conner Fromknecht
f94de2308f
chainntnfs/txconfnotifier: return HistoricalConfDispatch from Register 2018-10-26 18:32:29 -07:00
Conner Fromknecht
74122e00f5
chainntnfs/txconfnotifier: add PkScript to ConfNtfn 2018-10-26 18:32:29 -07:00
Conner Fromknecht
37c864d6f6
chainntnfs/txconfnotifier: add HistoricalConfDispatch struct 2018-10-26 18:32:29 -07:00
Conner Fromknecht
e804b30669
chainntnfs/txconfnotifier_test: update to use multi-value Register 2018-10-26 18:32:29 -07:00