Commit Graph

320 Commits

Author SHA1 Message Date
Wilmer Paulino
0f4ff2d09c
chainntnfs/txnotifier: support registration of scripts for confirmations 2019-01-21 13:57:43 -08:00
Wilmer Paulino
a0d3b7d9e3
chainntnfs: support caching confirm/spend hints for scripts
In this commit, we refactor the HeightHintCache and its underlying
interfaces to be able to manipulate hints for ConfRequests and
SpendRequests. By doing so, we'll be able to manipulate hints for
scripts if the request includes either a zero hash or a zero outpoint.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
de0adc4d52
chainntnfs/txnotifier: add ConfRequest and SpendRequest structs
These structs allow us to request notifications for either
txids/outpoints or output scripts. We'll be using these in later commits
to index confirmation and spend requests within the TxNotifier.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
5be8f08b16
chainntnfs/txnotiifer: define MaxNumConfs supported as ReorgSafetyLimit 2019-01-11 16:58:15 -08:00
Wilmer Paulino
fd2bfaa007
chainntnfs: add TestTxNotifierMaxConfs 2019-01-11 16:58:15 -08:00
Wilmer Paulino
7239e04696
chainntnfs: initialize TxNotifier with new ReorgSafetyLimit 2019-01-11 16:58:15 -08:00
Wilmer Paulino
9d51114546
chainntnfs/txnotifier: define ReorgSafetyLimit as 144 blocks 2019-01-11 16:58:15 -08:00
Olaoluwa Osuntokun
28eb8474f4
chainntnfs: modify all historical rescans to scan backwards
In this commit, we modify all existing historical rescans for
ChainNotifier backends to scan backwards rather than forwards. If we
know that a transaction has been confirmed, or outpoint spent, the it's
likely that the event has recently transpired assuming we've been
offline for a short period of time. Therefore, if we scan backwards
rather than forwards, then we can save potentially hundreds or thousands
of block fetches if the event recently happened close to the tip of the
chain.

We bound this search at the genesis block, to ensure we don't underflow
the uint32 used throughout the package in the main loop.
2018-11-24 15:50:12 -06:00
Wilmer Paulino
e402a4e146
chainntnfs: dispatch conf/spend notifications after blocks
In this commit, we alter the different ChainNotifier implementations to
dispatch confirmation and spend notifications after blocks. We do this
to ensure the external consistency of our registered clients.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
770e005943
chainntnfs/txnotifer: prevent dispatching notifications within ConnectTip
In this commit, we modify the TxNotifier's ConnectTip method to no
longer dispatch notifications to any clients who had a request fulfilled
within the height connected. Instead, it will queue the notifications
for dispatch and we add a new method NotifyHeight, which will actually
dispatch them. We do this to allow the users of the TxNotifier to be
more flexible when dispatching notifications.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
9a025867d0
Revert "chainntnfs/btcdnotify: disable height hint cache in testing"
This reverts commit 98e7c968d4bf5900a5d7be7f557eab8f623633c0.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
5127f2aa82
Revert "chainntnfs/bitcoindnotify: disable height hints in testing"
This reverts commit ab28db5b0dc780a09b7474e187cefff809ac2714.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
35bfb79099
Revert "chainntnfs/interface_test: run tests w/ disabled cache"
This reverts commit 12761a4f435a033e8425dbcb3b407bb094fcae9f.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
88ac985c95
Revert "chainntnfs/height_hint_cache_test: add tests for disabled cache"
This reverts commit 45a2c9aca8f10f341848845e1f51261bd09fcb68.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
716c20b18d
Revert "chainntnfs/height_hint_cache: add disable flag to hint cache"
This reverts commit 7df9ae026665de75175853e486143ab80122ce9d.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
e6b1a27cd7
chainntnfs/neutrinonotify: make filter update synchronous
In this commit, we modify the notifier to handle filter updates
synchronously. We do this to prevent race conditions between new block
notifications and filter updates. Otherwise, it's possible for a new
block to come in that should match our filter, but doesn't due to the
filter being updated after.

We also modify their order so that the filter is updated first. We do
this so we can immediately start watching for the event at tip while the
rescan is ongoing.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
60a1d73e08
chainntnfs/txnotifier: commit height hint after rescan is complete
In this commit, we'll now commit the current height of the TxNotifier as
the height hint for an outpoint/transaction after a rescan has been
completed and has determined that the outpoint is unspent/transaction is
unconfirmed. We do this to prevent another potentially long rescan if
the daemon is restarted before a new block comes in (which is when the
hints will be updated again).
2018-10-30 17:59:31 -07:00
Wilmer Paulino
6458868ba0
chainntnfs/interface_test: add spend reorg test 2018-10-30 17:59:31 -07:00
Wilmer Paulino
deca4cfe44
chainntnfs/neutrinonotify: remove old spend notification handling logic
In this commit, we remove the old spend notification logic within the
NeutrinoNotifier as it's been phased out by the TxNotifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
bfd11a251e
chainntnfs/neutrinonotify: handle spend notification registration w/ TxNotifier
In this commit, we modify the logic within RegisterSpendNtfn for the
NeutrinoNotifier to account for the recent changes made to the
TxNotifier. Since it is now able to handle spend notification
registration and dispatch, we can bypass all the current logic within
the NeutrinoNotifier and interact directly with the TxNotifier instead.

The most notable change is that now we'll only attempt a historical
rescan if the TxNotifier tells us so.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
74139c9a3f
chainntnfs/btcdnotify: remove old spend notification handling logic
In this commit, we remove the old spend notification logic within the
BtcdNotifier as it's been phased out by the TxNotifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
88edd320d5
chainntnfs/btcdnotify: handle spend notification registration w/ TxNotifier
In this commit, we modify the logic within RegisterSpendNtfn for the
BtcdNotifier to account for the recent changes made to the TxNotifier.
Since it is now able to handle spend notification registration and
dispatch, we can bypass all the current logic within the
BtcdNotifier and interact directly with the TxNotifier instead.

The most notable change is that now we'll only attempt a historical
rescan if the TxNotifier tells us so.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
0927f35dc1
chainntnfs/bitcoindnotify: remove old spend notification handling logic
In this commit, we remove the old spend notification logic within the
BitcoindNotifier as it's been phased out by the TxNotifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
180dffd154
chainntnfs/bitcoindnotify: handle spend notification registration w/ TxNotifier
In this commit, we modify the logic within RegisterSpendNtfn for the
BitcoindNotifier to account for the recent changes made to the
TxNotifier. Since it is now able to handle spend notification
registration and dispatch, we can bypass all the current logic within
the BitcoindNotifier and interact directly with the TxNotifier instead.

The most notable changes include the following:

  1. We'll only attempt a historical rescan if the TxNotifier tells us
  so.

  2. We'll dispatch the historical rescan within the main goroutine to
  prevent WaitGroup panics, due to the asynchronous nature of the
  notifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
1fe3d59836
chainntnfs/txnotifier_test: extend tests to handle spend notifications 2018-10-30 17:59:31 -07:00
Wilmer Paulino
2935392f51
chainntnfs/txnotifier: correctly update confirm/spend hints on chain updates
In this commit, we address an issue w.r.t. updating the confirm hints
for transactions and spend hints for outpoints on chain updates.
Previously, upon a block being disconnected, we'd attempt to commit a
new height hint for all outstanding confirmation notifications. This is
not correct because we'll end up modifying the height hint for things
that have confirmed at a previous height than the one being
disconnected. This would cause issues on restart when attempting a
historical dispatch, as we would start scanning at a height above which
the transaction actually confirmed in.

This has been addressed by only updating the hints for outstanding
notifications that are still unconfirmed/unspent and for notifications
that were confirmed/spent within the block being connected/disconnected.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
f27e73fcb8
chainntnfs/txnotifier: detect reorgs for spending transactions of registered outpoints
In this commit, we introduce support to the TxNotifier to detect
spending transactions of registered outpoints being reorged out of the
chain. In the event that a reorg does occur, we'll consume the Spend
notification if it hasn't been consumed yet, and dispatch a Reorg
notification instead.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
e6b2755a57
chainntnfs/txnotifier: watch for spends at tip
In this commit, we add support to allow the TxNotifier to properly
determine whether a new block extending the chain contains a transaction
that spends a registered outpoint. In the event that it does, spend
notifications will be dispatched to all active registered clients for
such outpoint.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
4d7fa9ecc4
chainntnfs/txnotifier: allow registration of spend notifications
In this commit, we introduce the registration logic for spend
notifications to the TxNotifier. Most of this logic was taken from the
different existing ChainNotifier implementations, however, it features
some useful additions in order to make the ChainNotifier a bit more robust.

Some of these additions include the following:

  1. RegisterSpend will now return a HistoricalSpendDispatch struct,
  which includes the details for successfully determining if an outpoint
  was spent in the past. A HistoricalSpendDispatch will only be returned
  upon the first registration of an outpoint. This is done as,
  previously, if multiple clients registered for the same outpoint, then
  multiple historical rescans would also be dispatched, incurring a toll
  on the backend itself.

  2. UpdateSpendDetails will now be used to determine when a historical
  rescan has completed, no matter if a spending transaction was found or
  not. This is needed in order to responsibly update the spend hints for
  outpoints at tip, otherwise we'd attempt to update them even though we
  haven't yet determined if they have been spent or not. This will
  dispatch notifications to all currently registered clients for the
  same outpoint. In the event that another client registers later on,
  then the spending details are cached in memory in order to prevent
  further historical rescans.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
fc7a33b64f
chainntnfs/txnotifier: add fields/structs to track spend notifications
In this commit, we introduce the required fields for the TxNotifier to
properly carry its duties in notifying its registered clients about the
spend of an outpoint. These are not yet used, but will be throughout the
some of the following commits.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
87123d5e23
chainntnfs: extend SpendEvent with reorg channel
In this commit, we add a new channel within the SpendEvent struct that
will be sent upon whenever the spending transaction of the registered
outpoint gets reorged out of the chain. This will pave the road for
successfully handling a funding transaction getting reorged out of the
chain among other things.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
f4128c9afb
chainntnfs/txnotifier: move rescanState & confNtfnSet decl to top 2018-10-30 17:59:31 -07:00
Wilmer Paulino
405e8f0fad
chainntnfs/txnotifier: rename hintCache -> confirmHintCache 2018-10-30 17:59:31 -07:00
Wilmer Paulino
f65401b439
chainntnfs/txnotifier: rename Register -> RegisterConf 2018-10-30 17:59:31 -07:00
Wilmer Paulino
82f6fd7a91
chainntnfs: rename TxConfNotifier -> TxNotifier 2018-10-30 17:59:31 -07:00
Wilmer Paulino
f8789e9db0
chainntnfs: rename txconfnotifier.go -> txnotifier.go 2018-10-30 17:59:31 -07:00
Wilmer Paulino
f4cf1073d4
chainntnfs/height_hint_cache: prevent db transactions with no updates
In this commit, we modify our height hint cache to no longer start a
database transaction if no outpoints/txids are provided to update the
height hints for.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
39d86d5731
chainntnfs/interface_test: stop UnsafeStart notifiers within test
In this commit, we modify the set of tests that start the different
backend notifiers with UnsafeStart to stop them within the tests
themselves. This prevents us from running into a panic when attempting
to run the package-level tests with a filter (using test.run).
2018-10-30 17:59:31 -07:00
Conner Fromknecht
ba28ec3be0
chainntnfs/neutrinonotify/neutrino: fix debug logs 2018-10-26 18:32:32 -07:00
Conner Fromknecht
e03c818aa7
chainntnfs/txconfnotifier: remove ntfn details, bound conf depth
Removes details field from conf notifications, in favor
of using the details on the confSet. We also bound the
requested conf depth to the reorg saftey limit, as the
behavior of state tracking within the notifier is
undefined otherwise.
2018-10-26 18:32:31 -07:00
Wilmer Paulino
589dc96d88
chainntnfs/tx_notifier: extract conf reorg dispatch into method 2018-10-26 18:32:31 -07:00
Wilmer Paulino
5ae8243d0d
chainntnfs/tx_notifier: remove cached conf details on reorg
In this commit, we address a small bug where it's possible to deliver a
confirmation notification with stale confirmation details upon
registration. This can happen if a transaction has confirmed but was
reorged out of the chain later on, and a subsequent notification is
registered.
2018-10-26 18:32:31 -07:00
Wilmer Paulino
b28145b69e
chainntnfs/tx_notifier: consume reorg notification for transactions on block inclusion
In this commit, we'll attempt to consume a reorg notification for a
transaction that was previously reorged out of the chain upon block
inclusion to ensure that it is not lingering due to a client not
handling it the first time.
2018-10-26 18:32:31 -07:00
Wilmer Paulino
a4dee14b20
chainntnfs/tx_notifier: mark rescan as complete for transactions confirmed at tip
In this commit, we mark the rescan status for a transaction as complete
if we happen to detect it has confirmed within a new block that extends
the chain. We do this as otherwise, it's possible for us to not
immediately dispatch the notification upon a subsequent registration due
to the rescan state machine.
2018-10-26 18:32:31 -07:00
Conner Fromknecht
eee5311557
chainntnfs/txconnotifier: add debug logs, log errs/warnings, godocs 2018-10-26 18:32:31 -07:00
Conner Fromknecht
2dcb86bced
chainntnfs/txconfnotifier: set confset details at tip
This commit ensures that a confSet's details
are assigned in the confNotifications index
after discovering the transaction at tip. The
recent changes allow a later notification to
be dispatched on registration if an earlier one
has already discovered the confirmation details.

Before this change, it was observed that a later
registration would attempt an immediate delivery,
but fail to do so because the confset's details
were nil. This commit remedies that dispatch path,
allowing the integration tests to pass again.
2018-10-26 18:32:30 -07:00
Conner Fromknecht
df9bb56068
chainntnfs/neutrino: use HistoricalConfDispatch in ntfn registry 2018-10-26 18:32:30 -07:00
Conner Fromknecht
6cd0f867ad
chainntnfs/btcd: use HistoricalConfDispatch in ntfn registry 2018-10-26 18:32:30 -07:00
Conner Fromknecht
a4c9f62c6b
chainntnfs/bitcoind: use HistoricalConfDispatch in ntfn registry 2018-10-26 18:32:30 -07:00
Conner Fromknecht
11c231d814
chainntnfs/txconfnotifier_test: remove clientID argument...
to UpdateConfDetails
2018-10-26 18:32:30 -07:00
Conner Fromknecht
32e7368e1e
chainntnfs/txconfnotifier: remove clientID from UpdateConfDetails signature 2018-10-26 18:32:30 -07:00
Conner Fromknecht
9ae6d43916
chainntnfs/txconfnotifier: split out ntfn dispatch into helper 2018-10-26 18:32:30 -07:00
Conner Fromknecht
217b1fc0ef
chainntnfs/txconfnotifier: return HistoricalConfDispatch from Register 2018-10-26 18:32:29 -07:00
Conner Fromknecht
f94de2308f
chainntnfs/txconfnotifier: return HistoricalConfDispatch from Register 2018-10-26 18:32:29 -07:00
Conner Fromknecht
74122e00f5
chainntnfs/txconfnotifier: add PkScript to ConfNtfn 2018-10-26 18:32:29 -07:00
Conner Fromknecht
37c864d6f6
chainntnfs/txconfnotifier: add HistoricalConfDispatch struct 2018-10-26 18:32:29 -07:00
Conner Fromknecht
e804b30669
chainntnfs/txconfnotifier_test: update to use multi-value Register 2018-10-26 18:32:29 -07:00
Conner Fromknecht
cf7700e6cb
chainntnfs/bitcoind+btcd+neutrino: let tcn query for height hint 2018-10-26 18:32:29 -07:00
Conner Fromknecht
8b8007bb5a
chainntnfs/txconfnotifier: query conf hint in Register 2018-10-26 18:32:28 -07:00
Conner Fromknecht
a1756b0b1b
chainntnfs/bitcoind+btcd+neutrino: pass nil conf details 2018-10-26 18:32:28 -07:00
Conner Fromknecht
1babec971f
chainntnfs/txconfnotifier: isolate scanning ntfns 2018-10-26 18:32:28 -07:00
Conner Fromknecht
2f0b5596da
chainntnfs/txconfnotifier: add rescanStates 2018-10-26 18:32:28 -07:00
Conner Fromknecht
012d17efaa
chainntnfs/txnotifier_test: update nil spend details to restore tests 2018-10-26 18:32:28 -07:00
Conner Fromknecht
7661d00d5a
chainntnfs/txconfnotifier_test: update height hint cache test 2018-10-26 18:32:28 -07:00
Conner Fromknecht
d3bde428ea
chainntnfs/txconfnotifier_test: use tcn instead of txConfNotifier 2018-10-26 18:32:28 -07:00
Johan T. Halseth
dbf9b4ea4c
chainntnfs+queue: move ConcurrentQueue to own package 'queue' 2018-10-18 12:38:10 -07:00
Johan T. Halseth
6db0bc4b05
chainntnfs/neutrino_debug: new neutrino API 2018-10-16 19:27:07 -07:00
Johan T. Halseth
e0fd163096
chainntnfs/neutrino: new neutrino API 2018-10-16 19:26:41 -07:00
Conner Fromknecht
59b459674d
multi: init subsystem loggers via build pkg 2018-10-05 13:04:45 +09:00
Conner Fromknecht
3403794e81
chainntnfs: switch debug tag to dev
This avoids collisions with the logging level tags
2018-10-05 12:59:34 +09:00
Conner Fromknecht
91b0e9cae5
chainntnfs/txconfnotifier: commit conf hints before notifying 2018-10-02 01:09:26 -07:00
Conner Fromknecht
7dceac92d2
chainntnfs/neutrino: commit spend hints before notifying 2018-10-02 01:09:26 -07:00
Conner Fromknecht
e498a837a3
chainntnfs/btcd: commit spend hints before notifying 2018-10-02 01:09:26 -07:00
Conner Fromknecht
fb7b12a7db
chainntnfs/bitcoind: commit spend hints before notifying 2018-10-01 16:47:01 -07:00
Olaoluwa Osuntokun
6afee3d099
Merge pull request #1856 from maurycy/typos
multi: fix various typos in comments
2018-09-27 20:38:10 -07:00
Conner Fromknecht
b5b8602820
chainntnfs/neutrinonotify/neutrino: remove height shadowing
This commit removes shadowing of the currentHeight
variable when registering for neutrino spend
notifications. Currently, a locally scoped variable
is used when determining if the backend is fully
synced before attempting to call GetUtxo, which
means that the variable won't be updated after
breaking out of the loop. As a result, this could
cause us to scan unnecessarily if the backend is
catching up, e.g. after being offline for some time.
2018-09-20 18:47:04 -07:00
Wilmer Paulino
2fa64dd6ff multi: update to new BitcoindClient API 2018-09-12 21:05:19 -07:00
maurycy
ac24b12bf2
multi: fix various typos in comments 2018-09-07 06:51:49 +02:00
Valentine Wallace
98d1482942
chainntnfs/interface_test: fix unreliable historical block ntfns test
After joining the two forked chains, it is necessary to ensure they both agree on the same best hash before proceeding to UnsafeStart the notifier.
This is because when the BitcoindClient starts, it retrieves its best known block then calls GetBlockHeaderVerbose on the hash of the retrieved block. This block could be a reorged block if JoinNodes has not completed sync. If it is the case that the best block retrieved has been reorged out of the chain, GetBlockHeaderVerbose errors because bitcoind sets the number of confirmations to -1 on reorged blocks, and the btcd rpc client panics when parsing a block whose number of confirmations is negative.

This parsing error is expected to be fixed, and as a more permanent solution chain backends should ensure that the `best block` they retrieve during startup has not been reorged out of the chain.
2018-08-31 16:18:25 -07:00
Conner Fromknecht
98e7c968d4
chainntnfs/btcdnotify: disable height hint cache in testing 2018-08-26 15:34:20 -07:00
Conner Fromknecht
ab28db5b0d
chainntnfs/bitcoindnotify: disable height hints in testing 2018-08-26 15:34:05 -07:00
Conner Fromknecht
12761a4f43
chainntnfs/interface_test: run tests w/ disabled cache 2018-08-26 15:32:10 -07:00
Conner Fromknecht
45a2c9aca8
chainntnfs/height_hint_cache_test: add tests for disabled cache 2018-08-26 15:32:09 -07:00
Conner Fromknecht
7df9ae0266
chainntnfs/height_hint_cache: add disable flag to hint cache 2018-08-26 15:32:06 -07:00
Conner Fromknecht
20ba489b4a
chainntnfs/interface: fix TxConfStatus godocs for linter 2018-08-24 11:29:09 -07:00
Johan T. Halseth
f82fd5fe86
chainntnfs/bitcoind: fallback to scan manually only in case of err != nil 2018-08-24 14:46:20 +02:00
Johan T. Halseth
01d8953737
chainntnfs/btcd: fallback to scan manually only in case of err != nil 2018-08-24 14:46:20 +02:00
Johan T. Halseth
31dc387e0f
chainntnfs/btcd+bitcoind tests: use common TxConfStatus 2018-08-24 14:46:20 +02:00
Johan T. Halseth
79b2ee7da5
chainntnfs/btcd+bitcoind: use common TxConfStatus 2018-08-24 14:46:20 +02:00
Johan T. Halseth
b552984fcd
chainntnfs: define common TxConfStatus type 2018-08-24 14:46:20 +02:00
Johan T. Halseth
89c33731fe
chainntnfs/bitcoind: remove unnecessary check for tx==nil 2018-08-24 14:46:20 +02:00
Johan T. Halseth
4a60887974
chainntnfs/btcd: remove unnecessary check for tx==nil 2018-08-24 14:46:19 +02:00
Johan T. Halseth
a91466e9fa
chainntnfs/bitcoind tests: check found mempool case 2018-08-24 13:51:54 +02:00
Johan T. Halseth
cb3ec07685
chainntnfs/test_util: set 10ms trickleInterval for miner 2018-08-24 13:50:47 +02:00
Conner Fromknecht
ecc91305ac
chainntnfs/btcdnotify: initialize with height hint cache 2018-08-24 04:05:46 -07:00
Conner Fromknecht
16a321f6c5
chainntnfs/bitcoind: initialize with height hint cache 2018-08-24 04:05:18 -07:00
Olaoluwa Osuntokun
9b6b78a932
chainntnfs/neutrinonotify: update to latest API changes 2018-08-24 03:36:25 -07:00
Wilmer Paulino
191c4f3e28
multi: enable txindex on miner harness 2018-08-24 03:36:25 -07:00
Wilmer Paulino
cb688de7ad
chainntnfs: guard test chain notifier under debug flag 2018-08-24 03:36:25 -07:00
Wilmer Paulino
7895f01e2e
chainntnfs: ensure proper fallback to scanning tx manually
In this commit, we address a bug where it's possible that we still
attempt to manually scan for a transaction to determine whether it's
been included in the chain even after successfully checking the txindex
and not finding it there. Now, we'll short-circuit this process by
exiting early if the txindex lookup was successful but the transaction
in question was not found. Otherwise, we'll fall back to the manual
scan.
2018-08-24 03:36:24 -07:00