Commit Graph

300 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
d86fe393cd
multi: update build to latest version of neutrino+btcwallet
In this commit, we update the build to point to the latest version of
neutrino and btcwallet. The latest version of neutrino includes a number
of bug fixes, and new features like reliably transaction broadcast. The
latest version of btcwallet contains a number of bug fixes related to
properly remove invalid transactions from its database.
2019-03-13 15:03:39 -07:00
Olaoluwa Osuntokun
07722e5db2
chainntnfs: skip catch up re-org test for neutrino temporarily
In this commit, we modify the main interface-level tests for neutrino to
skip the block catch up re-org test. We do this, as the test itself is
very unstable and only passes occasionally by the grace of RNGesus.
What's needed to ensure it passes reliably is to extend neutrino with a
cache of the last N blocks disconnected. Sample output of the affected
test when run:
```
   --- SKIP: TestInterfaces/neutrino_test_catch_up_on_missed_blocks_w/_reorged_best_block (0.00s)
        interface_test.go:1574: skipping re-org test for neutrino
```
2019-02-05 19:38:48 -08:00
Johan T. Halseth
19ecc19d55
chainnrnfs/neutrino: return on historicalConf error
Would otherwise update height hint cache even on error during rescan.
2019-02-04 21:31:24 +01:00
Wilmer Paulino
26d53c64f1
chainntnfs: dispatch block notification for current tip upon registration 2019-01-21 13:58:52 -08:00
Wilmer Paulino
568f8271a0
chainntnfs/neutrinonotify: track best block hash 2019-01-21 13:58:52 -08:00
Wilmer Paulino
060f2f7774
chainntnfs: add Done chan to events to signal reorg safety 2019-01-21 13:58:52 -08:00
Wilmer Paulino
52a80f2d37
chainntnfs/txnotifier: track reorg risk for transactions within ConnectTip 2019-01-21 13:58:52 -08:00
Wilmer Paulino
0c579b110c
chainntnfs: add support to cancel pending registered confirmation 2019-01-21 13:58:52 -08:00
Wilmer Paulino
be2c321c8c
chainntnfs: declare common ErrChainNotifierShuttingDown for all backends 2019-01-21 13:58:52 -08:00
Wilmer Paulino
50650d054e
chainntnfs: include transaction in confirmation details 2019-01-21 13:58:52 -08:00
Wilmer Paulino
5ab30cf7ea
chainntnfs/interface_test: add script dispatch test cases 2019-01-21 13:57:43 -08:00
Wilmer Paulino
83c7f204cd
chainntnfs: modify GetTestTxidAndScript to generate new P2PKH scripts
In this commit, we modify GetTestTxidAndScript to generate new P2PKH
scripts. This is needed to properly test confirmations and spends of
unique scripts on-chain within the set of interface-level test cases.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
52db5ed682
chainntnfs/neutrinonotify: support registration for script spends
In this commit, we extend the NeutrinoNotifier to support registering
scripts for spends notifications. Once the script has been detected as
spent within the chain, a spend notification will be dispatched through
the Spend channel of the SpendEvent returned upon registration.

For scripts that have been spent in the past, the rescan logic has been
modified to match on the script rather than the outpoint. A concurrent
queue for relevant transactions has been added to proxy notifications
from the underlying rescan to the txNotifier. This is needed for
scripts, as we cannot perform a historical rescan for scripts through
`GetUtxo`, like we do with outpoints.

For scripts that are unspent, a filter update is sent to the underlying
rescan to ensure that we match and dispatch on the script when
processing new blocks.

Along the way, we also address an issue where we'd miss detecting that
an outpoint/script has been spent in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always request the backend to notify us of the spend
once it detects it at tip, regardless of whether a historical rescan was
detected or not.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
482f05a3bc
chainntnfs/bitcoindnotify: support registration for script spends
In this commit, we extend the BitcoindNotifier to support registering
scripts for spends notifications. Once the script has been detected as
spent within the chain, a spend notification will be dispatched through
the Spend channel of the SpendEvent returned upon registration.

For scripts that have been spent in the past, the rescan logic has been
modified to match on the script rather than the outpoint. This is done
by re-deriving the script of the output a transaction input is spending
and checking whether it matches ours.

For scripts that are unspent, a request to the backend will be sent to
alert the BitcoindNotifier of when the script was spent by a
transaction. To make this request we encode the script as an address, as
this is what the backend uses to detect the spend. The transaction will
then be proxied through the txUpdates concurrent queue, which will hand
it off to the underlying txNotifier and dispatch spend notifications to
the relevant clients.

Along the way, we also address an issue where we'd miss detecting that
an outpoint/script has been spent in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always request the backend to notify us of the spend
once it detects it at tip, regardless of whether a historical rescan was
detected or not.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
808c6ae660
chainntnfs/btcdnotify: support registration for script spends
In this commit, we extend the BtcdNotifier to support registering
scripts for spends notifications. Once the script has been detected as
spent within the chain, a spend notification will be dispatched through
the Spend channel of the SpendEvent returned upon registration.

For scripts that have been spent in the past, the rescan logic has been
modified to match on the script rather than the outpoint. This is done
by encoding the script as an address.

For scripts that are unspent, a request to the backend will be sent to
alert the BtcdNotifier of when the script was spent by a transaction. To
make this request we encode the script as an address, as this is what
the backend uses to detect the spend. The transaction will then be
proxied through the txUpdates concurrent queue, which will hand it off
to the underlying txNotifier and dispatch spend notifications to the
relevant clients.

Along the way, we also address an issue where we'd miss detecting that
an outpoint/script has been spent in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always request the backend to notify us of the spend
once it detects it at tip, regardless of whether a historical rescan was
detected or not.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
f02590d8c0
chainntnfs/neutrinonotify: support registration for script confirmations
In this commit, we extend the NeutrinoNotifier to support registering
scripts for confirmation notifications. Once the script has been
detected as confirmed within the chain, a confirmation notification will
be dispatched to through the Confirmed channel of the ConfirmationEvent
returned upon registration.

For scripts that have confirmed in the past, the `historicalConfDetails`
method has been modified to determine whether the script has been
confirmed by locating the script in an output of a confirmed
transaction.

For scripts that have yet to confirm, a filter update is sent to the
underlying rescan to ensure that we match and dispatch on the script
when processing new blocks.

Along the way, we also address an issue where we'd miss detecting that a
transaction/script has confirmed in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always update our filters to detect the confirmation
at tip, regardless of whether a historical rescan was detected or not.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
1a41e23bf4
chainntnfs/bitcoindnotify: support registration for script confirmations
In this commit, we extend the BitcoindNotifier to support registering
scripts for confirmation notifications. Once the script has been
detected as confirmed within the chain, a confirmation notification will
be dispatched to through the Confirmed channel of the ConfirmationEvent
returned upon registration.

For scripts that have confirmed in the past, the `historicalConfDetails`
method has been modified to skip the txindex and go straight to scanning
the chain manually if confirmation request is for a script. When
scanning the chain, we'll determine whether the script has been
confirmed by locating the script in an output of a confirmed
transaction.

For scripts that have yet to confirm, they will be properly tracked
within the TxNotifier.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
1323c92947
chainntnfs/btcdnotify: support registration for script confirmations
In this commit, we extend the BtcdNotifier to support registering
scripts for confirmation notifications. Once the script has been
detected as confirmed within the chain, a confirmation notification will
be dispatched to through the Confirmed channel of the ConfirmationEvent
returned upon registration.

For scripts that have confirmed in the past, the `historicalConfDetails`
method has been modified to skip the txindex and go straight to scanning
the chain manually if confirmation request is for a script. When
scanning the chain, we'll determine whether the script has been
confirmed by locating the script in an output of a confirmed
transaction.

For scripts that have yet to confirm, they will be properly tracked
within the TxNotifier.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
72cc843e92
chainntnfs/interface: update docs to note support for scripts 2019-01-21 13:57:43 -08:00
Wilmer Paulino
c72da48111
multi: update to latest ChainNotifier impl New method 2019-01-21 13:57:43 -08:00
Wilmer Paulino
969acf6145
chainntnfs/neutrinonotify: clean up neutrinonotify.New 2019-01-21 13:57:43 -08:00
Wilmer Paulino
686e734e22
chainntnfs/bitcoindnotify: add chainParams parameter to bitcoindnotify.New
In this commit, we add the current chain parameters to the
BitcoindNotifier. This will be used in a future commit in order to
convert outputs scripts into addresses. This is needed since the
bitcoind backend uses these addresses to detect whether the script
encoded within it was spent by a transaction in the chain.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
1ee8d7518c
chainntnfs/btcdnotify: add chainParams parameter to btcdnotify.New
In this commit, we add the current chain parameters to the BtcdNotifier.
This will be used in a future commit in order to convert outputs scripts
into addresses. This is needed since the btcd backend uses these
addresses to detect whether the script encoded within it was spent by a
transaction in the chain.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
7f644019b8
chainntnfs/txnotifier_test: update tests with new registration changes 2019-01-21 13:57:43 -08:00
Wilmer Paulino
c3d5204f09
chainntnfs/txnotifier: refactor ProcessRelevantSpendTx to detect script spends
In this commit, we refactor the TxNotifier's ProcessRelevantSpendTx to
also detect script spends. This can easily be done as the transaction
filtering logic was refactored into its own method in a previous commit.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
8042d4f1c8
chainntnfs/txnotifier: batch update confirm and spends hints for scripts 2019-01-21 13:57:43 -08:00
Wilmer Paulino
f4f5c1ef8b
chainntnfs/txnotifier: support reorg detection for scripts 2019-01-21 13:57:43 -08:00
Wilmer Paulino
83c8ccb3eb
chainntnfs/txnotifier: notify confirmations and spends of scripts at tip 2019-01-21 13:57:43 -08:00
Wilmer Paulino
ecd70deb8c
chainntnfs/txnotifer: detect confirmations and spends of scripts at tip
In this commit, we modify the TxNotifier's ConnectTip method to also
detect whether a registered script has been confirmed or spent on-chain
by a transaction in the connected block. Once detected, notifications
for them will be queued up for dispatch as with txids/outpoints.

We've also refactored the ConnectTip method into smaller and reusable
methods, which will serve useful later.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
b579c23310
chainntnfs/txnotifier: support registration of scripts for spends 2019-01-21 13:57:43 -08:00
Wilmer Paulino
0f4ff2d09c
chainntnfs/txnotifier: support registration of scripts for confirmations 2019-01-21 13:57:43 -08:00
Wilmer Paulino
a0d3b7d9e3
chainntnfs: support caching confirm/spend hints for scripts
In this commit, we refactor the HeightHintCache and its underlying
interfaces to be able to manipulate hints for ConfRequests and
SpendRequests. By doing so, we'll be able to manipulate hints for
scripts if the request includes either a zero hash or a zero outpoint.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
de0adc4d52
chainntnfs/txnotifier: add ConfRequest and SpendRequest structs
These structs allow us to request notifications for either
txids/outpoints or output scripts. We'll be using these in later commits
to index confirmation and spend requests within the TxNotifier.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
5be8f08b16
chainntnfs/txnotiifer: define MaxNumConfs supported as ReorgSafetyLimit 2019-01-11 16:58:15 -08:00
Wilmer Paulino
fd2bfaa007
chainntnfs: add TestTxNotifierMaxConfs 2019-01-11 16:58:15 -08:00
Wilmer Paulino
7239e04696
chainntnfs: initialize TxNotifier with new ReorgSafetyLimit 2019-01-11 16:58:15 -08:00
Wilmer Paulino
9d51114546
chainntnfs/txnotifier: define ReorgSafetyLimit as 144 blocks 2019-01-11 16:58:15 -08:00
Olaoluwa Osuntokun
28eb8474f4
chainntnfs: modify all historical rescans to scan backwards
In this commit, we modify all existing historical rescans for
ChainNotifier backends to scan backwards rather than forwards. If we
know that a transaction has been confirmed, or outpoint spent, the it's
likely that the event has recently transpired assuming we've been
offline for a short period of time. Therefore, if we scan backwards
rather than forwards, then we can save potentially hundreds or thousands
of block fetches if the event recently happened close to the tip of the
chain.

We bound this search at the genesis block, to ensure we don't underflow
the uint32 used throughout the package in the main loop.
2018-11-24 15:50:12 -06:00
Wilmer Paulino
e402a4e146
chainntnfs: dispatch conf/spend notifications after blocks
In this commit, we alter the different ChainNotifier implementations to
dispatch confirmation and spend notifications after blocks. We do this
to ensure the external consistency of our registered clients.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
770e005943
chainntnfs/txnotifer: prevent dispatching notifications within ConnectTip
In this commit, we modify the TxNotifier's ConnectTip method to no
longer dispatch notifications to any clients who had a request fulfilled
within the height connected. Instead, it will queue the notifications
for dispatch and we add a new method NotifyHeight, which will actually
dispatch them. We do this to allow the users of the TxNotifier to be
more flexible when dispatching notifications.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
9a025867d0
Revert "chainntnfs/btcdnotify: disable height hint cache in testing"
This reverts commit 98e7c968d4bf5900a5d7be7f557eab8f623633c0.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
5127f2aa82
Revert "chainntnfs/bitcoindnotify: disable height hints in testing"
This reverts commit ab28db5b0dc780a09b7474e187cefff809ac2714.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
35bfb79099
Revert "chainntnfs/interface_test: run tests w/ disabled cache"
This reverts commit 12761a4f435a033e8425dbcb3b407bb094fcae9f.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
88ac985c95
Revert "chainntnfs/height_hint_cache_test: add tests for disabled cache"
This reverts commit 45a2c9aca8f10f341848845e1f51261bd09fcb68.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
716c20b18d
Revert "chainntnfs/height_hint_cache: add disable flag to hint cache"
This reverts commit 7df9ae026665de75175853e486143ab80122ce9d.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
e6b1a27cd7
chainntnfs/neutrinonotify: make filter update synchronous
In this commit, we modify the notifier to handle filter updates
synchronously. We do this to prevent race conditions between new block
notifications and filter updates. Otherwise, it's possible for a new
block to come in that should match our filter, but doesn't due to the
filter being updated after.

We also modify their order so that the filter is updated first. We do
this so we can immediately start watching for the event at tip while the
rescan is ongoing.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
60a1d73e08
chainntnfs/txnotifier: commit height hint after rescan is complete
In this commit, we'll now commit the current height of the TxNotifier as
the height hint for an outpoint/transaction after a rescan has been
completed and has determined that the outpoint is unspent/transaction is
unconfirmed. We do this to prevent another potentially long rescan if
the daemon is restarted before a new block comes in (which is when the
hints will be updated again).
2018-10-30 17:59:31 -07:00
Wilmer Paulino
6458868ba0
chainntnfs/interface_test: add spend reorg test 2018-10-30 17:59:31 -07:00
Wilmer Paulino
deca4cfe44
chainntnfs/neutrinonotify: remove old spend notification handling logic
In this commit, we remove the old spend notification logic within the
NeutrinoNotifier as it's been phased out by the TxNotifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
bfd11a251e
chainntnfs/neutrinonotify: handle spend notification registration w/ TxNotifier
In this commit, we modify the logic within RegisterSpendNtfn for the
NeutrinoNotifier to account for the recent changes made to the
TxNotifier. Since it is now able to handle spend notification
registration and dispatch, we can bypass all the current logic within
the NeutrinoNotifier and interact directly with the TxNotifier instead.

The most notable change is that now we'll only attempt a historical
rescan if the TxNotifier tells us so.
2018-10-30 17:59:31 -07:00