Commit Graph

69 Commits

Author SHA1 Message Date
Conner Fromknecht
e10d4e9047
chainntnfs/multi: replace txid with ConfRequest in txindex matching 2019-06-27 20:24:21 -07:00
Conner Fromknecht
91a44c001c
chainntnfs/multi: add interfaced ConfDetailsFromTxIndex
Removes the code duplication between bitcoind and btcd, by creating an
abstracted version of confDetailsFromTxIndex in the main chainntfns
package.
2019-06-27 20:23:16 -07:00
Conner Fromknecht
90ffc4a0ab
chainntnfs/bitcoindnotify: improve historical conf dispatch logging 2019-04-01 18:21:07 -07:00
Conner Fromknecht
7a08825b1e
chainntnfs/bitcoindnotify: always call UpdateSpendDetails
This commit fixes a bug that would cause the
notifier not to commit spend hints for items
that are not found. This is done by calling
UpdateSpendDetails with a nil detail, permitting
the notifier to begin updating the spend hints
with new blocks that arrive at tip. The change
is designed to mimic the behavior for historical
confirmation dispatch.

The symptom of this bug is needing to do many
long rescans on startup, even if new blocks
arrive after the rescan had completed. With
this change, nodes will have to do the scans
once more before their hints will be properly
updated. Restarts from then on should not
have this behavior.
2019-04-01 18:20:52 -07:00
Wilmer Paulino
26d53c64f1
chainntnfs: dispatch block notification for current tip upon registration 2019-01-21 13:58:52 -08:00
Wilmer Paulino
0c579b110c
chainntnfs: add support to cancel pending registered confirmation 2019-01-21 13:58:52 -08:00
Wilmer Paulino
be2c321c8c
chainntnfs: declare common ErrChainNotifierShuttingDown for all backends 2019-01-21 13:58:52 -08:00
Wilmer Paulino
50650d054e
chainntnfs: include transaction in confirmation details 2019-01-21 13:58:52 -08:00
Wilmer Paulino
83c7f204cd
chainntnfs: modify GetTestTxidAndScript to generate new P2PKH scripts
In this commit, we modify GetTestTxidAndScript to generate new P2PKH
scripts. This is needed to properly test confirmations and spends of
unique scripts on-chain within the set of interface-level test cases.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
482f05a3bc
chainntnfs/bitcoindnotify: support registration for script spends
In this commit, we extend the BitcoindNotifier to support registering
scripts for spends notifications. Once the script has been detected as
spent within the chain, a spend notification will be dispatched through
the Spend channel of the SpendEvent returned upon registration.

For scripts that have been spent in the past, the rescan logic has been
modified to match on the script rather than the outpoint. This is done
by re-deriving the script of the output a transaction input is spending
and checking whether it matches ours.

For scripts that are unspent, a request to the backend will be sent to
alert the BitcoindNotifier of when the script was spent by a
transaction. To make this request we encode the script as an address, as
this is what the backend uses to detect the spend. The transaction will
then be proxied through the txUpdates concurrent queue, which will hand
it off to the underlying txNotifier and dispatch spend notifications to
the relevant clients.

Along the way, we also address an issue where we'd miss detecting that
an outpoint/script has been spent in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always request the backend to notify us of the spend
once it detects it at tip, regardless of whether a historical rescan was
detected or not.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
1a41e23bf4
chainntnfs/bitcoindnotify: support registration for script confirmations
In this commit, we extend the BitcoindNotifier to support registering
scripts for confirmation notifications. Once the script has been
detected as confirmed within the chain, a confirmation notification will
be dispatched to through the Confirmed channel of the ConfirmationEvent
returned upon registration.

For scripts that have confirmed in the past, the `historicalConfDetails`
method has been modified to skip the txindex and go straight to scanning
the chain manually if confirmation request is for a script. When
scanning the chain, we'll determine whether the script has been
confirmed by locating the script in an output of a confirmed
transaction.

For scripts that have yet to confirm, they will be properly tracked
within the TxNotifier.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
686e734e22
chainntnfs/bitcoindnotify: add chainParams parameter to bitcoindnotify.New
In this commit, we add the current chain parameters to the
BitcoindNotifier. This will be used in a future commit in order to
convert outputs scripts into addresses. This is needed since the
bitcoind backend uses these addresses to detect whether the script
encoded within it was spent by a transaction in the chain.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
7239e04696
chainntnfs: initialize TxNotifier with new ReorgSafetyLimit 2019-01-11 16:58:15 -08:00
Olaoluwa Osuntokun
28eb8474f4
chainntnfs: modify all historical rescans to scan backwards
In this commit, we modify all existing historical rescans for
ChainNotifier backends to scan backwards rather than forwards. If we
know that a transaction has been confirmed, or outpoint spent, the it's
likely that the event has recently transpired assuming we've been
offline for a short period of time. Therefore, if we scan backwards
rather than forwards, then we can save potentially hundreds or thousands
of block fetches if the event recently happened close to the tip of the
chain.

We bound this search at the genesis block, to ensure we don't underflow
the uint32 used throughout the package in the main loop.
2018-11-24 15:50:12 -06:00
Wilmer Paulino
e402a4e146
chainntnfs: dispatch conf/spend notifications after blocks
In this commit, we alter the different ChainNotifier implementations to
dispatch confirmation and spend notifications after blocks. We do this
to ensure the external consistency of our registered clients.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
5127f2aa82
Revert "chainntnfs/bitcoindnotify: disable height hints in testing"
This reverts commit ab28db5b0dc780a09b7474e187cefff809ac2714.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
0927f35dc1
chainntnfs/bitcoindnotify: remove old spend notification handling logic
In this commit, we remove the old spend notification logic within the
BitcoindNotifier as it's been phased out by the TxNotifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
180dffd154
chainntnfs/bitcoindnotify: handle spend notification registration w/ TxNotifier
In this commit, we modify the logic within RegisterSpendNtfn for the
BitcoindNotifier to account for the recent changes made to the
TxNotifier. Since it is now able to handle spend notification
registration and dispatch, we can bypass all the current logic within
the BitcoindNotifier and interact directly with the TxNotifier instead.

The most notable changes include the following:

  1. We'll only attempt a historical rescan if the TxNotifier tells us
  so.

  2. We'll dispatch the historical rescan within the main goroutine to
  prevent WaitGroup panics, due to the asynchronous nature of the
  notifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
f65401b439
chainntnfs/txnotifier: rename Register -> RegisterConf 2018-10-30 17:59:31 -07:00
Wilmer Paulino
82f6fd7a91
chainntnfs: rename TxConfNotifier -> TxNotifier 2018-10-30 17:59:31 -07:00
Conner Fromknecht
a4c9f62c6b
chainntnfs/bitcoind: use HistoricalConfDispatch in ntfn registry 2018-10-26 18:32:30 -07:00
Conner Fromknecht
cf7700e6cb
chainntnfs/bitcoind+btcd+neutrino: let tcn query for height hint 2018-10-26 18:32:29 -07:00
Conner Fromknecht
a1756b0b1b
chainntnfs/bitcoind+btcd+neutrino: pass nil conf details 2018-10-26 18:32:28 -07:00
Johan T. Halseth
dbf9b4ea4c
chainntnfs+queue: move ConcurrentQueue to own package 'queue' 2018-10-18 12:38:10 -07:00
Conner Fromknecht
3403794e81
chainntnfs: switch debug tag to dev
This avoids collisions with the logging level tags
2018-10-05 12:59:34 +09:00
Conner Fromknecht
fb7b12a7db
chainntnfs/bitcoind: commit spend hints before notifying 2018-10-01 16:47:01 -07:00
Olaoluwa Osuntokun
6afee3d099
Merge pull request #1856 from maurycy/typos
multi: fix various typos in comments
2018-09-27 20:38:10 -07:00
Wilmer Paulino
2fa64dd6ff multi: update to new BitcoindClient API 2018-09-12 21:05:19 -07:00
maurycy
ac24b12bf2
multi: fix various typos in comments 2018-09-07 06:51:49 +02:00
Conner Fromknecht
ab28db5b0d
chainntnfs/bitcoindnotify: disable height hints in testing 2018-08-26 15:34:05 -07:00
Johan T. Halseth
f82fd5fe86
chainntnfs/bitcoind: fallback to scan manually only in case of err != nil 2018-08-24 14:46:20 +02:00
Johan T. Halseth
31dc387e0f
chainntnfs/btcd+bitcoind tests: use common TxConfStatus 2018-08-24 14:46:20 +02:00
Johan T. Halseth
79b2ee7da5
chainntnfs/btcd+bitcoind: use common TxConfStatus 2018-08-24 14:46:20 +02:00
Johan T. Halseth
89c33731fe
chainntnfs/bitcoind: remove unnecessary check for tx==nil 2018-08-24 14:46:20 +02:00
Johan T. Halseth
a91466e9fa
chainntnfs/bitcoind tests: check found mempool case 2018-08-24 13:51:54 +02:00
Conner Fromknecht
16a321f6c5
chainntnfs/bitcoind: initialize with height hint cache 2018-08-24 04:05:18 -07:00
Wilmer Paulino
cb688de7ad
chainntnfs: guard test chain notifier under debug flag 2018-08-24 03:36:25 -07:00
Wilmer Paulino
7895f01e2e
chainntnfs: ensure proper fallback to scanning tx manually
In this commit, we address a bug where it's possible that we still
attempt to manually scan for a transaction to determine whether it's
been included in the chain even after successfully checking the txindex
and not finding it there. Now, we'll short-circuit this process by
exiting early if the txindex lookup was successful but the transaction
in question was not found. Otherwise, we'll fall back to the manual
scan.
2018-08-24 03:36:24 -07:00
Wilmer Paulino
94beabf34b
chainntnfs: cache spend hints within the different chain notifiers
In this commit, we extend the different ChainNotifier implementations to
cache height hints for our spend events. Each outpoint we've requested a
spend notification for will have its initial height hint cached. We then
increment this height hint at every new block for unspent outpoints.
This allows us to retrieve the *exact* height at which the outpoint has
been spent. By doing this, we optimize the different ChainNotifier
implementations since they will no longer have to rescan forward (and
possibly fetch blocks in the neutrino/pruned node case) from the initial
height hint.
2018-08-21 12:55:53 -04:00
Wilmer Paulino
7e872566c4
chainntnfs: query the hint cache before registering a conf ntfn
In this commit, we alter the different chain notifiers to query their
height hint cache before registering a confimation notification. We do
this as it's possible that the cache has a higher height hint, which
can potentially reduce the amount of blocked fetched when attempting
historical dispatches.
2018-08-21 12:55:52 -04:00
Wilmer Paulino
30fd219b1c
chainntnfs: add height hint caches to chain notifiers 2018-08-21 12:55:51 -04:00
Valentine Wallace
b3a5b3b576
chainntnfs: add ChainNotifier test interface for test methods
TestChainNotifier wraps the ChainNotifier interface to allow adding additional testing methods with access to private fields in the notifiers. These testing methods are only compiled when the build tag "debug" is set. UnsafeStart allows starting a notifier with a specified best block.

UnsafeStart is useful for the purpose of testing cases where a notifier's best block is out of date when it receives a new block.
2018-08-11 22:45:03 -07:00
Valentine Wallace
79cbea1c9c
chainntnfs: enable notifiers to catch up on missed blocks
This resolves the situation where a notifier's chain backend skips a series of blocks, causing the notifier to need to dispatch historical block notifications to clients.

Additionally, if the current notifier's best block has been reorged out, this logic enables the notifier to rewind to the common ancestor between the current chain and the outdated best block and dispatches notifications from the ancestor.
2018-08-10 01:08:58 -07:00
Valentine Wallace
3df5b26699
chainntnfs: notify clients after block connect has succeeded
This prevents the situation where we notify clients about a newly connected block, and then the block connection itself fails. We also want to set our best block in between connecting the block and notifying clients, in case a client makes queries about the new block they have received.
2018-08-10 01:08:58 -07:00
Valentine Wallace
cbf1799c40
chainntnfs: rewind chain on missed disconnected blocks
If the chain backend misses telling the notifier about a series of disconnected blocks, the notifier is now able to disconnect the tip to its new best block.
2018-08-10 01:08:57 -07:00
Valentine Wallace
a5e1cf9c97
chainntnfs: dispatch historical block ntfns to clients
If a client passes in their best known block when registering for block notifications, check to see if it's behind our best block. If so, dispatch the missed block notifications to the client.

This is necessary because clients that persist their best known block can miss new blocks while registering for notifications.
2018-08-10 01:08:57 -07:00
Valentine Wallace
d4cf271526
chainntnfs: track best block in btcd and bitcoind 2018-08-10 01:08:57 -07:00
Valentine Wallace
71a81f59a9
chainntnfs: allow clients to pass in best block
Clients can optionally pass their best block known into RegisterBlockEpochNtfn. This enables the notifiers to catch up clients on blocks they may have missed.
2018-08-10 01:08:54 -07:00
Wilmer Paulino
936fcc1f16
multi: update to latest BitcoindClient interface
In this commit, we introduce a nice optimization with regards to lnd's
interaction with a bitcoind backend. Within lnd, we currently have three
different subsystems responsible for watching the chain: chainntnfs,
lnwallet, and routing/chainview. Each of these subsystems has an active
RPC and ZMQ connection to the underlying bitcoind node. This would incur
a toll on the underlying bitcoind node and would cause us to miss ZMQ
events, which are crucial to lnd. We remedy this issue by sharing the
same connection to a bitcoind node between the different clients within
lnd.
2018-08-08 20:26:30 -07:00
Conner Fromknecht
1ded697e8d
multi: sort import paths with gofmt 2018-08-02 18:20:49 -07:00