Commit Graph

56 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
28eb8474f4
chainntnfs: modify all historical rescans to scan backwards
In this commit, we modify all existing historical rescans for
ChainNotifier backends to scan backwards rather than forwards. If we
know that a transaction has been confirmed, or outpoint spent, the it's
likely that the event has recently transpired assuming we've been
offline for a short period of time. Therefore, if we scan backwards
rather than forwards, then we can save potentially hundreds or thousands
of block fetches if the event recently happened close to the tip of the
chain.

We bound this search at the genesis block, to ensure we don't underflow
the uint32 used throughout the package in the main loop.
2018-11-24 15:50:12 -06:00
Wilmer Paulino
e402a4e146
chainntnfs: dispatch conf/spend notifications after blocks
In this commit, we alter the different ChainNotifier implementations to
dispatch confirmation and spend notifications after blocks. We do this
to ensure the external consistency of our registered clients.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
5127f2aa82
Revert "chainntnfs/bitcoindnotify: disable height hints in testing"
This reverts commit ab28db5b0dc780a09b7474e187cefff809ac2714.
2018-10-31 09:20:22 -07:00
Wilmer Paulino
0927f35dc1
chainntnfs/bitcoindnotify: remove old spend notification handling logic
In this commit, we remove the old spend notification logic within the
BitcoindNotifier as it's been phased out by the TxNotifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
180dffd154
chainntnfs/bitcoindnotify: handle spend notification registration w/ TxNotifier
In this commit, we modify the logic within RegisterSpendNtfn for the
BitcoindNotifier to account for the recent changes made to the
TxNotifier. Since it is now able to handle spend notification
registration and dispatch, we can bypass all the current logic within
the BitcoindNotifier and interact directly with the TxNotifier instead.

The most notable changes include the following:

  1. We'll only attempt a historical rescan if the TxNotifier tells us
  so.

  2. We'll dispatch the historical rescan within the main goroutine to
  prevent WaitGroup panics, due to the asynchronous nature of the
  notifier.
2018-10-30 17:59:31 -07:00
Wilmer Paulino
f65401b439
chainntnfs/txnotifier: rename Register -> RegisterConf 2018-10-30 17:59:31 -07:00
Wilmer Paulino
82f6fd7a91
chainntnfs: rename TxConfNotifier -> TxNotifier 2018-10-30 17:59:31 -07:00
Conner Fromknecht
a4c9f62c6b
chainntnfs/bitcoind: use HistoricalConfDispatch in ntfn registry 2018-10-26 18:32:30 -07:00
Conner Fromknecht
cf7700e6cb
chainntnfs/bitcoind+btcd+neutrino: let tcn query for height hint 2018-10-26 18:32:29 -07:00
Conner Fromknecht
a1756b0b1b
chainntnfs/bitcoind+btcd+neutrino: pass nil conf details 2018-10-26 18:32:28 -07:00
Johan T. Halseth
dbf9b4ea4c
chainntnfs+queue: move ConcurrentQueue to own package 'queue' 2018-10-18 12:38:10 -07:00
Conner Fromknecht
3403794e81
chainntnfs: switch debug tag to dev
This avoids collisions with the logging level tags
2018-10-05 12:59:34 +09:00
Conner Fromknecht
fb7b12a7db
chainntnfs/bitcoind: commit spend hints before notifying 2018-10-01 16:47:01 -07:00
Olaoluwa Osuntokun
6afee3d099
Merge pull request #1856 from maurycy/typos
multi: fix various typos in comments
2018-09-27 20:38:10 -07:00
Wilmer Paulino
2fa64dd6ff multi: update to new BitcoindClient API 2018-09-12 21:05:19 -07:00
maurycy
ac24b12bf2
multi: fix various typos in comments 2018-09-07 06:51:49 +02:00
Conner Fromknecht
ab28db5b0d
chainntnfs/bitcoindnotify: disable height hints in testing 2018-08-26 15:34:05 -07:00
Johan T. Halseth
f82fd5fe86
chainntnfs/bitcoind: fallback to scan manually only in case of err != nil 2018-08-24 14:46:20 +02:00
Johan T. Halseth
31dc387e0f
chainntnfs/btcd+bitcoind tests: use common TxConfStatus 2018-08-24 14:46:20 +02:00
Johan T. Halseth
79b2ee7da5
chainntnfs/btcd+bitcoind: use common TxConfStatus 2018-08-24 14:46:20 +02:00
Johan T. Halseth
89c33731fe
chainntnfs/bitcoind: remove unnecessary check for tx==nil 2018-08-24 14:46:20 +02:00
Johan T. Halseth
a91466e9fa
chainntnfs/bitcoind tests: check found mempool case 2018-08-24 13:51:54 +02:00
Conner Fromknecht
16a321f6c5
chainntnfs/bitcoind: initialize with height hint cache 2018-08-24 04:05:18 -07:00
Wilmer Paulino
cb688de7ad
chainntnfs: guard test chain notifier under debug flag 2018-08-24 03:36:25 -07:00
Wilmer Paulino
7895f01e2e
chainntnfs: ensure proper fallback to scanning tx manually
In this commit, we address a bug where it's possible that we still
attempt to manually scan for a transaction to determine whether it's
been included in the chain even after successfully checking the txindex
and not finding it there. Now, we'll short-circuit this process by
exiting early if the txindex lookup was successful but the transaction
in question was not found. Otherwise, we'll fall back to the manual
scan.
2018-08-24 03:36:24 -07:00
Wilmer Paulino
94beabf34b
chainntnfs: cache spend hints within the different chain notifiers
In this commit, we extend the different ChainNotifier implementations to
cache height hints for our spend events. Each outpoint we've requested a
spend notification for will have its initial height hint cached. We then
increment this height hint at every new block for unspent outpoints.
This allows us to retrieve the *exact* height at which the outpoint has
been spent. By doing this, we optimize the different ChainNotifier
implementations since they will no longer have to rescan forward (and
possibly fetch blocks in the neutrino/pruned node case) from the initial
height hint.
2018-08-21 12:55:53 -04:00
Wilmer Paulino
7e872566c4
chainntnfs: query the hint cache before registering a conf ntfn
In this commit, we alter the different chain notifiers to query their
height hint cache before registering a confimation notification. We do
this as it's possible that the cache has a higher height hint, which
can potentially reduce the amount of blocked fetched when attempting
historical dispatches.
2018-08-21 12:55:52 -04:00
Wilmer Paulino
30fd219b1c
chainntnfs: add height hint caches to chain notifiers 2018-08-21 12:55:51 -04:00
Valentine Wallace
b3a5b3b576
chainntnfs: add ChainNotifier test interface for test methods
TestChainNotifier wraps the ChainNotifier interface to allow adding additional testing methods with access to private fields in the notifiers. These testing methods are only compiled when the build tag "debug" is set. UnsafeStart allows starting a notifier with a specified best block.

UnsafeStart is useful for the purpose of testing cases where a notifier's best block is out of date when it receives a new block.
2018-08-11 22:45:03 -07:00
Valentine Wallace
79cbea1c9c
chainntnfs: enable notifiers to catch up on missed blocks
This resolves the situation where a notifier's chain backend skips a series of blocks, causing the notifier to need to dispatch historical block notifications to clients.

Additionally, if the current notifier's best block has been reorged out, this logic enables the notifier to rewind to the common ancestor between the current chain and the outdated best block and dispatches notifications from the ancestor.
2018-08-10 01:08:58 -07:00
Valentine Wallace
3df5b26699
chainntnfs: notify clients after block connect has succeeded
This prevents the situation where we notify clients about a newly connected block, and then the block connection itself fails. We also want to set our best block in between connecting the block and notifying clients, in case a client makes queries about the new block they have received.
2018-08-10 01:08:58 -07:00
Valentine Wallace
cbf1799c40
chainntnfs: rewind chain on missed disconnected blocks
If the chain backend misses telling the notifier about a series of disconnected blocks, the notifier is now able to disconnect the tip to its new best block.
2018-08-10 01:08:57 -07:00
Valentine Wallace
a5e1cf9c97
chainntnfs: dispatch historical block ntfns to clients
If a client passes in their best known block when registering for block notifications, check to see if it's behind our best block. If so, dispatch the missed block notifications to the client.

This is necessary because clients that persist their best known block can miss new blocks while registering for notifications.
2018-08-10 01:08:57 -07:00
Valentine Wallace
d4cf271526
chainntnfs: track best block in btcd and bitcoind 2018-08-10 01:08:57 -07:00
Valentine Wallace
71a81f59a9
chainntnfs: allow clients to pass in best block
Clients can optionally pass their best block known into RegisterBlockEpochNtfn. This enables the notifiers to catch up clients on blocks they may have missed.
2018-08-10 01:08:54 -07:00
Wilmer Paulino
936fcc1f16
multi: update to latest BitcoindClient interface
In this commit, we introduce a nice optimization with regards to lnd's
interaction with a bitcoind backend. Within lnd, we currently have three
different subsystems responsible for watching the chain: chainntnfs,
lnwallet, and routing/chainview. Each of these subsystems has an active
RPC and ZMQ connection to the underlying bitcoind node. This would incur
a toll on the underlying bitcoind node and would cause us to miss ZMQ
events, which are crucial to lnd. We remedy this issue by sharing the
same connection to a bitcoind node between the different clients within
lnd.
2018-08-08 20:26:30 -07:00
Conner Fromknecht
1ded697e8d
multi: sort import paths with gofmt 2018-08-02 18:20:49 -07:00
Olaoluwa Osuntokun
6781b17056
chainntnfs: update bitcoind and btcd backends to match new spend ntfn API 2018-07-31 21:28:53 -07:00
Olaoluwa Osuntokun
f45e41afc1
chainntnfs/bitcoindnotify: update RegisterConfirmationsNtfn to take pkScript 2018-07-31 21:28:48 -07:00
Wilmer Paulino
65b6257e1e chainntnfs/bitcoindnotify: make historical spend rescans async 2018-07-31 18:23:25 -07:00
Wilmer Paulino
12816a910d chainntnfs: make historical confirmation rescans async 2018-07-31 18:23:25 -07:00
Wilmer Paulino
c43506dee9 chainntnfs: add unique ID field to track conf ntfns within notifier 2018-07-31 18:23:25 -07:00
Johan T. Halseth
181014c363
chainntnfs/bitcoindnotify: remove all mempool spend clients 2018-07-22 23:09:09 +02:00
Johan T. Halseth
44d7b84df0
chainntnfs: remove mempool option from RegisterSpendNtfn 2018-07-22 23:09:08 +02:00
Johan T. Halseth
4e47e4e7f1
chainntnfs/bitcoindnotify: filter out mempool spends from relevant txs
This commit fix a bug within the bitcoind notifier logic, which would
ignore the passed mempool argument, and notify spentness whether the
spending transaction was confirmed or not. The logic used to fix this is
similar to what is already done for the btcd backend.
2018-07-16 12:27:05 +02:00
Olaoluwa Osuntokun
6f60f139f4 multi: switch over import paths from roasbeef/* to btcsuite/* 2018-07-13 17:05:39 -07:00
Olaoluwa Osuntokun
005510b54f
chainntnfs/bitcoindnotify: fix possible panic with lack of txindex
In this commit, we fix a recently introduced bug which can result in a
panic when bitcoind nodes without a txindex active are started. The
issue was that we would still defence the transaction's blockhash, which
would be nil if we detected that the backend didn't have the txindex
active.
2018-04-17 20:08:58 -07:00
Wilmer Paulino
13fb866574
chainntnfs: remove txindex requirement when registering notfications
Before this commit, we relied on the need of full nodes to enable the
transaction index. This allowed us to fetch historical details about
transactions in order to register and dispatch confirmation and spend
notifications.

This commit allows us to drop that requirement by providing a fallback
method to use when the transaction index is not enabled. This fallback
method relies on manually scanning blocks for the transactions
requested, starting from the earliest height the transactions could have
been included in, to the current height in the chain.
2018-04-17 14:59:51 -04:00
Olaoluwa Osuntokun
86ad6d318e
Merge pull request #889 from wpaulino/chainnotifier-updates
chainntnfs: add incremental update notifications within ChainNotifier
2018-04-12 19:21:26 -07:00
Alex
89e2ba41c9 chainntnfs/bitcoindnotify: rescan blocks manually instead of rewinding 2018-04-06 20:35:27 -06:00