Commit Graph

325 Commits

Author SHA1 Message Date
Johan T. Halseth
753f30a6ff
chainntnfs/neutrino: don't ignore failed filter fetches
Earlier GetCFilter could return a nil-error along with a nil-filter in
cases where we failed to fetch the filter. We would just ignore the
block causing us to miss potential crucial information.

Now that GetCFilter will return an error in cases where the fetch
failed, the filter will never be nil (even for blocks with only coinbase
transactions), and we'll instead return an error.

We also increase the number of retries for the filter from the default 2
to 5.
2019-10-15 11:25:31 +02:00
Wilmer Paulino
194a9dea81
multi: support sync freelist option within btcwallet 2019-10-04 12:19:40 -04:00
Johan T. Halseth
e88c635b7e
Merge pull request #3573 from wpaulino/log-conf-ntfn-once
chainntnfs: only log confirmation notification registration once
2019-10-04 13:06:42 +02:00
Wilmer Paulino
e51df8d810
chainntnfs: only log confirmation notification registration once 2019-10-03 14:05:28 -04:00
Joost Jager
b58dbb2d70
multi: fix canceled spelling 2019-10-03 17:27:36 +02:00
Johan T. Halseth
d195acc632
chainntnfs/neutrino: start concurrent queues prior to connection
Similar to what was done for btcd, just to make sure we won't be blocked
on any incoming notifications, start the queues first thing during
startup.
2019-09-16 08:54:06 +02:00
Johan T. Halseth
6b6beb4d7d
chainntnfs/btcd: start concurrent queues prior to connection
We would establish the connection to the chain backend and start getting
block notifications before we had started the concurrent queues, which
would lead to the OnBlockConnected call being blocked, and a deadlock
(since GetBestBlock would never return).

Instead we make sure to start the queues before establishing the
connection, consuming the notifications right away.
2019-09-16 08:54:06 +02:00
Olaoluwa Osuntokun
5ecde6d552
lnwallet+chainntnfs: update to use latest neutrino APIs 2019-09-11 05:57:06 -07:00
Joost Jager
3d7de2ad39
multi: remove dead code 2019-09-10 17:21:59 +02:00
Wilmer Paulino
2f6ec77b3c
chainntnfs/interface_test: lower mempool spend check timeout
In this commit, we lower the mempool spend check timeout to be twice as
long as the trickle interval of the miner node, which will greatly
improve the execution time of this specific test. We're able to do this
now since we can specify custom trickle intervals for our test
harnesses.
2019-08-29 18:05:46 -07:00
Wilmer Paulino
cb1d6683d9
chainntnfs: acquire TxNotifier lock after closing quit chan
This prevents a deadlock while tearing down the TxNotifier if it's
currently blocked delivering a notification. By closing the quit chan
first, we ensure blocked sends/reads can exit and allow the TxNotifier
to proceed tearing down.
2019-08-22 13:49:38 -07:00
Wilmer Paulino
17b6205f3a
chainntnfs: validate conf/spend ntfn registration parameters
A height hint not being set would cause lnd to scan for the
confirmation/spend of a txid/outpoint/address from genesis.

The number of confirmations not being set within a confirmation request
would cause the internal TxNotifier to deadlock when dispatching
updates.
2019-08-22 13:49:37 -07:00
Wilmer Paulino
7821eb6ffb
chainntnfs: refactor common registration code within RegisterSpendNtfn 2019-08-22 13:49:35 -07:00
Wilmer Paulino
5089311ca1
chainntnfs: refactor common registration code within RegisterConfirmationsNtfn 2019-08-22 13:49:33 -07:00
Conner Fromknecht
d5da18827e
chainntnfs/interface: stricter conf dispatch for txindex 2019-06-27 20:24:23 -07:00
Conner Fromknecht
50f7f7f1b7
chainntnfs/txnotifier: stricter confirmation matching via scripts 2019-06-27 20:24:23 -07:00
Conner Fromknecht
e10d4e9047
chainntnfs/multi: replace txid with ConfRequest in txindex matching 2019-06-27 20:24:21 -07:00
Conner Fromknecht
922a980bd7
chainntnfs/interface: fix incorrect statuses for deserialization errors 2019-06-27 20:23:18 -07:00
Conner Fromknecht
91a44c001c
chainntnfs/multi: add interfaced ConfDetailsFromTxIndex
Removes the code duplication between bitcoind and btcd, by creating an
abstracted version of confDetailsFromTxIndex in the main chainntfns
package.
2019-06-27 20:23:16 -07:00
Johan T. Halseth
3a44574625
chainntnfs/neutrinonotify: async call GetUtxo from RegisterSpendNtfn
Since GetUtxo is a potentially long running call, we would see
RegisterSpendNtfn block in some cases, in particular on starting the
chain watcher, causing lnd to hang on startup.

This commit makes the call to GetUtxo in a go routine, letting
RegisterSpendNtfn return immediately when the notification events are
created.
2019-06-04 09:58:04 +02:00
Johan T. Halseth
183adf6e61
chainntnfs/neutrino: provide n.quit as neutrino QuitChan option
This will make sure a long-running rescan can be canceled in the case
the notifier is shutting down.
2019-05-09 14:44:53 +02:00
Conner Fromknecht
17b2140cb5
multi: fix spelling mistakes 2019-05-04 15:35:37 -07:00
Conner Fromknecht
90ffc4a0ab
chainntnfs/bitcoindnotify: improve historical conf dispatch logging 2019-04-01 18:21:07 -07:00
Conner Fromknecht
7a08825b1e
chainntnfs/bitcoindnotify: always call UpdateSpendDetails
This commit fixes a bug that would cause the
notifier not to commit spend hints for items
that are not found. This is done by calling
UpdateSpendDetails with a nil detail, permitting
the notifier to begin updating the spend hints
with new blocks that arrive at tip. The change
is designed to mimic the behavior for historical
confirmation dispatch.

The symptom of this bug is needing to do many
long rescans on startup, even if new blocks
arrive after the rescan had completed. With
this change, nodes will have to do the scans
once more before their hints will be properly
updated. Restarts from then on should not
have this behavior.
2019-04-01 18:20:52 -07:00
Wilmer Paulino
bde417e2ef
chainntnfs: fix RegisterSpend and RegisterConf height hint logs 2019-03-26 18:13:02 -07:00
Olaoluwa Osuntokun
d86fe393cd
multi: update build to latest version of neutrino+btcwallet
In this commit, we update the build to point to the latest version of
neutrino and btcwallet. The latest version of neutrino includes a number
of bug fixes, and new features like reliably transaction broadcast. The
latest version of btcwallet contains a number of bug fixes related to
properly remove invalid transactions from its database.
2019-03-13 15:03:39 -07:00
Olaoluwa Osuntokun
07722e5db2
chainntnfs: skip catch up re-org test for neutrino temporarily
In this commit, we modify the main interface-level tests for neutrino to
skip the block catch up re-org test. We do this, as the test itself is
very unstable and only passes occasionally by the grace of RNGesus.
What's needed to ensure it passes reliably is to extend neutrino with a
cache of the last N blocks disconnected. Sample output of the affected
test when run:
```
   --- SKIP: TestInterfaces/neutrino_test_catch_up_on_missed_blocks_w/_reorged_best_block (0.00s)
        interface_test.go:1574: skipping re-org test for neutrino
```
2019-02-05 19:38:48 -08:00
Johan T. Halseth
19ecc19d55
chainnrnfs/neutrino: return on historicalConf error
Would otherwise update height hint cache even on error during rescan.
2019-02-04 21:31:24 +01:00
Wilmer Paulino
26d53c64f1
chainntnfs: dispatch block notification for current tip upon registration 2019-01-21 13:58:52 -08:00
Wilmer Paulino
568f8271a0
chainntnfs/neutrinonotify: track best block hash 2019-01-21 13:58:52 -08:00
Wilmer Paulino
060f2f7774
chainntnfs: add Done chan to events to signal reorg safety 2019-01-21 13:58:52 -08:00
Wilmer Paulino
52a80f2d37
chainntnfs/txnotifier: track reorg risk for transactions within ConnectTip 2019-01-21 13:58:52 -08:00
Wilmer Paulino
0c579b110c
chainntnfs: add support to cancel pending registered confirmation 2019-01-21 13:58:52 -08:00
Wilmer Paulino
be2c321c8c
chainntnfs: declare common ErrChainNotifierShuttingDown for all backends 2019-01-21 13:58:52 -08:00
Wilmer Paulino
50650d054e
chainntnfs: include transaction in confirmation details 2019-01-21 13:58:52 -08:00
Wilmer Paulino
5ab30cf7ea
chainntnfs/interface_test: add script dispatch test cases 2019-01-21 13:57:43 -08:00
Wilmer Paulino
83c7f204cd
chainntnfs: modify GetTestTxidAndScript to generate new P2PKH scripts
In this commit, we modify GetTestTxidAndScript to generate new P2PKH
scripts. This is needed to properly test confirmations and spends of
unique scripts on-chain within the set of interface-level test cases.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
52db5ed682
chainntnfs/neutrinonotify: support registration for script spends
In this commit, we extend the NeutrinoNotifier to support registering
scripts for spends notifications. Once the script has been detected as
spent within the chain, a spend notification will be dispatched through
the Spend channel of the SpendEvent returned upon registration.

For scripts that have been spent in the past, the rescan logic has been
modified to match on the script rather than the outpoint. A concurrent
queue for relevant transactions has been added to proxy notifications
from the underlying rescan to the txNotifier. This is needed for
scripts, as we cannot perform a historical rescan for scripts through
`GetUtxo`, like we do with outpoints.

For scripts that are unspent, a filter update is sent to the underlying
rescan to ensure that we match and dispatch on the script when
processing new blocks.

Along the way, we also address an issue where we'd miss detecting that
an outpoint/script has been spent in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always request the backend to notify us of the spend
once it detects it at tip, regardless of whether a historical rescan was
detected or not.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
482f05a3bc
chainntnfs/bitcoindnotify: support registration for script spends
In this commit, we extend the BitcoindNotifier to support registering
scripts for spends notifications. Once the script has been detected as
spent within the chain, a spend notification will be dispatched through
the Spend channel of the SpendEvent returned upon registration.

For scripts that have been spent in the past, the rescan logic has been
modified to match on the script rather than the outpoint. This is done
by re-deriving the script of the output a transaction input is spending
and checking whether it matches ours.

For scripts that are unspent, a request to the backend will be sent to
alert the BitcoindNotifier of when the script was spent by a
transaction. To make this request we encode the script as an address, as
this is what the backend uses to detect the spend. The transaction will
then be proxied through the txUpdates concurrent queue, which will hand
it off to the underlying txNotifier and dispatch spend notifications to
the relevant clients.

Along the way, we also address an issue where we'd miss detecting that
an outpoint/script has been spent in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always request the backend to notify us of the spend
once it detects it at tip, regardless of whether a historical rescan was
detected or not.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
808c6ae660
chainntnfs/btcdnotify: support registration for script spends
In this commit, we extend the BtcdNotifier to support registering
scripts for spends notifications. Once the script has been detected as
spent within the chain, a spend notification will be dispatched through
the Spend channel of the SpendEvent returned upon registration.

For scripts that have been spent in the past, the rescan logic has been
modified to match on the script rather than the outpoint. This is done
by encoding the script as an address.

For scripts that are unspent, a request to the backend will be sent to
alert the BtcdNotifier of when the script was spent by a transaction. To
make this request we encode the script as an address, as this is what
the backend uses to detect the spend. The transaction will then be
proxied through the txUpdates concurrent queue, which will hand it off
to the underlying txNotifier and dispatch spend notifications to the
relevant clients.

Along the way, we also address an issue where we'd miss detecting that
an outpoint/script has been spent in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always request the backend to notify us of the spend
once it detects it at tip, regardless of whether a historical rescan was
detected or not.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
f02590d8c0
chainntnfs/neutrinonotify: support registration for script confirmations
In this commit, we extend the NeutrinoNotifier to support registering
scripts for confirmation notifications. Once the script has been
detected as confirmed within the chain, a confirmation notification will
be dispatched to through the Confirmed channel of the ConfirmationEvent
returned upon registration.

For scripts that have confirmed in the past, the `historicalConfDetails`
method has been modified to determine whether the script has been
confirmed by locating the script in an output of a confirmed
transaction.

For scripts that have yet to confirm, a filter update is sent to the
underlying rescan to ensure that we match and dispatch on the script
when processing new blocks.

Along the way, we also address an issue where we'd miss detecting that a
transaction/script has confirmed in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always update our filters to detect the confirmation
at tip, regardless of whether a historical rescan was detected or not.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
1a41e23bf4
chainntnfs/bitcoindnotify: support registration for script confirmations
In this commit, we extend the BitcoindNotifier to support registering
scripts for confirmation notifications. Once the script has been
detected as confirmed within the chain, a confirmation notification will
be dispatched to through the Confirmed channel of the ConfirmationEvent
returned upon registration.

For scripts that have confirmed in the past, the `historicalConfDetails`
method has been modified to skip the txindex and go straight to scanning
the chain manually if confirmation request is for a script. When
scanning the chain, we'll determine whether the script has been
confirmed by locating the script in an output of a confirmed
transaction.

For scripts that have yet to confirm, they will be properly tracked
within the TxNotifier.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
1323c92947
chainntnfs/btcdnotify: support registration for script confirmations
In this commit, we extend the BtcdNotifier to support registering
scripts for confirmation notifications. Once the script has been
detected as confirmed within the chain, a confirmation notification will
be dispatched to through the Confirmed channel of the ConfirmationEvent
returned upon registration.

For scripts that have confirmed in the past, the `historicalConfDetails`
method has been modified to skip the txindex and go straight to scanning
the chain manually if confirmation request is for a script. When
scanning the chain, we'll determine whether the script has been
confirmed by locating the script in an output of a confirmed
transaction.

For scripts that have yet to confirm, they will be properly tracked
within the TxNotifier.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
72cc843e92
chainntnfs/interface: update docs to note support for scripts 2019-01-21 13:57:43 -08:00
Wilmer Paulino
c72da48111
multi: update to latest ChainNotifier impl New method 2019-01-21 13:57:43 -08:00
Wilmer Paulino
969acf6145
chainntnfs/neutrinonotify: clean up neutrinonotify.New 2019-01-21 13:57:43 -08:00
Wilmer Paulino
686e734e22
chainntnfs/bitcoindnotify: add chainParams parameter to bitcoindnotify.New
In this commit, we add the current chain parameters to the
BitcoindNotifier. This will be used in a future commit in order to
convert outputs scripts into addresses. This is needed since the
bitcoind backend uses these addresses to detect whether the script
encoded within it was spent by a transaction in the chain.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
1ee8d7518c
chainntnfs/btcdnotify: add chainParams parameter to btcdnotify.New
In this commit, we add the current chain parameters to the BtcdNotifier.
This will be used in a future commit in order to convert outputs scripts
into addresses. This is needed since the btcd backend uses these
addresses to detect whether the script encoded within it was spent by a
transaction in the chain.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
7f644019b8
chainntnfs/txnotifier_test: update tests with new registration changes 2019-01-21 13:57:43 -08:00
Wilmer Paulino
c3d5204f09
chainntnfs/txnotifier: refactor ProcessRelevantSpendTx to detect script spends
In this commit, we refactor the TxNotifier's ProcessRelevantSpendTx to
also detect script spends. This can easily be done as the transaction
filtering logic was refactored into its own method in a previous commit.
2019-01-21 13:57:43 -08:00