Commit Graph
62 Commits
Author SHA1 Message Date
Yong 582b164c46 kvdb: add timeout options for bbolt (#4787)
* mod: bump btcwallet version to accept db timeout

* btcwallet: add DBTimeOut in config

* kvdb: add database timeout option for bbolt

This commit adds a DBTimeout option in bbolt config. The relevant
functions walletdb.Open/Create are updated to use this config. In
addition, the bolt compacter also applies the new timeout option.

* channeldb: add DBTimeout in db options

This commit adds the DBTimeout option for channeldb. A new unit
test file is created to test the default options. In addition,
the params used in kvdb.Create inside channeldb_test is updated
with a DefaultDBTimeout value.

* contractcourt+routing: use DBTimeout in kvdb

This commit touches multiple test files in contractcourt and routing.
The call of function kvdb.Create and kvdb.Open are now updated with
the new param DBTimeout, using the default value kvdb.DefaultDBTimeout.

* lncfg: add DBTimeout option in db config

The DBTimeout option is added to db config. A new unit test is
added to check the default DB config is created as expected.

* migration: add DBTimeout param in kvdb.Create/kvdb.Open

* keychain: update tests to use DBTimeout param

* htlcswitch+chainreg: add DBTimeout option

* macaroons: support DBTimeout config in creation

This commit adds the DBTimeout during the creation of macaroons.db.
The usage of kvdb.Create and kvdb.Open in its tests are updated with
a timeout value using kvdb.DefaultDBTimeout.

* walletunlocker: add dbTimeout option in UnlockerService

This commit adds a new param, dbTimeout, during the creation of
UnlockerService. This param is then passed to wallet.NewLoader
inside various service calls, specifying a timeout value to be
used when opening the bbolt. In addition, the macaroonService
is also called with this dbTimeout param.

* watchtower/wtdb: add dbTimeout param during creation

This commit adds the dbTimeout param for the creation of both
watchtower.db and wtclient.db.

* multi: add db timeout param for walletdb.Create

This commit adds the db timeout param for the function call
walletdb.Create. It touches only the test files found in chainntnfs,
lnwallet, and routing.

* lnd: pass DBTimeout config to relevant services

This commit enables lnd to pass the DBTimeout config to the following
services/config/functions,
  - chainControlConfig
  - walletunlocker
  - wallet.NewLoader
  - macaroons
  - watchtower
In addition, the usage of wallet.Create is updated too.

* sample-config: add dbtimeout option
2020-12-07 15:31:49 -08:00
Oliver Gugger 2edc3cf98f multi: update to latest btcd version
With this commit we update btcd to the latest version which allows us to
specify the btcd binary we want to use when spinning up a new node.
2020-12-03 11:30:22 +01:00
Johan T. Halseth 610a93e2ff routing/chainview: check GetCFilter error
Since the filter will never be nil in non-error cases now, remove the
nil-check.
2019-10-15 10:31:25 +02:00
Wilmer Paulino 194a9dea81 multi: support sync freelist option within btcwallet 2019-10-04 12:19:40 -04:00
Joost Jager 3d7de2ad39 multi: remove dead code 2019-09-10 17:21:59 +02:00
Olaoluwa Osuntokun d86fe393cd multi: update build to latest version of neutrino+btcwallet
In this commit, we update the build to point to the latest version of
neutrino and btcwallet. The latest version of neutrino includes a number
of bug fixes, and new features like reliably transaction broadcast. The
latest version of btcwallet contains a number of bug fixes related to
properly remove invalid transactions from its database.
2019-03-13 15:03:39 -07:00
Johan T. Halseth 29acb9f1f8 routing/chainview: new neutrino API 2018-10-16 19:28:29 -07:00
Johan T. Halseth d8863bea8c routing/chainview: demote logging of new utxos for filter to Trace
It was previously Debug, which would spam the logs at startup, and not
provide very useful (human-readable) information.
2018-10-10 15:00:50 +02:00
Conner Fromknecht 59b459674d multi: init subsystem loggers via build pkg 2018-10-05 13:04:45 +09:00
Wilmer Paulino 2fa64dd6ff multi: update to new BitcoindClient API 2018-09-12 21:05:19 -07:00
Olaoluwa Osuntokun 85c0dd3de0 routing/chainview: update to latest API changes 2018-08-24 03:36:25 -07:00
Wilmer Paulino 191c4f3e28 multi: enable txindex on miner harness 2018-08-24 03:36:25 -07:00
Olaoluwa Osuntokun 518b0d2cf7 Merge pull request #1679 from cfromknecht/move-linter-v2
Makefile: move linter v2
2018-08-09 21:03:05 -07:00
Wilmer Paulino 9d2eeb6304 multi: update to latest fee estimation interface 2018-08-09 17:29:52 -07:00
Wilmer Paulino 936fcc1f16 multi: update to latest BitcoindClient interface
In this commit, we introduce a nice optimization with regards to lnd's
interaction with a bitcoind backend. Within lnd, we currently have three
different subsystems responsible for watching the chain: chainntnfs,
lnwallet, and routing/chainview. Each of these subsystems has an active
RPC and ZMQ connection to the underlying bitcoind node. This would incur
a toll on the underlying bitcoind node and would cause us to miss ZMQ
events, which are crucial to lnd. We remedy this issue by sharing the
same connection to a bitcoind node between the different clients within
lnd.
2018-08-08 20:26:30 -07:00
Conner Fromknecht a76b9a86cd routing/chainview/interface_test: use EdgePoint field names 2018-08-07 16:57:40 -07:00
Olaoluwa Osuntokun 3c60861005 routing/chainview: give neutrino backend more time to catch up for re-org test
In this commit, we modify the test to explitlcy give the neutrino
backend more time to catch up compared to the RPC backends. We do this
as a recent change has been made in the neutrino backend to wait for the
filter headers to finish syncing before proceeding with the rescan
itself. As a result, we'll need to account for this in the test and
sleep enough to give the backend a chance to catch up.
2018-07-31 21:29:03 -07:00
Olaoluwa Osuntokun fb98f59407 routing/chainview: ensure bitcoind impl meets new interface requirements 2018-07-31 21:28:55 -07:00
Olaoluwa Osuntokun 9b9029cf4b routing/chainview: ensure btcd impl meets new interface requirements 2018-07-31 21:28:55 -07:00
Olaoluwa Osuntokun 7e6997d6d3 routing/chainview: update neutrino API due to recent changes
In this commit, we ensure that the neutrino backend meets the target
interface, and also we update the API usage for the internal neutrino
rescan struct to use the new InputWithScript struct.
2018-07-31 21:28:54 -07:00
Olaoluwa Osuntokun ec0fb9c537 routing/chainview: modify the UpdateFilter method to take a channeldb.EdgePoint
In this commit, we update the existing UpdateFilter method to take the
new channeldb.EdgePoint struct in place of the prior wire.OutPoint. We
must do this as the caller now typically has this type due to the
preparation to enable lnd to be able to be compatible with the new
neutrino protocol.
2018-07-31 21:28:54 -07:00
Olaoluwa Osuntokun 505544e1b3 routing/chainview: update API usage for neutrino due to recent changes 2018-07-13 17:15:12 -07:00
Olaoluwa Osuntokun 6f60f139f4 multi: switch over import paths from roasbeef/* to btcsuite/* 2018-07-13 17:05:39 -07:00
Conner Fromknecht daa28e0d0f routing/chainview/bitcoind: improve filterMtx locking
In this commit, we modify the granularity of the locking
around the filterMtx in the bitcoind chainview, such that
we only lock once per block connected or filter update.
Currently, we acquire and release the lock for every
update to the map.

We also fix a bug that would cause us to not fully remove
all previous outpoints spent by a txn when doing manual
filter, as we previously would only remove the first output
detected.
2018-07-06 19:08:17 -07:00
Conner Fromknecht 0f6bc3538d routing/chainview/btcd: improve filterMtx locking
In this commit, we modify the granularity of the locking
around the filterMtx in the btcd chainview, such that we
only lock once per block connected or filter update.
Currently, we acquire and release the lock for every
update to the map.

We also fix a bug that would cause us to not fully remove
all previous outpoints spent by a txn when doing manual
filter, as we previously would only remove the first output
detected.
2018-07-06 19:08:17 -07:00
maurycy 3be08e69cf multi: 64bit aligment of atomic vars on arm/x86-32 2018-06-04 20:02:34 -07:00
Conner Fromknecht 1b1f5f197a routing/chainview/neutrino: cache filter entries
This commit alters the neutrino chainview such that it
caches the filter entries corresponding to watched
outpoints at the moment they are added to the filter.
Previously, we would rederive each filter entry when
reconstructing the relevant filter entries, which
would lead to unnecessary work on the gc. Now, each is
created at most once, and reused across subsequent
reconstructions.
2018-05-08 20:46:46 -07:00
Kenneth Perry f7c5a7a19e routing/chainview/neutrino: error if we get a nil cfilter back (#1176) 2018-05-03 15:21:56 -07:00
Dimitris Tsapakidis 4009f7f874 multi: fix typos in comments 2018-04-17 19:03:27 -07:00
practicalswift 663c396235 multi: fix a-vs-an typos 2018-04-17 19:02:04 -07:00
Alex 948cb09c14 routing/chainview: bitcoind back-end now requires explicit NotifyBlocks() 2018-03-14 12:16:40 -06:00
practicalswift a93736d21e multi: comprehensive typo fixes across all packages 2018-02-06 19:11:11 -08:00
Alex 187f59556a multi: add bitcoind drivers and tests 2018-01-15 13:59:34 -08:00
Matt Drollette adf0d98194 multi: fix several typos in godoc comments 2017-12-17 18:40:05 -08:00
Olaoluwa Osuntokun d2c8f01768 routing/chainview: remove possibility of deadlock in shutdown of blockEventQueue 2017-11-16 16:01:21 -08:00
Conner Fromknecht e62bb0319d routing/chainview/btcd: makes rescan logging statement print to trace 2017-11-11 15:53:42 -08:00
Olaoluwa Osuntokun 5f915280bc routing/chainview: if column wrapping on comments in neutrino.go 2017-11-06 15:55:21 -08:00
Alex 1c408adcab routing/chainview: update neutrino driver to comply with BIP 2017-11-02 20:04:07 -07:00
Johan T. Halseth 8eb994c14b routing/chainView test: add testFilterBlockDisconnected
This commit adds a test for the FilteredChainView interfaces,
making sure they notify about disconnected/connected blocks
in the correct order during a reorg.
2017-11-03 00:05:31 +01:00
Johan T. Halseth 6d15be5b79 routing/chainview: use blockEventQueue for neutrino block events
This commit makes use of the blockEventQueue within the neutrino
implementation of FilteredChainView to ensure connected and
disconnected blocks are consumed in order by the reader.

It also specifies that neutrino is not to send disconnected blocks
notifications during rescans, making it consistent with the btcd
implementation.
2017-11-03 00:05:30 +01:00
Johan T. Halseth 082f012fcf routing/chainview: implement staleBlocks for btcd view.
This commit moves btcd view away from using the deprecated
callbacks onBlockConnected/Disconnected, and instead use
onFilteredBlockConnected/disconnected.

This commit also implements the sending of disconnected blocks
over the staleBlocks channel. To send these blocks, the
blockEventQueue is used to ensure the ordering of blocks are
correctly kept.

It also changes the way filter updates are handled. Since we
now load the tx filter to the rpc server itself, we can call
RescanBlocks instead of manually filtering blocks. These
rescanned blocks are also added to the blockEventQueue,
ensuring the ordering is kept.
2017-11-03 00:05:30 +01:00
Johan T. Halseth e9c2f703d9 routing/chainview: additional comments to FilteredChainView godoc 2017-11-03 00:05:29 +01:00
Johan T. Halseth a219aca0d3 routing/chainView: add blockEventQueue data structure
blockEventQueue is an ordered queue for block events sent from a
FilteredChainView. The two types of possible block events are
connected/new blocks, and disconencted/stale blocks. The
blockEventQueue keeps the order of these events intact, while
still being non-blocking. This is important in order for the
chainView's call to onFilteredBlockConnected/Disconnected to not
get blocked, and for the consumer of the block events to always
get the events in the correct order.
2017-11-03 00:05:28 +01:00
Jim Posen d8a2ed27b8 routing/chainview: Fix data race in block disconnected callback. 2017-09-29 13:53:02 -07:00
Olaoluwa Osuntokun e5f3ee0fb6 chainntnfs+routing/chainview: reduce neutrino.WaitForMoreCFHeaders value
This commit reduces the neutrino.WaitForMoreCFHeaders parameter when
instantiating a neutrino instance as a lower value will allow the tests
to complete more quickly.
2017-09-13 16:46:11 +02:00
Olaoluwa Osuntokun 9f0efddc20 multi: switch from btcrpcclient to rpcclient 2017-08-24 18:54:24 -07:00
Olaoluwa Osuntokun 9daa659bb3 routing/chainview: convert chainview integration tests to use sub-tests 2017-07-04 15:53:58 -07:00
Andrey Samokhvalov 8fa2b95c12 lnd: remove seelog logger
The btclog package has been changed to defining its own logging
interface (rather than seelog's) and provides a default implementation
for callers to use.

There are two primary advantages to the new logger implementation.

First, all log messages are created before the call returns.  Compared
to seelog, this prevents data races when mutable variables are logged.

Second, the new logger does not implement any kind of artifical rate
limiting (what seelog refers to as "adaptive logging").  Log messages
are outputted as soon as possible and the application will appear to
perform much better when watching standard output.

Because log rotation is not a feature of the btclog logging
implementation, it is handled by the main package by importing a file
rotation package that provides an io.Reader interface for creating
output to a rotating file output.  The rotator has been configured
with the same defaults that btcd previously used in the seelog config
(10MB file limits with maximum of 3 rolls) but now compresses newly
created roll files.  Due to the high compressibility of log text, the
compressed files typically reduce to around 15-30% of the original
10MB file.
2017-06-25 14:19:56 +01:00
Olaoluwa Osuntokun e4563ca13b routing/chainview: make filter updates synchronous for neutrino
This commit fixes a possible race condition wherein a call to
FilterBlock after a call to UpdateFilter would result in the call to
FilterBlock not yet using the updated filter. We fix this by ensuring
the internal chain filter is updated by the time the call to
FilterBlock returns.
2017-06-09 12:18:33 -07:00
Olaoluwa Osuntokun 0c134a8cb3 routing/chainview: if the chain filter is empty, don't fetch the cfilter 2017-06-08 23:11:24 -07:00