Commit Graph

62 Commits

Author SHA1 Message Date
Yong
582b164c46
kvdb: add timeout options for bbolt (#4787)
* mod: bump btcwallet version to accept db timeout

* btcwallet: add DBTimeOut in config

* kvdb: add database timeout option for bbolt

This commit adds a DBTimeout option in bbolt config. The relevant
functions walletdb.Open/Create are updated to use this config. In
addition, the bolt compacter also applies the new timeout option.

* channeldb: add DBTimeout in db options

This commit adds the DBTimeout option for channeldb. A new unit
test file is created to test the default options. In addition,
the params used in kvdb.Create inside channeldb_test is updated
with a DefaultDBTimeout value.

* contractcourt+routing: use DBTimeout in kvdb

This commit touches multiple test files in contractcourt and routing.
The call of function kvdb.Create and kvdb.Open are now updated with
the new param DBTimeout, using the default value kvdb.DefaultDBTimeout.

* lncfg: add DBTimeout option in db config

The DBTimeout option is added to db config. A new unit test is
added to check the default DB config is created as expected.

* migration: add DBTimeout param in kvdb.Create/kvdb.Open

* keychain: update tests to use DBTimeout param

* htlcswitch+chainreg: add DBTimeout option

* macaroons: support DBTimeout config in creation

This commit adds the DBTimeout during the creation of macaroons.db.
The usage of kvdb.Create and kvdb.Open in its tests are updated with
a timeout value using kvdb.DefaultDBTimeout.

* walletunlocker: add dbTimeout option in UnlockerService

This commit adds a new param, dbTimeout, during the creation of
UnlockerService. This param is then passed to wallet.NewLoader
inside various service calls, specifying a timeout value to be
used when opening the bbolt. In addition, the macaroonService
is also called with this dbTimeout param.

* watchtower/wtdb: add dbTimeout param during creation

This commit adds the dbTimeout param for the creation of both
watchtower.db and wtclient.db.

* multi: add db timeout param for walletdb.Create

This commit adds the db timeout param for the function call
walletdb.Create. It touches only the test files found in chainntnfs,
lnwallet, and routing.

* lnd: pass DBTimeout config to relevant services

This commit enables lnd to pass the DBTimeout config to the following
services/config/functions,
  - chainControlConfig
  - walletunlocker
  - wallet.NewLoader
  - macaroons
  - watchtower
In addition, the usage of wallet.Create is updated too.

* sample-config: add dbtimeout option
2020-12-07 15:31:49 -08:00
Oliver Gugger
2edc3cf98f
multi: update to latest btcd version
With this commit we update btcd to the latest version which allows us to
specify the btcd binary we want to use when spinning up a new node.
2020-12-03 11:30:22 +01:00
Johan T. Halseth
610a93e2ff
routing/chainview: check GetCFilter error
Since the filter will never be nil in non-error cases now, remove the
nil-check.
2019-10-15 10:31:25 +02:00
Wilmer Paulino
194a9dea81
multi: support sync freelist option within btcwallet 2019-10-04 12:19:40 -04:00
Joost Jager
3d7de2ad39
multi: remove dead code 2019-09-10 17:21:59 +02:00
Olaoluwa Osuntokun
d86fe393cd
multi: update build to latest version of neutrino+btcwallet
In this commit, we update the build to point to the latest version of
neutrino and btcwallet. The latest version of neutrino includes a number
of bug fixes, and new features like reliably transaction broadcast. The
latest version of btcwallet contains a number of bug fixes related to
properly remove invalid transactions from its database.
2019-03-13 15:03:39 -07:00
Johan T. Halseth
29acb9f1f8
routing/chainview: new neutrino API 2018-10-16 19:28:29 -07:00
Johan T. Halseth
d8863bea8c
routing/chainview: demote logging of new utxos for filter to Trace
It was previously Debug, which would spam the logs at startup, and not
provide very useful (human-readable) information.
2018-10-10 15:00:50 +02:00
Conner Fromknecht
59b459674d
multi: init subsystem loggers via build pkg 2018-10-05 13:04:45 +09:00
Wilmer Paulino
2fa64dd6ff multi: update to new BitcoindClient API 2018-09-12 21:05:19 -07:00
Olaoluwa Osuntokun
85c0dd3de0
routing/chainview: update to latest API changes 2018-08-24 03:36:25 -07:00
Wilmer Paulino
191c4f3e28
multi: enable txindex on miner harness 2018-08-24 03:36:25 -07:00
Olaoluwa Osuntokun
518b0d2cf7
Merge pull request #1679 from cfromknecht/move-linter-v2
Makefile: move linter v2
2018-08-09 21:03:05 -07:00
Wilmer Paulino
9d2eeb6304
multi: update to latest fee estimation interface 2018-08-09 17:29:52 -07:00
Wilmer Paulino
936fcc1f16
multi: update to latest BitcoindClient interface
In this commit, we introduce a nice optimization with regards to lnd's
interaction with a bitcoind backend. Within lnd, we currently have three
different subsystems responsible for watching the chain: chainntnfs,
lnwallet, and routing/chainview. Each of these subsystems has an active
RPC and ZMQ connection to the underlying bitcoind node. This would incur
a toll on the underlying bitcoind node and would cause us to miss ZMQ
events, which are crucial to lnd. We remedy this issue by sharing the
same connection to a bitcoind node between the different clients within
lnd.
2018-08-08 20:26:30 -07:00
Conner Fromknecht
a76b9a86cd
routing/chainview/interface_test: use EdgePoint field names 2018-08-07 16:57:40 -07:00
Olaoluwa Osuntokun
3c60861005
routing/chainview: give neutrino backend more time to catch up for re-org test
In this commit, we modify the test to explitlcy give the neutrino
backend more time to catch up compared to the RPC backends. We do this
as a recent change has been made in the neutrino backend to wait for the
filter headers to finish syncing before proceeding with the rescan
itself. As a result, we'll need to account for this in the test and
sleep enough to give the backend a chance to catch up.
2018-07-31 21:29:03 -07:00
Olaoluwa Osuntokun
fb98f59407
routing/chainview: ensure bitcoind impl meets new interface requirements 2018-07-31 21:28:55 -07:00
Olaoluwa Osuntokun
9b9029cf4b
routing/chainview: ensure btcd impl meets new interface requirements 2018-07-31 21:28:55 -07:00
Olaoluwa Osuntokun
7e6997d6d3
routing/chainview: update neutrino API due to recent changes
In this commit, we ensure that the neutrino backend meets the target
interface, and also we update the API usage for the internal neutrino
rescan struct to use the new InputWithScript struct.
2018-07-31 21:28:54 -07:00
Olaoluwa Osuntokun
ec0fb9c537
routing/chainview: modify the UpdateFilter method to take a channeldb.EdgePoint
In this commit, we update the existing UpdateFilter method to take the
new channeldb.EdgePoint struct in place of the prior wire.OutPoint. We
must do this as the caller now typically has this type due to the
preparation to enable lnd to be able to be compatible with the new
neutrino protocol.
2018-07-31 21:28:54 -07:00
Olaoluwa Osuntokun
505544e1b3
routing/chainview: update API usage for neutrino due to recent changes 2018-07-13 17:15:12 -07:00
Olaoluwa Osuntokun
6f60f139f4 multi: switch over import paths from roasbeef/* to btcsuite/* 2018-07-13 17:05:39 -07:00
Conner Fromknecht
daa28e0d0f
routing/chainview/bitcoind: improve filterMtx locking
In this commit, we modify the granularity of the locking
around the filterMtx in the bitcoind chainview, such that
we only lock once per block connected or filter update.
Currently, we acquire and release the lock for every
update to the map.

We also fix a bug that would cause us to not fully remove
all previous outpoints spent by a txn when doing manual
filter, as we previously would only remove the first output
detected.
2018-07-06 19:08:17 -07:00
Conner Fromknecht
0f6bc3538d
routing/chainview/btcd: improve filterMtx locking
In this commit, we modify the granularity of the locking
around the filterMtx in the btcd chainview, such that we
only lock once per block connected or filter update.
Currently, we acquire and release the lock for every
update to the map.

We also fix a bug that would cause us to not fully remove
all previous outpoints spent by a txn when doing manual
filter, as we previously would only remove the first output
detected.
2018-07-06 19:08:17 -07:00
maurycy
3be08e69cf multi: 64bit aligment of atomic vars on arm/x86-32 2018-06-04 20:02:34 -07:00
Conner Fromknecht
1b1f5f197a
routing/chainview/neutrino: cache filter entries
This commit alters the neutrino chainview such that it
caches the filter entries corresponding to watched
outpoints at the moment they are added to the filter.
Previously, we would rederive each filter entry when
reconstructing the relevant filter entries, which
would lead to unnecessary work on the gc. Now, each is
created at most once, and reused across subsequent
reconstructions.
2018-05-08 20:46:46 -07:00
Kenneth Perry
f7c5a7a19e routing/chainview/neutrino: error if we get a nil cfilter back (#1176) 2018-05-03 15:21:56 -07:00
Dimitris Tsapakidis
4009f7f874 multi: fix typos in comments 2018-04-17 19:03:27 -07:00
practicalswift
663c396235 multi: fix a-vs-an typos 2018-04-17 19:02:04 -07:00
Alex
948cb09c14 routing/chainview: bitcoind back-end now requires explicit NotifyBlocks() 2018-03-14 12:16:40 -06:00
practicalswift
a93736d21e multi: comprehensive typo fixes across all packages 2018-02-06 19:11:11 -08:00
Alex
187f59556a multi: add bitcoind drivers and tests 2018-01-15 13:59:34 -08:00
Matt Drollette
adf0d98194 multi: fix several typos in godoc comments 2017-12-17 18:40:05 -08:00
Olaoluwa Osuntokun
d2c8f01768
routing/chainview: remove possibility of deadlock in shutdown of blockEventQueue 2017-11-16 16:01:21 -08:00
Conner Fromknecht
e62bb0319d
routing/chainview/btcd: makes rescan logging statement print to trace 2017-11-11 15:53:42 -08:00
Olaoluwa Osuntokun
5f915280bc
routing/chainview: if column wrapping on comments in neutrino.go 2017-11-06 15:55:21 -08:00
Alex
1c408adcab routing/chainview: update neutrino driver to comply with BIP 2017-11-02 20:04:07 -07:00
Johan T. Halseth
8eb994c14b
routing/chainView test: add testFilterBlockDisconnected
This commit adds a test for the FilteredChainView interfaces,
making sure they notify about disconnected/connected blocks
in the correct order during a reorg.
2017-11-03 00:05:31 +01:00
Johan T. Halseth
6d15be5b79
routing/chainview: use blockEventQueue for neutrino block events
This commit makes use of the blockEventQueue within the neutrino
implementation of FilteredChainView to ensure connected and
disconnected blocks are consumed in order by the reader.

It also specifies that neutrino is not to send disconnected blocks
notifications during rescans, making it consistent with the btcd
implementation.
2017-11-03 00:05:30 +01:00
Johan T. Halseth
082f012fcf
routing/chainview: implement staleBlocks for btcd view.
This commit moves btcd view away from using the deprecated
callbacks onBlockConnected/Disconnected, and instead use
onFilteredBlockConnected/disconnected.

This commit also implements the sending of disconnected blocks
over the staleBlocks channel. To send these blocks, the
blockEventQueue is used to ensure the ordering of blocks are
correctly kept.

It also changes the way filter updates are handled. Since we
now load the tx filter to the rpc server itself, we can call
RescanBlocks instead of manually filtering blocks. These
rescanned blocks are also added to the blockEventQueue,
ensuring the ordering is kept.
2017-11-03 00:05:30 +01:00
Johan T. Halseth
e9c2f703d9
routing/chainview: additional comments to FilteredChainView godoc 2017-11-03 00:05:29 +01:00
Johan T. Halseth
a219aca0d3
routing/chainView: add blockEventQueue data structure
blockEventQueue is an ordered queue for block events sent from a
FilteredChainView. The two types of possible block events are
connected/new blocks, and disconencted/stale blocks. The
blockEventQueue keeps the order of these events intact, while
still being non-blocking. This is important in order for the
chainView's call to onFilteredBlockConnected/Disconnected to not
get blocked, and for the consumer of the block events to always
get the events in the correct order.
2017-11-03 00:05:28 +01:00
Jim Posen
d8a2ed27b8 routing/chainview: Fix data race in block disconnected callback. 2017-09-29 13:53:02 -07:00
Olaoluwa Osuntokun
e5f3ee0fb6
chainntnfs+routing/chainview: reduce neutrino.WaitForMoreCFHeaders value
This commit reduces the neutrino.WaitForMoreCFHeaders parameter when
instantiating a neutrino instance as a lower value will allow the tests
to complete more quickly.
2017-09-13 16:46:11 +02:00
Olaoluwa Osuntokun
9f0efddc20
multi: switch from btcrpcclient to rpcclient 2017-08-24 18:54:24 -07:00
Olaoluwa Osuntokun
9daa659bb3
routing/chainview: convert chainview integration tests to use sub-tests 2017-07-04 15:53:58 -07:00
Andrey Samokhvalov
8fa2b95c12 lnd: remove seelog logger
The btclog package has been changed to defining its own logging
interface (rather than seelog's) and provides a default implementation
for callers to use.

There are two primary advantages to the new logger implementation.

First, all log messages are created before the call returns.  Compared
to seelog, this prevents data races when mutable variables are logged.

Second, the new logger does not implement any kind of artifical rate
limiting (what seelog refers to as "adaptive logging").  Log messages
are outputted as soon as possible and the application will appear to
perform much better when watching standard output.

Because log rotation is not a feature of the btclog logging
implementation, it is handled by the main package by importing a file
rotation package that provides an io.Reader interface for creating
output to a rotating file output.  The rotator has been configured
with the same defaults that btcd previously used in the seelog config
(10MB file limits with maximum of 3 rolls) but now compresses newly
created roll files.  Due to the high compressibility of log text, the
compressed files typically reduce to around 15-30% of the original
10MB file.
2017-06-25 14:19:56 +01:00
Olaoluwa Osuntokun
e4563ca13b
routing/chainview: make filter updates synchronous for neutrino
This commit fixes a possible race condition wherein a call to
FilterBlock after a call to UpdateFilter would result in the call to
FilterBlock not yet using the updated filter. We fix this by ensuring
the internal chain filter is updated by the time the call to
FilterBlock returns.
2017-06-09 12:18:33 -07:00
Olaoluwa Osuntokun
0c134a8cb3
routing/chainview: if the chain filter is empty, don't fetch the cfilter 2017-06-08 23:11:24 -07:00