Commit Graph

6785 Commits

Author SHA1 Message Date
Francisco Calderón
4c1bd14c94
Remove equal sign on verifychanbackup cmd on multi_file argument description 2019-04-04 19:06:23 -03:00
Olaoluwa Osuntokun
3a19afe46d
Merge pull request #2882 from wpaulino/sync-manager-stale-syncer
discovery: only replace stale active syncer if disconnected
2019-04-03 20:47:12 -07:00
Wilmer Paulino
00338c5ec2
discovery: properly handle SyncManager shutdown signal 2019-04-03 19:32:56 -07:00
Wilmer Paulino
46ceaf8cf6
discovery: only replace stale active syncer if disconnected
In this commit, we address a bug where we'd attempt to replace the
stale active syncer when it transitioned to a passive syncer. This
replacement logic is only intended to happen when the active syncer
disconnects, as rotateActiveSyncerCandidate chooses and queues its own
replacement.
2019-04-03 16:43:31 -07:00
Olaoluwa Osuntokun
2cc6687ff3
build: bump version to 0.6-beta 2019-04-03 15:48:17 -07:00
Olaoluwa Osuntokun
30f2b1ca01
Merge pull request #2740 from wpaulino/gossip-sync-manager
discovery: introduce gossiper syncManager subsystem
2019-04-03 15:46:12 -07:00
Wilmer Paulino
ca01695330
rpc: expose peer's GossipSyncer sync type 2019-04-03 15:44:47 -07:00
Wilmer Paulino
8b6a9bb5d3
discovery: make timestamp range check inclusive within FilterGossipMsgs
As required by the spec:

> SHOULD send all gossip messages whose timestamp is greater or equal to
first_timestamp, and less than first_timestamp plus timestamp_range.
2019-04-03 15:44:46 -07:00
Wilmer Paulino
70be812747
discovery+server: use new gossiper's SyncManager subsystem 2019-04-03 15:44:43 -07:00
Wilmer Paulino
80b84eef9c
config+peer: replace NoChanUpdates flag with NumGraphSyncPeers
In this commit, we replace the NoChanUpdates flag with a flag that
allows us to specify the number of peers we want to actively receive new
graph updates from. This will be required when integrating the new
gossiper SyncManager subsystem with the rest of lnd.
2019-04-03 15:43:51 -07:00
Wilmer Paulino
a188657b2f
discovery: introduce gossiper SyncManager subsystem
In this commit, we introduce a new subsystem for the gossiper: the
SyncManager. This subsystem is a major overhaul on the way the daemon
performs the graph query sync state machine with peers.

Along with this subsystem, we also introduce the concept of an active
syncer. An active syncer is simply a GossipSyncer currently operating
under an ActiveSync sync type. Before this commit, all GossipSyncer's
would act as active syncers, which means that we were receiving new
graph updates from all of them. This isn't necessary, as it greatly
increases bandwidth usage as the network grows. The SyncManager changes
this by requiring a specific number of active syncers. Once we reach
this specified number, any future peers will have a GossipSyncer with a
PassiveSync sync type.

It is responsible for three main things:

1. Choosing different peers randomly to receive graph updates from to
ensure we don't only receive them from the same set of peers.

2. Choosing different peers to force a historical sync with to ensure we
have as much of the public network as possible. The first syncer
registered with the manager will also attempt a historical sync.

3. Managing an in-order queue of active syncers where the next cannot be
started until the current one has completed its state machine to ensure
they don't overlap and request the same set of channels, which
significantly reduces bandwidth usage and addresses a number of issues.
2019-04-03 15:08:32 -07:00
Wilmer Paulino
e075817e44
discovery: introduce GossipSyncer signal delivery of chansSynced state
In this commit, we introduce another feature to the GossipSyncer in
which it can deliver a signal to an external caller once it reaches its
terminal chansSynced state. This is yet to be used, but will serve
useful with a round-robin sync mechanism, where we wait for to finish
syncing with a specific peer before moving on to the next.
2019-04-03 15:08:31 -07:00
Wilmer Paulino
042241dc48
discovery: allow gossip syncer to perform historical syncs
In this commit, we introduce the ability for gossip syncers to perform
historical syncs. This allows us to reconcile any channels we're missing
that the remote peer has starting from the genesis block of the chain.
This commit serves as a prerequisite to the SyncManager, introduced in a
later commit, where we'll be able to make spot checks by performing
historical syncs with peers to ensure we have as much of the graph as
possible.
2019-04-03 15:08:30 -07:00
Wilmer Paulino
ca4fbd598c
discovery: introduce GossipSyncer sync transitions
In this commit, we introduce the ability for GossipSyncer's to
transition their sync type. This allows us to be more flexible with our
gossip syncers, as we can now prevent them from receiving new graph
updates at any time. It's now possible to transition between the
different sync types, as long as the GossipSyncer has reached its
terminal chansSynced sync state. Certain transitions require some
additional wire messages to be sent, like in the case of an ActiveSync
GossipSyncer transitioning to a PassiveSync type.
2019-04-03 15:08:29 -07:00
Wilmer Paulino
acc42c1b68
discovery: set GossipSyncer update horizon to current time
With the introduction of the gossip sync manager in a later commit,
retrieving the backlog of updates within the last hour is no longer
necessary as we'll be forcing full syncs periodically.
2019-04-03 15:08:28 -07:00
Wilmer Paulino
8d7c0a9899
discovery: replace GossipSyncer syncChanUpdates flag with SyncerType
In this commit, we introduce a new type: SyncerType. This type denotes
the type of sync a GossipSyncer is currently under. We only introduce
the two possible entry states, ActiveSync and PassiveSync. An ActiveSync
GossipSyncer will exchange channels with the remote peer and receive new
graph updates from them, while a PassiveSync GossipSyncer will not and
will only response to the remote peer's queries.

This commit does not modify the behavior and is only meant to be a
refactor.
2019-04-03 15:08:27 -07:00
Wilmer Paulino
7e92b9a4e2
discovery: export gossipSyncer 2019-04-03 15:08:26 -07:00
Wilmer Paulino
d954cfc4ba
discovery: include peerPub in gossipSyncerCfg 2019-04-03 15:08:25 -07:00
Wilmer Paulino
c72db902f0
discovery: replace waitPredicate with lntest version 2019-04-03 15:08:12 -07:00
Conner Fromknecht
b96da69476
Merge pull request #2880 from Roasbeef/windows-signals
signal: don't attempt to catch SIGSTOP
2019-04-03 14:36:10 -07:00
Olaoluwa Osuntokun
2a09db7c76
Merge pull request #2876 from Roasbeef/scb-follow-ups
lnrpc: add REST endpoints for new SCB related calls
2019-04-03 14:30:48 -07:00
Olaoluwa Osuntokun
0ca65208c5
signal: don't attempt to catch SIGSTOP
In this commit, we modify the set of default signals we attempt to catch in
order to execute a graceful shutdown. Before this commit, we would attempt
to catch/register for `SIGSTOP`. There're two issues with this

  1. `SIGSTOP` is meant to be used in combination with `SIGCONCT` to allow a
  process to be paused/resumed. Therefore, our action of shutting down once
  received was incorrect.

  2. `SIGSTOP` doesn't exist on windows, so users aren't able to compile on
  this platform without modifying the codebase.
2019-04-03 13:13:03 -07:00
Olaoluwa Osuntokun
fa73a60167
Merge pull request #2868 from cfromknecht/commit-bitcoind-spendhints
chainntnfs: Always commit bitcoind spendhints
2019-04-02 21:13:30 -07:00
Olaoluwa Osuntokun
6f2087023b
lnrpc: add REST endpoints for new SCB related calls 2019-04-02 20:46:33 -07:00
Olaoluwa Osuntokun
1dc1e8510a
Merge pull request #2847 from cfromknecht/reject-and-channel-cache
channeldb: add reject and channel caches
2019-04-02 18:23:52 -07:00
Johan T. Halseth
38e4f03cbd
Merge pull request #2870 from halseth/stop-chansubswapper
[trivial] server: stop chansubswapper on shutdown instead of start
2019-04-02 10:58:10 +02:00
Johan T. Halseth
96f4cd1df9
server: stop chansubswapper on shutdown instead of start 2019-04-02 09:58:13 +02:00
Conner Fromknecht
90ffc4a0ab
chainntnfs/bitcoindnotify: improve historical conf dispatch logging 2019-04-01 18:21:07 -07:00
Conner Fromknecht
7a08825b1e
chainntnfs/bitcoindnotify: always call UpdateSpendDetails
This commit fixes a bug that would cause the
notifier not to commit spend hints for items
that are not found. This is done by calling
UpdateSpendDetails with a nil detail, permitting
the notifier to begin updating the spend hints
with new blocks that arrive at tip. The change
is designed to mimic the behavior for historical
confirmation dispatch.

The symptom of this bug is needing to do many
long rescans on startup, even if new blocks
arrive after the rescan had completed. With
this change, nodes will have to do the scans
once more before their hints will be properly
updated. Restarts from then on should not
have this behavior.
2019-04-01 18:20:52 -07:00
Olaoluwa Osuntokun
893cff60c4
Merge pull request #2839 from joostjager/htlc-expiry-check
link: increase expiry grace delta for exit hop and forwarded htlcs
2019-04-01 18:15:56 -07:00
Conner Fromknecht
5d98a94d60
channeldb: write chan updates through reject+channel cache 2019-04-01 16:34:51 -07:00
Conner Fromknecht
63b15fd8fb
lnd: pass CLI reject+channel cache sizes to channeldb Open 2019-04-01 16:34:30 -07:00
Conner Fromknecht
433665932c
config: expose Caches subconfig 2019-04-01 16:34:17 -07:00
Conner Fromknecht
7504d46e84
lncfg/caches: adds Caches sub config 2019-04-01 16:34:04 -07:00
Conner Fromknecht
5c8ba59790
lncfg/interface: define Validator iface + variadic Validate
This commit introduces the Validator interface, which
is intended to be implemented by any sub configs. It
specifies a Validate() error method that should fail
if a sub configuration contains any invalid or insane
parameters.

In addition, a package-level Validate method can be
used to check a variadic number of sub configs
implementing the Validator interface. This allows the
primary config struct to be extended via targeted
and/or specialized sub configs, and validate all of
them in sequence without bloating the main package
with the actual validation logic.
2019-04-01 16:33:50 -07:00
Conner Fromknecht
6d3e081f7b
channeldb: accept cache sizes in ChannelGraph 2019-04-01 16:33:36 -07:00
Conner Fromknecht
baa968b1ff
channeldb/options: add Options w/ functional modifiers 2019-04-01 16:33:13 -07:00
Conner Fromknecht
ae3a00a5da
channeldb/graph+db: integrate reject and channel caches 2019-04-01 16:32:52 -07:00
Conner Fromknecht
b20a254faa
channeldb/channel_cache: add channelCache w/ randomized eviction 2019-04-01 16:25:31 -07:00
Conner Fromknecht
af0ea3590b
channeldb/reject_cache: add rejectCache w/ randomized eviction 2019-04-01 16:25:31 -07:00
Conner Fromknecht
3c46ceec1d
channeldb/db: init one ChannelGraph per channeldb.DB 2019-04-01 16:25:30 -07:00
Conner Fromknecht
ecbb78651a
channeldb/graph: add newChannelGraph constructor 2019-04-01 16:25:30 -07:00
Conner Fromknecht
ac315fd051
channeldb/db: remove unused buffer pool 2019-04-01 16:25:30 -07:00
Conner Fromknecht
2f927493c8
Revert "channeldb: convert invoice settle/cancel calls to use Batch"
This reverts commit da76c34418a40761fdc99e11200755661699f7eb.
2019-04-01 16:25:30 -07:00
Conner Fromknecht
e35f676b4c
Revert "channeldb: convert concurrent channel state machine calls to use Batch"
This reverts commit e8da6dd0b48843a48cecdcff894f48e8020d940e.
2019-04-01 16:25:30 -07:00
Conner Fromknecht
f6d93c8e5d
Revert "channeldb: convert all Update calls to use Batch"
This reverts commit 3555d3dbd431fe3ddf2660d4dc270b5a595fe517.
2019-04-01 16:25:26 -07:00
Olaoluwa Osuntokun
c37ea68ba6
Merge pull request #2313 from Roasbeef/static-chan-backups
multi: implement new safe static channel backup and recovery scheme, RPCs, and cli commands
2019-04-01 16:22:37 -07:00
Johan T. Halseth
8087ea4c4c
Merge pull request #2775 from wpaulino/reliable-sender-chan-update
discovery: check if stale within isMsgStale for ChannelUpdate messages
2019-04-01 12:08:33 +02:00
Olaoluwa Osuntokun
f2160273b7
test: add new series of itests for various SCB restore scenarios
In this commit, we add 4 new itests for exercising the SCB restore
process via 4 primary scenarios: recover from backup using RPC, recover
from file using RPC, recover channels during init/creation, recover
channels during unlock. With all fields populated there're a total of 24
new scenarios to cover. At the time of authoring of this commit, the
other scenarios (bits are: initiator, updates, private) have been left
out for now, as they increased the run time of the integration tests
significantly.
2019-03-28 17:54:15 -07:00
Olaoluwa Osuntokun
e3029dee45
test: add new itest for exporting SCBs 2019-03-28 17:54:13 -07:00