Commit Graph

6927 Commits

Author SHA1 Message Date
Wilmer Paulino
042241dc48
discovery: allow gossip syncer to perform historical syncs
In this commit, we introduce the ability for gossip syncers to perform
historical syncs. This allows us to reconcile any channels we're missing
that the remote peer has starting from the genesis block of the chain.
This commit serves as a prerequisite to the SyncManager, introduced in a
later commit, where we'll be able to make spot checks by performing
historical syncs with peers to ensure we have as much of the graph as
possible.
2019-04-03 15:08:30 -07:00
Wilmer Paulino
ca4fbd598c
discovery: introduce GossipSyncer sync transitions
In this commit, we introduce the ability for GossipSyncer's to
transition their sync type. This allows us to be more flexible with our
gossip syncers, as we can now prevent them from receiving new graph
updates at any time. It's now possible to transition between the
different sync types, as long as the GossipSyncer has reached its
terminal chansSynced sync state. Certain transitions require some
additional wire messages to be sent, like in the case of an ActiveSync
GossipSyncer transitioning to a PassiveSync type.
2019-04-03 15:08:29 -07:00
Wilmer Paulino
acc42c1b68
discovery: set GossipSyncer update horizon to current time
With the introduction of the gossip sync manager in a later commit,
retrieving the backlog of updates within the last hour is no longer
necessary as we'll be forcing full syncs periodically.
2019-04-03 15:08:28 -07:00
Wilmer Paulino
8d7c0a9899
discovery: replace GossipSyncer syncChanUpdates flag with SyncerType
In this commit, we introduce a new type: SyncerType. This type denotes
the type of sync a GossipSyncer is currently under. We only introduce
the two possible entry states, ActiveSync and PassiveSync. An ActiveSync
GossipSyncer will exchange channels with the remote peer and receive new
graph updates from them, while a PassiveSync GossipSyncer will not and
will only response to the remote peer's queries.

This commit does not modify the behavior and is only meant to be a
refactor.
2019-04-03 15:08:27 -07:00
Wilmer Paulino
7e92b9a4e2
discovery: export gossipSyncer 2019-04-03 15:08:26 -07:00
Wilmer Paulino
d954cfc4ba
discovery: include peerPub in gossipSyncerCfg 2019-04-03 15:08:25 -07:00
Wilmer Paulino
c72db902f0
discovery: replace waitPredicate with lntest version 2019-04-03 15:08:12 -07:00
Conner Fromknecht
b96da69476
Merge pull request #2880 from Roasbeef/windows-signals
signal: don't attempt to catch SIGSTOP
2019-04-03 14:36:10 -07:00
Olaoluwa Osuntokun
2a09db7c76
Merge pull request #2876 from Roasbeef/scb-follow-ups
lnrpc: add REST endpoints for new SCB related calls
2019-04-03 14:30:48 -07:00
Olaoluwa Osuntokun
0ca65208c5
signal: don't attempt to catch SIGSTOP
In this commit, we modify the set of default signals we attempt to catch in
order to execute a graceful shutdown. Before this commit, we would attempt
to catch/register for `SIGSTOP`. There're two issues with this

  1. `SIGSTOP` is meant to be used in combination with `SIGCONCT` to allow a
  process to be paused/resumed. Therefore, our action of shutting down once
  received was incorrect.

  2. `SIGSTOP` doesn't exist on windows, so users aren't able to compile on
  this platform without modifying the codebase.
2019-04-03 13:13:03 -07:00
Olaoluwa Osuntokun
fa73a60167
Merge pull request #2868 from cfromknecht/commit-bitcoind-spendhints
chainntnfs: Always commit bitcoind spendhints
2019-04-02 21:13:30 -07:00
Olaoluwa Osuntokun
6f2087023b
lnrpc: add REST endpoints for new SCB related calls 2019-04-02 20:46:33 -07:00
Olaoluwa Osuntokun
1dc1e8510a
Merge pull request #2847 from cfromknecht/reject-and-channel-cache
channeldb: add reject and channel caches
2019-04-02 18:23:52 -07:00
Johan T. Halseth
38e4f03cbd
Merge pull request #2870 from halseth/stop-chansubswapper
[trivial] server: stop chansubswapper on shutdown instead of start
2019-04-02 10:58:10 +02:00
Johan T. Halseth
c2af637d3c
routing/notifications: check router startd on SubscribeTopology
If the router is not yet started, return an error to avoid a deadlock
waiting for it to handle the subscription request.
2019-04-02 10:21:08 +02:00
Johan T. Halseth
96f4cd1df9
server: stop chansubswapper on shutdown instead of start 2019-04-02 09:58:13 +02:00
Conner Fromknecht
90ffc4a0ab
chainntnfs/bitcoindnotify: improve historical conf dispatch logging 2019-04-01 18:21:07 -07:00
Conner Fromknecht
7a08825b1e
chainntnfs/bitcoindnotify: always call UpdateSpendDetails
This commit fixes a bug that would cause the
notifier not to commit spend hints for items
that are not found. This is done by calling
UpdateSpendDetails with a nil detail, permitting
the notifier to begin updating the spend hints
with new blocks that arrive at tip. The change
is designed to mimic the behavior for historical
confirmation dispatch.

The symptom of this bug is needing to do many
long rescans on startup, even if new blocks
arrive after the rescan had completed. With
this change, nodes will have to do the scans
once more before their hints will be properly
updated. Restarts from then on should not
have this behavior.
2019-04-01 18:20:52 -07:00
Olaoluwa Osuntokun
893cff60c4
Merge pull request #2839 from joostjager/htlc-expiry-check
link: increase expiry grace delta for exit hop and forwarded htlcs
2019-04-01 18:15:56 -07:00
Conner Fromknecht
5d98a94d60
channeldb: write chan updates through reject+channel cache 2019-04-01 16:34:51 -07:00
Conner Fromknecht
63b15fd8fb
lnd: pass CLI reject+channel cache sizes to channeldb Open 2019-04-01 16:34:30 -07:00
Conner Fromknecht
433665932c
config: expose Caches subconfig 2019-04-01 16:34:17 -07:00
Conner Fromknecht
7504d46e84
lncfg/caches: adds Caches sub config 2019-04-01 16:34:04 -07:00
Conner Fromknecht
5c8ba59790
lncfg/interface: define Validator iface + variadic Validate
This commit introduces the Validator interface, which
is intended to be implemented by any sub configs. It
specifies a Validate() error method that should fail
if a sub configuration contains any invalid or insane
parameters.

In addition, a package-level Validate method can be
used to check a variadic number of sub configs
implementing the Validator interface. This allows the
primary config struct to be extended via targeted
and/or specialized sub configs, and validate all of
them in sequence without bloating the main package
with the actual validation logic.
2019-04-01 16:33:50 -07:00
Conner Fromknecht
6d3e081f7b
channeldb: accept cache sizes in ChannelGraph 2019-04-01 16:33:36 -07:00
Conner Fromknecht
baa968b1ff
channeldb/options: add Options w/ functional modifiers 2019-04-01 16:33:13 -07:00
Conner Fromknecht
ae3a00a5da
channeldb/graph+db: integrate reject and channel caches 2019-04-01 16:32:52 -07:00
Conner Fromknecht
b20a254faa
channeldb/channel_cache: add channelCache w/ randomized eviction 2019-04-01 16:25:31 -07:00
Conner Fromknecht
af0ea3590b
channeldb/reject_cache: add rejectCache w/ randomized eviction 2019-04-01 16:25:31 -07:00
Conner Fromknecht
3c46ceec1d
channeldb/db: init one ChannelGraph per channeldb.DB 2019-04-01 16:25:30 -07:00
Conner Fromknecht
ecbb78651a
channeldb/graph: add newChannelGraph constructor 2019-04-01 16:25:30 -07:00
Conner Fromknecht
ac315fd051
channeldb/db: remove unused buffer pool 2019-04-01 16:25:30 -07:00
Conner Fromknecht
2f927493c8
Revert "channeldb: convert invoice settle/cancel calls to use Batch"
This reverts commit da76c34418a40761fdc99e11200755661699f7eb.
2019-04-01 16:25:30 -07:00
Conner Fromknecht
e35f676b4c
Revert "channeldb: convert concurrent channel state machine calls to use Batch"
This reverts commit e8da6dd0b48843a48cecdcff894f48e8020d940e.
2019-04-01 16:25:30 -07:00
Conner Fromknecht
f6d93c8e5d
Revert "channeldb: convert all Update calls to use Batch"
This reverts commit 3555d3dbd431fe3ddf2660d4dc270b5a595fe517.
2019-04-01 16:25:26 -07:00
Olaoluwa Osuntokun
c37ea68ba6
Merge pull request #2313 from Roasbeef/static-chan-backups
multi: implement new safe static channel backup and recovery scheme, RPCs, and cli commands
2019-04-01 16:22:37 -07:00
Johan T. Halseth
8087ea4c4c
Merge pull request #2775 from wpaulino/reliable-sender-chan-update
discovery: check if stale within isMsgStale for ChannelUpdate messages
2019-04-01 12:08:33 +02:00
Olaoluwa Osuntokun
f2160273b7
test: add new series of itests for various SCB restore scenarios
In this commit, we add 4 new itests for exercising the SCB restore
process via 4 primary scenarios: recover from backup using RPC, recover
from file using RPC, recover channels during init/creation, recover
channels during unlock. With all fields populated there're a total of 24
new scenarios to cover. At the time of authoring of this commit, the
other scenarios (bits are: initiator, updates, private) have been left
out for now, as they increased the run time of the integration tests
significantly.
2019-03-28 17:54:15 -07:00
Olaoluwa Osuntokun
e3029dee45
test: add new itest for exporting SCBs 2019-03-28 17:54:13 -07:00
Olaoluwa Osuntokun
266ddbaceb
test: add new test for streaming SCB updates 2019-03-28 17:54:12 -07:00
Olaoluwa Osuntokun
acc37f7449
test: refactor testDataLossProtection to extract core DLP scenario out to new func
In this commit, we modify the core testDataLossProtection test to
extract the primary DLP assertion logic into a new function. We do this,
as the upcoming SCB tests will fallback to this test after some initial
set up.
2019-03-28 17:54:11 -07:00
Olaoluwa Osuntokun
e3626d1af9
lntest: update to new RestoreNodeWithSeed API 2019-03-28 17:54:09 -07:00
Olaoluwa Osuntokun
ea6ed7b8d2
test: update to new getChanPointFundingTxid
In this commit, we update all uses of the `getChanPointFundingTxid` to
match the new function signature. We no longer need to convert to a
chainhash.Hash, as the method does so underneath now.
2019-03-28 17:54:08 -07:00
Olaoluwa Osuntokun
b451536483
lntest: extend the restore/restart methods to also accept optional SCBs
In this commit, we modify the `RestoreNodeWithSeed` and `RestartNode`
methods to also accept an SCB. This will be useful in new integration
tests to properly exercise the various restore/restart scenarios using
static channel backups.
2019-03-28 17:54:07 -07:00
Olaoluwa Osuntokun
b8cbe3a1f8
channeldb: in RestoreChannelShells don't exit if edge already exists
During the restore process, it may be possible that we have already
heard about our prior edge from a node on the network (or our channel
peers). As a result, we shouldn't exit if this happens, and instead
should continue with the rest of the restoration process.
2019-03-28 17:54:06 -07:00
Olaoluwa Osuntokun
26c52892cb
server: convert Start/Stop methods to use sync.Once
In this commit, we convert the server's Start/Stop methods to use the
sync.Once. We do this in order to fix concurrency issues that would
allow certain queries to be sent to the server before it has actually
fully start up. Before this commit, we would set started to 1 at the
very top of the method, allowing certain queries to pass before the rest
of the daemon was had started up.

In order to fix this issue, we've converted the server to using a
sync.Once, and two new atomic variables for clients to query to see if
the server has fully started up, or is in the process of stopping.
2019-03-28 17:54:05 -07:00
Olaoluwa Osuntokun
1f187c0116
cmd/lncli: add new verifychanbackup command 2019-03-28 17:54:03 -07:00
Olaoluwa Osuntokun
ead8aed6d6
rpc: implement new VerifyChanBackup command 2019-03-28 17:54:02 -07:00
Olaoluwa Osuntokun
7ab8096d4a
lnrpc: add new VerifyChanBackup command 2019-03-28 17:54:01 -07:00
Olaoluwa Osuntokun
26f6fd7db2
contractcourt: ignore all other dispatch cases in closeObserver when recovered chan
In this commit, we modify the `closeObserver` to fast path the DLP
dispatch case if we detect that the channel has been restored. We do
this as otherwise, we may inadvertently enter one of the other cases
erroneously, causing us to now properly look up their dlp commitment
point.
2019-03-28 17:54:00 -07:00