Commit Graph

288 Commits

Author SHA1 Message Date
Wilmer Paulino
04a7cda3d5
Merge pull request #3534 from alrs/discovery-test-improvements
discovery: Goroutine Test Fixes and Linting
2019-09-25 16:12:30 -07:00
Lars Lehtonen
0cae1e69ab discovery: error string lint fixes
discovery: lint fix to remove append loop
2019-09-25 18:42:38 +00:00
Lars Lehtonen
58c23074d1 discovery: use error channels with test goroutines 2019-09-25 18:41:42 +00:00
Joost Jager
c80feeb4b3
routing+discovery: extract local channel manager
The policy update logic that resided part in the gossiper and
part in the rpc server is extracted into its own object.

This prepares for additional validation logic to be added for policy
updates that would otherwise make the gossiper heavier.

It is also a small first step towards separation of our own channel data
from the rest of the graph.
2019-09-23 13:07:08 +02:00
Joost Jager
4b2eb9cb81
discovery: push max htlc migration further up the call tree
As a preparation for making the gossiper less responsible for validating
and supplementing local channel policy updates, this commits moves the
on-the-fly max htlc migration up the call tree. The plan for a follow up
commit is to move it out of the gossiper completely for local channel
updates, so that we don't need to return a list of final applied policies
anymore.
2019-09-23 13:07:06 +02:00
Joost Jager
339ff357d1
channeldb: invalidate channel signature cache on update 2019-09-23 13:07:04 +02:00
Joost Jager
5090bb27ad
discovery: remove redundant signature setting
The signature is retrieved, not used and overwritten with a
new signature.
2019-09-23 13:07:02 +02:00
Conner Fromknecht
1d41d4d666
multi: move WaitPredicate, WaitNoError, WaitInvariant to lntest/wait 2019-09-19 12:46:29 -07:00
Johan T. Halseth
92123c603d
gossiper: retransmit self NodeAnnouncement 2019-09-16 10:54:42 +02:00
Johan T. Halseth
24004fcb37
gossiper+server: define SelfNodeAnnouncement 2019-09-16 10:54:42 +02:00
Johan T. Halseth
e36d15582c
discovery/gossiper test: add TestRetransmit
This commit adds a test that ensures outdated announcements are
retransmitted when the RetransmitTicker ticks.
2019-09-16 10:54:38 +02:00
Johan T. Halseth
70d63abe9f
discovery/test: set global test timestamp 2019-09-16 10:23:01 +02:00
Johan T. Halseth
8b9fd039ec
discovery/gossiper test: remove mockGraphSource.SelfEdges 2019-09-16 10:23:01 +02:00
Johan T. Halseth
e201fbe396
discovery+server: RetransmitDelay->RetransmitTicker
Also let retransmitStaleChannels take a timestamp, to make it easier to
test.
2019-09-16 10:23:01 +02:00
Johan T. Halseth
74c9551564
discovery+server: make RebroadcastInterval part of config 2019-09-16 10:23:00 +02:00
Johan T. Halseth
3d8f194670
discovery/gossiper: extract adding nodeAnnouncement into method 2019-09-16 10:23:00 +02:00
Joost Jager
3d7de2ad39
multi: remove dead code 2019-09-10 17:21:59 +02:00
Valentine Wallace
8ce7f82da0 discovery+switch: apply zero forwarding policy updates in-memory as well as on disk
In this commit, we fix a bug where if a user updates a forwarding policy to be
zero, the update will be applied to the policy correctly on-disk, but not
in-memory.

We solve this issue by having the gossiper return the list of on-disk updated
policies and passing these policies to the switch, so the switch can assume
that zero-valued fields are intentional and not just uninitialized.
2019-09-09 23:39:44 -07:00
Wilmer Paulino
2e122a807b
Merge pull request #3406 from cfromknecht/die-spew
pilot+discovery: die spew
2019-08-22 15:33:56 -07:00
Wilmer Paulino
e15e524637
discovery: prevent broadcast of anns received during initial graph sync
There's no need to broadcast these as we assume that online nodes have
already received them. For nodes that were offline, they should receive
them as part of their initial graph sync.
2019-08-21 12:06:33 -07:00
Conner Fromknecht
e2a53f71d0
pilot+discovery: remove info spews 2019-08-20 14:13:05 -07:00
Wilmer Paulino
c405e89197
discovery: check non-nil syncer upon historical sync tick 2019-08-13 18:23:05 -07:00
Wilmer Paulino
977c139f3c
discovery: handle graph synced status after stalled initial historical sync
This ensures that the graph synced status is marked true at some point
once a historical sync has completed. Before this commit, a stalled
historical sync could cause us to never mark the graph as synced.
2019-08-06 17:56:55 -07:00
Wilmer Paulino
af4234f680
discovery: allow the SyncManager to report whether the graph is synced 2019-08-06 17:56:54 -07:00
Conner Fromknecht
8cebddfe50
discovery/gossiper: thread IgnoreHistoricalFilters to sync manager 2019-07-30 17:25:47 -07:00
Conner Fromknecht
a3e690e253
discovery/sync_manager: init all syncers with IgnoreHistoricalFilters 2019-07-30 17:25:31 -07:00
Conner Fromknecht
35a2de23a3
discovery/syncer: add flag to prevent historical gossip filter dump 2019-07-30 17:25:15 -07:00
Conner Fromknecht
a4097c113a
discovery/gossiper: remove unused SynchronizeNode method 2019-07-15 14:11:18 -07:00
Conner Fromknecht
933e723ec7
Merge pull request #3178 from federicobond/once-refactor
multi: replace manual CAS with sync.Once in several more modules
2019-07-08 20:33:44 -07:00
Olaoluwa Osuntokun
9c957193cf
discovery: remove retries from DNS based SampleNodeAddrs, allow down seeds
In this commit, we modify the `SampleNodeAddrs` method to no longer
retry itself. Instead, we'll now leave this task to the caller of the
this method. Additionally, we'll no longer return with an error if we
can't hit a particular seed. Instead, we'll log the error and move onto
the next seed. Finally, we'll also no longer require that the DNS seed
has a secondary seed in order to support a wider array of DNS seeds.
2019-06-28 16:10:47 -07:00
Wilmer Paulino
67132d4ee3
discovery: set source of node announcement broadcast to belonging node
We do this to ensure the node announcement propagates to our channel
counterparty. At times, the node announcement does not propagate to them
when opening our first channel due to a race condition between
IsPublicNode and processing announcement signatures. This isn't
necessary for channel updates and announcement signatures as we send
those to our channel counterparty directly through the reliable sender.
2019-06-20 13:59:37 -07:00
Federico Bond
0a9141763e multi: replace manual CAS with sync.Once in several more modules 2019-06-12 09:37:26 -03:00
Federico Bond
9bd3055fb8 discovery,fundingmanager: avoid serialization in NotifyWhenOnline 2019-06-04 16:36:21 -03:00
Conner Fromknecht
f8287b0080
Merge pull request #2985 from johng/sub-batch
Broadcast gossip announcements in sub batches
2019-05-28 17:05:06 -07:00
Johan T. Halseth
6ba6982ae7
discovery/sync_manager_test: add TestSyncManagerHistoricalSyncOnReconnect
TestSyncManagerHistoricalSyncOnReconnect tests that the sync manager will
re-trigger a historical sync when a new peer connects after a historical
sync has completed, but we have lost all peers.
2019-05-24 11:05:30 +02:00
Johan T. Halseth
526486ae24
discovery/sync_manager: restart historical sync on first connected peer
To handle the case where we have been without peers, and get a new
connection, we reset the historical scan booleans when the first active
syncer is connected to trigger another historical sync.
2019-05-24 11:05:29 +02:00
John Griffith
cf2885dd4a discovery: test we calculate and generate correct sub batch sizes 2019-05-23 10:51:25 +01:00
John Griffith
9eb5fe9587 discovery: split gossiper announcement into sub batches 2019-05-23 10:51:25 +01:00
Johan T. Halseth
c4415f0400
Merge pull request #3044 from cfromknecht/spelling-fixes
multi: fix spelling mistakes
2019-05-07 08:50:36 +02:00
Conner Fromknecht
17b2140cb5
multi: fix spelling mistakes 2019-05-04 15:35:37 -07:00
Olaoluwa Osuntokun
985902be27
Merge pull request #2916 from cfromknecht/split-syncer-query-reply
discovery: make gossip replies synchronous
2019-04-29 17:40:13 -07:00
Johan T. Halseth
ee257fd0eb
multi: move Route to sub-pkg routing/route 2019-04-29 14:52:33 +02:00
Conner Fromknecht
5251ebe117
discovery/syncer_test: fix benign off-by-one in DelayDOS test
Prior to this change, the numQueryResponses that we calculated would be
one more than what we actually wanted since it didn't account for the
initial QueryChannelRange msg. This resulted in the test sending one
extra delayed query than was configured. This doesn't fundamentally
impact the test, but does make what happens in the test more reflective
of the configuration.
2019-04-26 20:05:23 -07:00
Conner Fromknecht
bf4543e2bd
discovery/syncer: make gossip sends synchronous
This commit makes all replies in the gossip syncer synchronous, meaning
that they will wait for each message to be successfully written to the
remote peer before attempting to send the next. This helps throttle
messages the remote peer has requested, preventing unintended
disconnects when the remote peer is slow to process messages. This
changes also helps out congestion in the peer by forcing the syncer to
buffer the messages instead of dumping them into the peer's queue.
2019-04-26 20:05:10 -07:00
Conner Fromknecht
23d10336c2
discovery/syncer: separate query + reply handlers
This commit creates a distinct replyHandler, completely isolating the
requesting state machine from the processing of queries from the remote
peer. Before the two were interlaced, and the syncer could only reply to
messages in certain states. Now the two will be complete separated,
which is preliminary step to make the replies synchronous (as otherwise
we would be blocking our own requesting state machine).

With this changes, the channelGraphSyncer of each peer will drive the
replyHanlder of the other. The two can now operate independently, or
even spun up conditionally depending on advertised support for gossip
queries, as shown below:

          A                                 B
 channelGraphSyncer ---control-msg--->
                                        replyHandler
 channelGraphSyncer <--control-msg----
           gossiper <--gossip-msgs----

                    <--control-msg---- channelGraphSyncer
       replyHandler
                    ---control-msg---> channelGraphSyncer
                    ---gossip-msgs---> gossiper
2019-04-26 20:03:14 -07:00
Wilmer Paulino
d68842ee9e
discovery: queue active syncers until initial historical sync signal
In this commit, we begin to queue any active syncers until the initial
historical sync has completed. We do this to ensure we can properly
handle any new channel updates at tip. This is required for fresh nodes
that are syncing the channel graph for the first time. If we begin
accepting updates at tip while the initial historical sync is still
ongoing, then we risk not processing certain updates since we've yet to
learn of the channels themselves.
2019-04-24 13:20:57 -07:00
Wilmer Paulino
07136a5bc2
discovery: handle initial historical sync disconnection
In this commit, we add logic to handle a peer with whom we're performing
an initial historical sync disconnecting. This is required to ensure we
get as much of the graph as possible when starting a fresh node. It will
also serve useful to ensure we do not get stalled once we prevent active
GossipSyncers from starting until the initial historical sync has
completed.
2019-04-24 13:20:55 -07:00
Wilmer Paulino
227e492ccf
discovery: make historicalSync transition synchronous
We do this to ensure that the state transition from chansSynced to
syncingChans has occurred by the time we return back to the caller.
2019-04-24 13:20:18 -07:00
Wilmer Paulino
72e9674cff
discovery: simplify chooseRandomSyncer helper 2019-04-24 13:20:16 -07:00
Wilmer Paulino
29baa12254
discovery: synchronize new/stale GossipSyncers with syncerHandler
Now that the roundRobinHandler is no longer present, this commit aims to
clean up and simplify some of the logic surrounding initializing/tearing
down new/stale GossipSyncers from the SyncManager. Along the way, we
also synchronize these calls with the syncerHandler, which will serve
useful in future work that allows us to recovery from initial historical
sync disconnections.
2019-04-24 13:19:09 -07:00