Commit Graph

121 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
d98d4523e1
Merge pull request #1159 from wpaulino/onion-service-support
multi: add onion services support
2018-06-04 21:46:51 -07:00
Wilmer Paulino
2e0484be19
multi: ensure addresses are no longer assumed to be TCP addresses only
In this commit, we go through the codebase looking for TCP address
assumptions and modifying them to include the recently introduced onion
addresses. This enables us to fully support onion addresses within the
daemon.
2018-06-04 20:41:49 -07:00
Wilmer Paulino
3bb1733fa2
discovery+server: use network-specific functions for fallback SRV lookup
In this commit, we fix a bug where a fallback SRV lookup would leak
information if `lnd` was set to route connections over Tor. We solve
this by using the network-specific functions rather than the standard
ones found in the `net` package.
2018-06-04 20:41:43 -07:00
maurycy
3be08e69cf multi: 64bit aligment of atomic vars on arm/x86-32 2018-06-04 20:02:34 -07:00
Olaoluwa Osuntokun
e6d46f681b
discovery: if gossipSyncer is shutting down, don't filter messages 2018-06-04 17:18:54 -07:00
Olaoluwa Osuntokun
0ec4a06e6b
discovery: fix deadlock by ensure we don't hold sync mutex during send
In this commit, we fix an existing deadlock in the
gossiper->server->peer pipeline by ensuring that we're not holding the
syncer mutex while we attempt to have a syncer filter out the rest of
gossip messages.
2018-06-04 17:18:17 -07:00
Olaoluwa Osuntokun
a9340d22c4 discovery: ensure stopping/starting a gossipSyncer is idempotent 2018-06-04 17:18:07 -07:00
Olaoluwa Osuntokun
b3ac3492a0
discovery: if unable to find gossipSyncer for peer, create one
In this commit we fix an existing bug caused by a scheduling race
condition. We'll now ensure that if we get a gossip message from a peer
before we create an instance for it, then we create one on the spot so
we can service the message. Before this commit, we would drop the first
message, and therefore never sync up with the peer at all, causing them
to miss channel announcements.
2018-05-31 17:54:45 -07:00
Olaoluwa Osuntokun
994d9cf7e4
discovery: attempt to request the full chan ann for stray chan updates
In this commit, we extend the AuthenticatedGossiper to take advantage of
the new query features in the case that it gets a channel update w/o
first receiving the full channel announcement. If this happens, we'll
attempt to find a syncer that's fully synced, and request the channel
announcement from it.
2018-05-31 16:31:01 -07:00
Olaoluwa Osuntokun
6a6700e629
discovery: add new SyncState method to gossipSyncer
This new method allows outside callers to sample the current state of
the gossipSyncer in a concurrent-safe manner. In order to achieve this,
we now only modify the g.state variable atomically.
2018-05-31 16:31:00 -07:00
Olaoluwa Osuntokun
55dfc17c05
discovery: de-duplicate node announcements with legacy chan graph sync
Fixes #1072.
2018-05-31 16:30:58 -07:00
Olaoluwa Osuntokun
0016b3f458
discovery+lnd: create new chanSeries impl of the ChannelGraphTimeSeries interface
In this commit, we create a new concrete implementation for the new
discovery.ChannelGraphTimeSeries interface. We also export the
createChannelAnnouncement method to allow the chanSeries struct to
re-use the existing code for creating wire messages from the database
structs.
2018-05-31 16:30:55 -07:00
Olaoluwa Osuntokun
45bd544f11
discovery: update AuthenticatedGossiper to be aware of new gossipSyncers
In this commit, we update the logic in the AuthenticatedGossiper to
ensure that can properly create, manage, and dispatch messages to any
gossipSyncer instances created by the server.

With this set of changes, the gossip now has complete knowledge of the
current set of peers we're conneted to that support the new range
queries. Upon initial connect, InitSyncState will be called by the
server if the new peer understands the set of gossip queries. This will
then create a new spot in the peerSyncers map for the new syncer. For
each new gossip query message, we'll then attempt to dispatch the
message directly to the gossip syncer. When the peer has disconnected,
we then expect the server to call the PruneSyncState method which will
allow us to free up the resources.

Finally, when we go to broadcast messages, we'll send the messages
directly to the peers that have gossipSyncer instances active, so they
can properly be filtered out. For those that don't we'll broadcast
directly, ensuring we skip *all* peers that have an active gossip
syncer.
2018-05-31 16:30:53 -07:00
Olaoluwa Osuntokun
5789ef7c10
discovery: add new gossipSyncer struct to manage sync state for each peer
In this commit, introduce a new struct, the gossipSyncer. The role of
this struct is to encapsulate the state machine required to implement
the new gossip query range feature recently added to the spec. With this
change, each peer that knows of this new feature will have a new
goroutine that will be managed by the gossiper.

Once created and started, the gossipSyncer will start to progress
through each possible state, finally ending at the chansSynced stage. In
this stage, it has synchronized state with the remote peer, and is
simply awaiting any new messages from the gossiper to send directly to
the peer. Each message will only be sent if the remote peer actually has
a set update horizon, and the message isn't before or after that
horizon.

A set of unit tests has been added to ensure that two state machines
properly terminate and synchronize channel state.
2018-05-31 16:30:53 -07:00
Conner Fromknecht
3854c1ed68
discovery/gossiper: exit early on validation barrier quit 2018-05-08 16:32:08 -07:00
Dimitris Tsapakidis
4009f7f874 multi: fix typos in comments 2018-04-17 19:03:27 -07:00
practicalswift
663c396235 multi: fix a-vs-an typos 2018-04-17 19:02:04 -07:00
Jim Posen
4b2cb68fe6 discovery: Fix formatting issue in log line. 2018-04-09 20:49:23 -07:00
Olaoluwa Osuntokun
3fa2e08665
test: update testUpdateChannelPolicy to ensure Bob's link uses the proper policies
In this commit, we update the testUpdateChannelPolicy to exercise the
recent set of changes within the switch. If one applies this test to a
fresh branch (without those new changes) it should fail. This is due to
the fact that before, Bob would attempt to apply the constraints of the
incoming link (which we updated) instead of the outgoing link. With the
recent set of changes, the test now properly passes.
2018-04-06 14:52:01 -07:00
Olaoluwa Osuntokun
9d4cea93f0
discovery: fix deadlock in processChanPolicyUpdate
In this commit, we fix an existing deadlock in the
processChanPolicyUpdate method. Before this commit, within
processChanPolicyUpdate, we would directly call updateChannel *within*
the ForEachChannel closure. This would at times result in a deadlock, as
updateChannel will itself attempt to create a write transaction in order
to persist the newly updated channel.

We fix this deadlock by simply performing another loop once we know the
set of channels that we wish to update. This second loop will actually
update the channels on disk.
2018-04-06 14:51:57 -07:00
Johan T. Halseth
ed6682ea4a
multi test: make mock adhere to api change 2018-04-03 22:04:02 +02:00
Olaoluwa Osuntokun
2326d4afeb
discovery: bump rejected edges up to the trace logging level 2018-03-23 15:50:56 -07:00
Olaoluwa Osuntokun
ce85632390
discovery: ensure target host for DNS fallback is fully qualified
In this commit, fix the inability of some users to connect to the DNS
seed using our direct TCP fallback. We do this as some resolvers filter
out our large SRV requests due to their size (they also include public
keys). Instead, we’ll use a direct TCP resolution in this case.
However, after a recent change, we forgot the period at the end of the
target DNS host. This is an issue as the domain needs to be fully
qualified.

The fix is easy, add a period within our string formatting to target
the proper sub-domain and SRV target.

Fixes #854.
2018-03-18 16:05:10 -07:00
Matt Drollette
1b7b35a8bc discovery: format log message with Errorf 2018-03-16 16:25:18 -07:00
Olaoluwa Osuntokun
30019538c4
discovery: update DNS bootstrapper to adhere to new query schema
In this commit, we update the DNS bootstrapper to match the new query
semantics expected by the new DNS server. We no longer hard code the
target DNS host, and instead, we’ll re-use the same target endpoint as
we only need the soaShim in order to establish a direct TCP connection
for the queries.
2018-03-14 18:27:29 -07:00
Olaoluwa Osuntokun
800eea931f
build+multi: switch from bolt to bbolt
In this commit, we switch from boltbd/bolt to coreos/bbolt as the
former is no longer being actively maintained.
2018-03-10 19:01:13 -08:00
Olaoluwa Osuntokun
e7e4cdcf49
discovery: avoid always validating ECDSA sigs by asking router if item is fresh
In this commit, we reduce the amount of unnecessary work that the
gossiper can carry out. When CPU profiling some nodes, I noticed that
we’d spend a lot of time validating the signatures for an announcement,
only to realize that the router already had it.

To remedy this, we’ll use the new methods added to the channel router
in order to avoid unnecessarily validating an announcement that is
actually stale. This should reduce memory usage (since it uses big
int’s under the scenes), and also idle CPU usage.
2018-03-06 13:34:51 -05:00
practicalswift
b8e1351cf3 multi: fix some recently introduced typos 2018-02-18 15:27:29 -08:00
Olaoluwa Osuntokun
bf05e47780
discovery: add additional gossiper level reject cache
In this commit, we’ll add a new reject cache to ensure that we don’t
attempt to re-process any announcements already rejected by the
ChannelRouter.
2018-02-06 20:14:35 -08:00
Olaoluwa Osuntokun
cd9d2d7e6f
discovery: update graph API usage to match recent API changes 2018-02-06 20:14:32 -08:00
Daniel McNally
8543497dcc multi: fixing it's/its typos in comments 2018-02-06 19:13:07 -08:00
practicalswift
a93736d21e multi: comprehensive typo fixes across all packages 2018-02-06 19:11:11 -08:00
nsa
e132ad8433 torsvc: Added new module to house Tor functions
This commit adds a new module named 'torsvc' which houses all Tor
functionality in an attempt to isolate it and make it reusable in
other projecs. Some additional tweaks were made to config.go and
to the bootstrapper.
2018-02-05 17:37:46 -08:00
nsa
e2142c778f multi: Added Tor support
This commit adds Tor support. Users can set the --TorSocks flag
to specify which port Tor's SOCKS5 proxy is listening on so that
lnd can connect to it. When this flag is set, ALL traffic gets
routed over Tor including DNS traffic. Special functions for
DNS lookups were added, and since Tor doesn't natively support
SRV requests, the proxySRV function routes connects us to
a DNS server via Tor and SRV requests can be issued directly
to the DNS server.

Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
2018-02-05 17:37:46 -08:00
Olaoluwa Osuntokun
0a4de859a2
discovery+routing: reduce number of active validation barrier jobs
In order to reduce high CPU utilization during the initial network view
sync, we slash down the total number of active in-flight jobs that can
be launched.
2018-01-28 14:55:32 -08:00
Johan T. Halseth
a55c74f80c
discovery/gossiper: restrict database access per channel ID
This commit uses the multimutex.Mutex to esure database
state stays consistent when handling an announcement, by
restricting access to one goroutine per channel ID.

This fixes a bug where the goroutine would read the
database, make some decisions based on what was read,
then write its data to the database, but the read data
would be outdated at this point. For instance, an
AuthProof could have been added between reading the
database and when the decision whether to announce
the channel is made, making it not announce it.
Similarly, when receiving the AuthProof, the edge
policy could be added between reading the edge state
and adding the proof to the database, also resulting
in the edge not being announced.
2018-01-24 10:26:40 +01:00
Johan T. Halseth
3e01529de4
discovery/gossiper: don't reset batch after broadcast
This commit fixes a bug that could cause annoucements
to get lost, and resultet in flaky integration tests.
After a set of announcements was broadcastet, we would
reset (clear) the announcement batch, making any
annoucement that was added between the call to Emit()
and Reset() to be deleted, without ever being broadcast.

We can just remove the Reset() call, as the batch will
actually be reset within the call to Emit(), making
the previous call only delete those messages we hadn't
sent yet.
2018-01-24 10:26:39 +01:00
Olaoluwa Osuntokun
69e6ec9954
peer+funding: remove unneeded channel handoff code with the breach arbiter
We no longer need to hand off new channels that come online as the
chainWatcher will be persistent, and always have an active signal for
the entire lifetime of the channel.
2018-01-22 19:19:50 -08:00
Olaoluwa Osuntokun
8807d1d752
fundingmgr: add new function closure to send new channels to the ChainArbitrator 2018-01-22 19:19:38 -08:00
Olaoluwa Osuntokun
ebd97b83f3
discovery: ensure we only send node announcements that have active channels 2018-01-16 17:18:21 -08:00
Johan T. Halseth
483abbee5b
discovery: make sure ChannelUpdates always have incremented timestamp
This commit ensures that we always increment the timestamp of
ChannelUpdates we send telling the network about changes to
our channel policy. We do this because it could happen
(especially during tests) that we issued an update, but the
ChannelUpdate would have the same timestamp as our last
ChannelUpdate, and would be ignored by the network.
2018-01-12 22:57:03 +01:00
Johan T. Halseth
8370fa2cde
discovery/gossiper: apply TimeLockDelta to edge when processing policy update
Also rename various instances of "FeeUpdate" to "PolicyUpdate"
2018-01-12 22:56:32 +01:00
Johan T. Halseth
3f0a525331
discovery test: update gossiper test to account for timestamp deduping 2018-01-09 13:09:28 +01:00
Johan T. Halseth
ea95f37277
discovery: keep newest messages when deduping in gossiper
This commit makes the gossiper aware of the timestamps
of ChannelUpdates and NodeAnnouncements, such that it
only keeps the newest message when deduping. Earlier
it would always keep the last received message, which
in some cases could be outdated.
2018-01-09 13:09:25 +01:00
Johan T. Halseth
2ddd139084 discovery: check proof for nilness before creating chanAnn
This commit makes sure we are not attempting to create a
channel announcement with a nil ChannelAuthProof, as that
could cause a crash at startup whe the gossiper would
attempt to reprocess an edge coming from the fundingmanager.

It also makes sure we check the correct error returned from
processRejectedEdge.
2018-01-02 19:51:38 +01:00
Olaoluwa Osuntokun
6b0f984e31
discovery: update tests to reflect new broadcast related changes 2017-12-26 16:29:37 +01:00
Olaoluwa Osuntokun
38715ca768
discovery: refactor re-processing rejected announcements into new method 2017-12-26 16:28:23 +01:00
Olaoluwa Osuntokun
68dabe3594
discovery: update usage of d.cfg.Broadcast to reflect recent API change
When broadcasting a set of messages, we must now call broadcast for
each (msg, …senders) tuple, rather than just calling it once for the
entire batch.
2017-12-26 16:26:34 +01:00
Olaoluwa Osuntokun
f45502f890
discovery: update message processing to emit networkMsg rather than lnwire.Message
In this commit, we make an incremental step towards page of the new
feature of deDupedAnnoucnements to return the set of senders for each
message. All methods the process new channel announcements, will now
return an instance of networkMsg rather than lnwire.Message. This will
allow passing the returned announcement directly into
deDupedAnnoucnements.AddMsg().
2017-12-26 16:23:12 +01:00
Olaoluwa Osuntokun
f1b40e0b4d
discovery: modify deDupedAnnouncements to keep track of senders for each msg
In this commit, we modify the deDupedAnnouncements struct slightly. The
element of this struct will now keep track of the set of senders that
sent a particular message. Each time a message is added, we’ll replace
the new message with the old (as normal), but we’ll also add the new
sender to the set of known senders.

With this new feature, we’ll be able to avoid re-sending a message to
the peer that sent it to us in the first place.
2017-12-26 16:19:03 +01:00