Commit Graph

96 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
800eea931f
build+multi: switch from bolt to bbolt
In this commit, we switch from boltbd/bolt to coreos/bbolt as the
former is no longer being actively maintained.
2018-03-10 19:01:13 -08:00
Olaoluwa Osuntokun
e7e4cdcf49
discovery: avoid always validating ECDSA sigs by asking router if item is fresh
In this commit, we reduce the amount of unnecessary work that the
gossiper can carry out. When CPU profiling some nodes, I noticed that
we’d spend a lot of time validating the signatures for an announcement,
only to realize that the router already had it.

To remedy this, we’ll use the new methods added to the channel router
in order to avoid unnecessarily validating an announcement that is
actually stale. This should reduce memory usage (since it uses big
int’s under the scenes), and also idle CPU usage.
2018-03-06 13:34:51 -05:00
practicalswift
b8e1351cf3 multi: fix some recently introduced typos 2018-02-18 15:27:29 -08:00
Olaoluwa Osuntokun
bf05e47780
discovery: add additional gossiper level reject cache
In this commit, we’ll add a new reject cache to ensure that we don’t
attempt to re-process any announcements already rejected by the
ChannelRouter.
2018-02-06 20:14:35 -08:00
Olaoluwa Osuntokun
cd9d2d7e6f
discovery: update graph API usage to match recent API changes 2018-02-06 20:14:32 -08:00
Daniel McNally
8543497dcc multi: fixing it's/its typos in comments 2018-02-06 19:13:07 -08:00
practicalswift
a93736d21e multi: comprehensive typo fixes across all packages 2018-02-06 19:11:11 -08:00
nsa
e132ad8433 torsvc: Added new module to house Tor functions
This commit adds a new module named 'torsvc' which houses all Tor
functionality in an attempt to isolate it and make it reusable in
other projecs. Some additional tweaks were made to config.go and
to the bootstrapper.
2018-02-05 17:37:46 -08:00
nsa
e2142c778f multi: Added Tor support
This commit adds Tor support. Users can set the --TorSocks flag
to specify which port Tor's SOCKS5 proxy is listening on so that
lnd can connect to it. When this flag is set, ALL traffic gets
routed over Tor including DNS traffic. Special functions for
DNS lookups were added, and since Tor doesn't natively support
SRV requests, the proxySRV function routes connects us to
a DNS server via Tor and SRV requests can be issued directly
to the DNS server.

Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
2018-02-05 17:37:46 -08:00
Olaoluwa Osuntokun
0a4de859a2
discovery+routing: reduce number of active validation barrier jobs
In order to reduce high CPU utilization during the initial network view
sync, we slash down the total number of active in-flight jobs that can
be launched.
2018-01-28 14:55:32 -08:00
Johan T. Halseth
a55c74f80c
discovery/gossiper: restrict database access per channel ID
This commit uses the multimutex.Mutex to esure database
state stays consistent when handling an announcement, by
restricting access to one goroutine per channel ID.

This fixes a bug where the goroutine would read the
database, make some decisions based on what was read,
then write its data to the database, but the read data
would be outdated at this point. For instance, an
AuthProof could have been added between reading the
database and when the decision whether to announce
the channel is made, making it not announce it.
Similarly, when receiving the AuthProof, the edge
policy could be added between reading the edge state
and adding the proof to the database, also resulting
in the edge not being announced.
2018-01-24 10:26:40 +01:00
Johan T. Halseth
3e01529de4
discovery/gossiper: don't reset batch after broadcast
This commit fixes a bug that could cause annoucements
to get lost, and resultet in flaky integration tests.
After a set of announcements was broadcastet, we would
reset (clear) the announcement batch, making any
annoucement that was added between the call to Emit()
and Reset() to be deleted, without ever being broadcast.

We can just remove the Reset() call, as the batch will
actually be reset within the call to Emit(), making
the previous call only delete those messages we hadn't
sent yet.
2018-01-24 10:26:39 +01:00
Olaoluwa Osuntokun
69e6ec9954
peer+funding: remove unneeded channel handoff code with the breach arbiter
We no longer need to hand off new channels that come online as the
chainWatcher will be persistent, and always have an active signal for
the entire lifetime of the channel.
2018-01-22 19:19:50 -08:00
Olaoluwa Osuntokun
8807d1d752
fundingmgr: add new function closure to send new channels to the ChainArbitrator 2018-01-22 19:19:38 -08:00
Olaoluwa Osuntokun
ebd97b83f3
discovery: ensure we only send node announcements that have active channels 2018-01-16 17:18:21 -08:00
Johan T. Halseth
483abbee5b
discovery: make sure ChannelUpdates always have incremented timestamp
This commit ensures that we always increment the timestamp of
ChannelUpdates we send telling the network about changes to
our channel policy. We do this because it could happen
(especially during tests) that we issued an update, but the
ChannelUpdate would have the same timestamp as our last
ChannelUpdate, and would be ignored by the network.
2018-01-12 22:57:03 +01:00
Johan T. Halseth
8370fa2cde
discovery/gossiper: apply TimeLockDelta to edge when processing policy update
Also rename various instances of "FeeUpdate" to "PolicyUpdate"
2018-01-12 22:56:32 +01:00
Johan T. Halseth
3f0a525331
discovery test: update gossiper test to account for timestamp deduping 2018-01-09 13:09:28 +01:00
Johan T. Halseth
ea95f37277
discovery: keep newest messages when deduping in gossiper
This commit makes the gossiper aware of the timestamps
of ChannelUpdates and NodeAnnouncements, such that it
only keeps the newest message when deduping. Earlier
it would always keep the last received message, which
in some cases could be outdated.
2018-01-09 13:09:25 +01:00
Johan T. Halseth
2ddd139084 discovery: check proof for nilness before creating chanAnn
This commit makes sure we are not attempting to create a
channel announcement with a nil ChannelAuthProof, as that
could cause a crash at startup whe the gossiper would
attempt to reprocess an edge coming from the fundingmanager.

It also makes sure we check the correct error returned from
processRejectedEdge.
2018-01-02 19:51:38 +01:00
Olaoluwa Osuntokun
6b0f984e31
discovery: update tests to reflect new broadcast related changes 2017-12-26 16:29:37 +01:00
Olaoluwa Osuntokun
38715ca768
discovery: refactor re-processing rejected announcements into new method 2017-12-26 16:28:23 +01:00
Olaoluwa Osuntokun
68dabe3594
discovery: update usage of d.cfg.Broadcast to reflect recent API change
When broadcasting a set of messages, we must now call broadcast for
each (msg, …senders) tuple, rather than just calling it once for the
entire batch.
2017-12-26 16:26:34 +01:00
Olaoluwa Osuntokun
f45502f890
discovery: update message processing to emit networkMsg rather than lnwire.Message
In this commit, we make an incremental step towards page of the new
feature of deDupedAnnoucnements to return the set of senders for each
message. All methods the process new channel announcements, will now
return an instance of networkMsg rather than lnwire.Message. This will
allow passing the returned announcement directly into
deDupedAnnoucnements.AddMsg().
2017-12-26 16:23:12 +01:00
Olaoluwa Osuntokun
f1b40e0b4d
discovery: modify deDupedAnnouncements to keep track of senders for each msg
In this commit, we modify the deDupedAnnouncements struct slightly. The
element of this struct will now keep track of the set of senders that
sent a particular message. Each time a message is added, we’ll replace
the new message with the old (as normal), but we’ll also add the new
sender to the set of known senders.

With this new feature, we’ll be able to avoid re-sending a message to
the peer that sent it to us in the first place.
2017-12-26 16:19:03 +01:00
Matt Drollette
d6172ca8b4 discovery: correctly format ipv6 addresses from dns seed 2017-12-22 20:12:35 +01:00
Johan T. Halseth
c3aa23f061 discovery test: update gossiper test for new database structure 2017-12-19 13:01:59 -06:00
Johan T. Halseth
e3ae204fcb discovery: make gossiper able to resend failed AnnounceSignatures
This commit makes the gossiper track the state of a local
AnnounceSignature message, such that it can retry sending
it to the remote peer if needed. It will also persist this
state in the WaitingProofStore, such that it can resume
from this state at startup.
2017-12-19 13:01:59 -06:00
Matt Drollette
adf0d98194 multi: fix several typos in godoc comments 2017-12-17 18:40:05 -08:00
Johan T. Halseth
b74a281b57 discovery test: test for remote ChannelUpdate for unknown channel
This commit adds a test that ensures that if we receive a
ChannelUpdate for a channel we don't know about, it will
be reprocessed after we receive a ChannelAnnouncement for
that channel.
2017-12-17 18:35:34 -08:00
Johan T. Halseth
db829cd0c5 discovery: reprocess premature ChannelUpdates
This commit makes the gossiper store received ChannelUpdates
that is not for any known channel in a map, such that they
can be reprocessed when the ChannelAnnouncement arrives.

This is done to handle the case where we receive a ChannelUpdate
from our channel counterparty before we have been able to process
our own local ChannelAnnouncement.
2017-12-17 18:35:34 -08:00
Johan T. Halseth
10b838e9b4 gossiper: private channel cleanup + comments
This commit adds some comments and does some cleanup
of the logic that makes sure non-public channels
(channels with no AuthProof) are not broadcasted
to the network.
2017-12-17 18:35:34 -08:00
nsa
dab7f31e44 gossiper: Only send channel updates for edges with AuthProof. 2017-12-17 18:35:34 -08:00
Olaoluwa Osuntokun
37c54153fa
discovery: properly set node's color when syncing graph state
This commit fixes an existing bug wherein we would blank out a node’s
color instead of properly setting the field when syncing graph state
with another node This would cause the node to reject the node
announcement and we would generate an we would invalidate the signature
of the node announcement. We fix this simply by properly setting the
node announcement.
2017-12-02 18:33:21 -08:00
Olaoluwa Osuntokun
ee7c3ff01c
discovery: properly set node's color field when adding NodeAnnouncement to router
In this commit, we fix an existing bug when processing new node
announcement. Before this commit, we wouldn’t also copy over the color
field of a node’s announcement. As a result, when went to synchronize
our graph state with that of a connecting peer, we would generate an
invalid node announcement. We fix this by properly setting the color
field of a node.
2017-12-02 18:28:41 -08:00
Olaoluwa Osuntokun
3802cb90df
discovery: publicly export announcement validation related functions 2017-12-01 19:31:54 -08:00
Olaoluwa Osuntokun
12a0488bc6
discovery: use lnwire.ChanUpdateFlag masks when examinig ChannelUpdate msgs
In this commit, we now properly examine the Flag field within the
ChannelUpdate message as a bitfield. Before this commit we would
manually check the flags for zero or one. This was incorrect as a their
bit has now been defined. To properly dispatch the messages, we’ll now
treat it properly as a bitmask.
2017-11-30 22:37:21 -08:00
Olaoluwa Osuntokun
3e3de26ac9
discovery: set proper flag within createChanAnnouncement
In this commit, we fix an existing bug within the
createChanAnnouncement function. Before we would set the flag to be 0,
or 1 depending on which edge it was. This was incorrect as since then
additional flags have been defined. We now properly set the entire
flag, rather than taking a shortcut. With this, we’ll properly
advertise all ChannelUpdate announcements.
2017-11-30 22:33:00 -08:00
Olaoluwa Osuntokun
c0b8c29382
discovery: continue in loop after processing announcement signatures 2017-11-29 18:35:19 -08:00
Olaoluwa Osuntokun
4ab16b5c91
discovery: process AnnouncementSignatures msgs serially 2017-11-29 17:53:21 -08:00
Olaoluwa Osuntokun
3067d05ae8
discovery: validate incoming announcements in parallel 2017-11-29 16:45:14 -08:00
Olaoluwa Osuntokun
2dcd2b8a6d
discovery: add a mutex in order to make deDupedAnnouncements thread-safe 2017-11-29 16:21:16 -08:00
Olaoluwa Osuntokun
2861f6e5f4
discovery: add additional comments to TestDeDuplicatedAnnouncements 2017-11-15 18:25:33 -08:00
Laura Cressman
918ef2808f gossiper_test: add tests for deDupedAnnouncements struct
Add tests for new deDupedAnnouncements struct in gossiper_test.
Test the various functionalities of the struct - that empty
struct contains no announcements, that announcements of each type
can be added and properly de-duplicated, that the batch of
announcements is delivered correctly, and that after reset the
struct again contains no announcements.
2017-11-15 16:50:19 -08:00
Laura Cressman
ed6ad22e85 config+server+networktest: make trickleDelay configurable
Add option to set trickleDelay for AuthenticatedGossiper in
command line, with default value of 300 milliseconds. Pass this
value to newServer, which uses it when creating a new instance of
AuthenticatedGossiper. Also set this value to 300 milliseconds when
creating nodes in integration tests.
2017-11-15 16:50:19 -08:00
Laura Cressman
39d38da732 discovery: create deDupedAnnouncements struct in gosspier.go
For Part 1 of Issue #275. Create isolated private struct in
networkHandler goroutine that will de-duplicate
announcements added to the batch. The struct contains maps
for each of channel announcements, channel updates, and
node announcements to keep track of unique announcements.

The struct has a Reset method to reset stored announcements, an
AddMsg(lnwire.Message) method to add a new message to the current
batch, and a Batch method to return the set of de-duplicated
announcements.

Also fix a few minor typos.
2017-11-15 16:50:19 -08:00
Olaoluwa Osuntokun
8c181b6f4e
discovery: return an error if ChannelUpdate has unknown flags
In this commit, we fix an existing bug that could result in a panic if
we received a ChannelUpdate message with an unknown set of flags. If
the flag wasn’t set to zero or one, then the pubKey parameter would be
still nil when we attempted to validate it, causing an error to occur.

We remedy this by instead returning an error if the flags are unknown.
In a future commit, we will properly handle the set of flags that
indicates the channel should be disabled.
2017-11-11 14:59:16 -08:00
Olaoluwa Osuntokun
65cf83489a
discovery: fix linter warning 2017-10-30 19:04:00 -07:00
Olaoluwa Osuntokun
5b226a9d37
discovery: add TCP fallback for DNSSeedBootstrapper
In this commit, we add a TCP fallback option for the
DNSSeedBootstrapper. We’ve received many reports of users unable to
bootstrap properly to the network due to the size of the SRV records we
currently return. It has been observed that many revolvers will simply
truncate and ignore the response due to the (current size).

To resolve (no pun intended) we now attempt to detect this failure mode
and will fallback to a manual TCP resolution in the case that our SRV
query over UDP fails. We do this by querying the special record at the
"soa." sub-domain of supporting DNS servers. The retuned IP address
will be the IP address of the authoritative DNS server. Once we have
this IP address, we'll connect manually over TCP to request the SRV
record. This is necessary as the records we return are currently too
large for a class of resolvers, causing them to be filtered out.
2017-10-30 19:00:27 -07:00
Conner Fromknecht
3b2e4caa93 discovery/gossiper: isolate sync with node from query
This commit refactors the SynchronizeNode logic such that
  it can be called without interacting with the gossiper's
  main execution loop. This method does not require access
  to any of the gossiper's internal state, making the change
  fairly straightforward. The primary motivation behind
  this change is to minimize the possibility of introducing
  deadlock scenarios between the gossiper and server.
2017-10-27 15:48:37 -07:00