Commit Graph

2304 Commits

Author SHA1 Message Date
Jim Posen
9fd77a6e40 multi: Update lnd to use new feature vector API. 2017-10-17 22:47:20 -07:00
Jim Posen
af49752d4d peer: Update peer handling of received feature vectors.
This updates peer to be compatible with the new feature vector API.
2017-10-17 22:47:20 -07:00
Jim Posen
1633ab180f lnwire: New API for feature vectors.
This is a rewrite of feature vectors in lnwire. This has a few
benefits:

1) a simpler interface
2) separate structs for a plain set of feature bits and a feature
vector with associated feature names
their respective feature sets
3) loosened requirements that bits MUST be assigned in pairs
4) fix endianness of encoding/decoding
2017-10-17 22:47:20 -07:00
Olaoluwa Osuntokun
b4273d1eaa
routing: abstract path finding behind mission control
In this commit, from the PoV of the SendPayment method we now delegate
all path finding+verification to missionControl. This change doesn’t
materially affect anything, it simply expands the abstraction to make
way for future features that more heavily utilize mission control.
2017-10-17 19:44:17 -07:00
Olaoluwa Osuntokun
dff3ad05f0
config: add sanity check to ensure either BTC or LTC is specified 2017-10-17 19:44:15 -07:00
Olaoluwa Osuntokun
f953f94f71
peer: display reversed bytes of chain hash in message summaries 2017-10-17 19:44:15 -07:00
Olaoluwa Osuntokun
dc124baca1
peer: if cooperative closing transaction is rejected, return err to caller 2017-10-17 19:44:14 -07:00
Olaoluwa Osuntokun
03f9479eb4
discovery: only interpret remote announcements as pre-mature
This commit fixes a slight bug in the announcement processing logic
within the AuthenticatedGossiper. Before this commit, it was possible
for us to ignore one of our now announcements due to it being
pre-mature, rendering (atm) the channel unusable by the ChannelRouter
itself. To fix this, we know only check for a pre mature announcement
iff the message is coming from a remote node.
2017-10-17 19:44:13 -07:00
Olaoluwa Osuntokun
f3b8c912b4
test: bump up logging level for integration test nodes to debug
In light of the recent commit which added a new set of logging in the
debug verbosity and the commit which added exporting logs from test
execution to Travis, we’ve now bumped up the logging level to debug.
With this new set of logging infrastructure, we should be able to
quickly debug any issues or flakes in the integration tests running on
CI.
2017-10-17 19:44:12 -07:00
Olaoluwa Osuntokun
b7e193354a
peer: add message summaries of sent/recv'd message for debug log level
This commit fills in an existing logging gap by adding a new set of
message summaries that is shown for the debug logging level.

Before this commit, if a user wanted to get a close up feel for what
lnd was doing under the covers, they had to use the trace logging
level. Trace can be very verbose, so we now provide a debug logging
level with message “summaries”. The summaries may not contain all the
data in the message, hut have been crafted in order to provide
sufficient detail at a glance.
2017-10-17 19:44:12 -07:00
Sam Lewis
dd90a724e5 test: add upload of integration logs from TravisCI builds
This commit adds adds: 

- Functionality to `gotest.sh` to log the output of `lnd` instances used for
  integration testing to file.

- Uploading of those log files to [termbin.com ](http://www.termbin.com) from
  TravisCI integration builds.

As an example of a build with this enabled, you can look at [this
build](https://travis-ci.org/samvrlewis/lnd/builds/286942025), which is a build
from my `ci_logging_test` branch. That branch has the same diff as my
`ci_logging` branch but with a small inclusion to allow TravisCI builds to work
(as I described in [my
comment](https://github.com/lightningnetwork/lnd/issues/302#issuecomment-335133834)
the other day). If you scroll to the end of the job logs where `RACE=false`
you'll see a `after_script` section with an output like this:

```
output0.log uploaded to http://termbin.com/aloqr
output1.log uploaded to http://termbin.com/3ggh
output2.log uploaded to http://termbin.com/abb9
output3.log uploaded to http://termbin.com/rk9j
output4.log uploaded to http://termbin.com/956p
output5.log uploaded to http://termbin.com/iwwt
```

Fixes #302.
2017-10-17 15:42:22 -07:00
Olaoluwa Osuntokun
209fb98d0f
routing: properly use vertexDecay for vertexes in missionControl 2017-10-16 20:31:32 -07:00
Olaoluwa Osuntokun
f555f29c7f
build: update glide files to point to latest btcd
This commit updates the glide files to point to the latest btcd build
which has had stall detection temporary disabled in order to ensure
smooth neutrino syncs without the false positives currently present in
stall detection.
2017-10-16 20:30:49 -07:00
Olaoluwa Osuntokun
bbb34cebe0
routing: modify the TestSendPaymentRouteFailureFallback to clear missionControl between attempts
In order to maintain the original essence of the test, we need to clear
the state of missionControl with each attempt, essentially advancing
time between each payment attempt.
2017-10-16 19:07:40 -07:00
Olaoluwa Osuntokun
8ef829ed80
routing: modify SendPayment loop to be lazy, iterative, and use missionControl
In this commit we modify the SendPayment loop to optimize for
time-to-first-payment-success-or-failure. The prior logic would first
attempt to find at least 100 routes to the destination, then
iteratively prune them away as errors were encountered. In this commit,
we modify this approach to instead take a lazy approach: we first find
the current “best” path, attempt to send to that, and if an error
occurs we prune a section of the graph by reporting to missionControl,
then continue.

With this new approach, if the first known path has sufficient
capacity, and is available, then the payment speed is greatly improved
from the PoV of users. Additionally, we avoid the excessive computation
of crawling most of the graph in the k-shortest paths loop. With the
decay on missionControl, all routes will now feed information into the
central knowledge hung, allowing all payments to iteratively find out
the inactive portions of the payment graph.
2017-10-16 19:05:47 -07:00
Olaoluwa Osuntokun
276f2e467b
routing: end path finding on an additional set of critical-ish errors 2017-10-16 18:58:35 -07:00
Olaoluwa Osuntokun
e06177e55c
routing: introduce new missionControl system within ChannelRouter
This commit adds a new system within the ChannelRouter: missionControl.
The purpose of this system to is to act as a shared memory of sorts
between payment sending attempts, recording which edges/vertexes word
or didn’t work. Allowing execution attempts to pass on their iterative
knowledge of the graph to later attempts will reduce the number of
failures encountered, and generally lead to a better UX when sending
payments.

The current capabilities of missionControl are rather limited just to
introduce the new abstraction. Later follow up commits will also add
preferential treatment for reliable nodes, knowledge the impact that
target payments have on unbalancing the payment graph, etc.
2017-10-16 18:57:36 -07:00
Olaoluwa Osuntokun
ae6bde2d77
routing: avoid internal bolt db deadlock by reusing transaction in findPath
This commit fixes a bug that could lead to a deadlock inside bolt db
itself. In a recent commit we allowed a db transaction to be passed
directly into findPath, however, the initial call to graph.ForEachNode
instead passed a _nil_ transaction causing the method itself to create
a _new_ transaction, leading to a deadlock.

We fix this issue by instead re-using the transaction pointer.
2017-10-16 18:48:27 -07:00
Olaoluwa Osuntokun
0692d2d408
peer: log type of received unknown message 2017-10-16 18:43:19 -07:00
Olaoluwa Osuntokun
65c03c98d0
htlcswitch: always return an instance of ForwardingError from handleLocalDispatch
This commit modifies the errors that we return within the
handleLocalDispatch method. Rather than returning a regular error, or
simply the matching error code in some instances, we now _always_
return an instance of ForwardingError. This will allow the router to
make more intelligent decisions w.r.t routing HTLC’s as with this
information it will now be able to differentiate errors that occur
within the switch (before sending out the HTLC), from errors that occur
within the HTLC route itself.
2017-10-16 18:42:16 -07:00
Olaoluwa Osuntokun
61be23dc31
htlcswitch+server: add new field SelfKey to htlcswitch.Config
This commit adds a new field to the switch’s Config, namely the public
key of the backing lightning node. This field will soon be used to
return more detailed errors messages back to the ChannelRouter itself.
2017-10-16 18:39:17 -07:00
Olaoluwa Osuntokun
51d04e8922
htlcswitch: add new ExtraMsg field to ForwardingError
This commit adds a new field to the ForwardingError struct: ExtraMsg.
The purpose of this field is to allow the htlcswitch to tack on
additional error context to ForwardingError messages returned to the L3
router.
2017-10-16 18:15:51 -07:00
Olaoluwa Osuntokun
b29a73a0dd
routing: don't prune our own channels during zombie channel collection
This commit is a precautionary commit which ensure that we don’t
attempt to prune our _own_ channels during zombie channel collection.
2017-10-16 18:14:01 -07:00
Olaoluwa Osuntokun
9b0b945a3d
Revert "server: BroadcastMessage will no longer block until completion"
This reverts commit 6db90ef09ab974df0ff09aeaf75a3d80414d4f50.

The root cause was fixed by commit
f4e7c36c80e8e2be9edb78b3b317c69d28d6d78f. As a result, this commit is no
longer needed.
2017-10-16 14:54:30 -07:00
Olaoluwa Osuntokun
7300452225
Revert "server: make sendToPeer async"
This reverts commit b7704e2de3dc96a4aebc2f47908a8f9def1da7f6.

The root issue was fixed by commit
f4e7c36c80e8e2be9edb78b3b317c69d28d6d78f. As a result, this commit is no
longer needed.
2017-10-16 14:53:24 -07:00
Olaoluwa Osuntokun
f4e7c36c80
peer: eliminate excessive queueHandler spinning w/ blocked writeHandler
This commit fixes an existing bug within the iteration between the
queueHandler and the writeHandler. Under certain scenarios, if the
writeHandler was blocked for a non negligible period of time, then the
queueHandler would enter a very tight spinning loop. This was due to
the fact that the break statement in the inner select loop of the
queueHandler wouldn’t actually break the inner for loop, instead it
would cause the execution logic to re-enter that same select loop,
causing a very tight spin.

In this commit, we fix the issue by adding to things: we now label the
inner select loop so we can break out of it if we detect that the
writeHandler has blocked. Secondly, we introduce a new channel between
the queueHandler and the writeHandler to signal the queueHandler that
the writeHandler has finished processing the last message.
2017-10-15 15:19:57 -07:00
Olaoluwa Osuntokun
cbdf139696
peer: add an idle timer to the readHandler
In this commit, we add an idle timer to the readHandler itself. This
will serve to slowly prune away inactive TCP connections as a result of
remote peer being blocked either upon reading or writing to the socket.
Our ping timer interval is 1 minute, so an idle timer interval of 5
minutes seem reasonable.
2017-10-15 15:14:35 -07:00
Olaoluwa Osuntokun
7a2ce62346
config+lnd: add new option --cpuprofile for writing cpuprofile to file
In this commit we add a new option to lnd, cpuprofile. With this option
we add additional telemetry hooks into and, allowing users to generate
CPU profiling files to measure hot spots within the daemon.
2017-10-14 17:08:34 -07:00
Olaoluwa Osuntokun
b7704e2de3
server: make sendToPeer async
This commit removes another case of unnecessary blockage, by modifying
the sendToPeer method to be fully asynchronous. From the PoV of the
callers that utilize this method currently, there’s no reason to block
until the completion of this method. Additionally, as the graph grows
larger without more intelligent the number of messages sent during
initial dump will start to be prohibitive to waiting for full
completion before proceeding.
2017-10-14 17:07:08 -07:00
Olaoluwa Osuntokun
6db90ef09a
server: BroadcastMessage will no longer block until completion
In this commit, we make the BroadcastMessage method on the server more
asynchronous by abandoning the two wait groups that it used for
synchronization. It has been observed that a circular waiting loop
between the AuthenticatedGossiper and a peer’s readHandler can cause
the system to dead lock.

By removing this unnecessary synchronization, we avoid the deadlock
case and allow the gossiper itself to no longer block in this scenario.
2017-10-14 16:04:06 -07:00
Conner Fromknecht
7aa64d58da server: improves readibility of peer connection logic 2017-10-13 16:12:31 -07:00
Conner Fromknecht
96ff63d219 server: fixes race condition during unexpected peer disconnect 2017-10-13 16:12:31 -07:00
Olaoluwa Osuntokun
f20cb89982
build: update lock file for glide 2017-10-10 22:37:17 -07:00
Olaoluwa Osuntokun
3deb94442f
build: update glide to point to latest sphinx impl 2017-10-10 22:30:50 -07:00
Olaoluwa Osuntokun
458e576af7
build: update glide to latest neutrino w/ re-org fixes 2017-10-10 22:19:31 -07:00
Olaoluwa Osuntokun
2eaa9fd683
htlcswitch: fix linter error 2017-10-10 22:19:29 -07:00
Olaoluwa Osuntokun
51b072c4b5
routing: return proper error if encounter non ForwardingError in SendPayment 2017-10-10 22:19:28 -07:00
Olaoluwa Osuntokun
03d8399d0a
build: update glide files to point to latest neutrino+btcwallet 2017-10-10 22:19:28 -07:00
Olaoluwa Osuntokun
646f79f566
routing: perform path finding inside a single DB transaction
This commit modifies the path finding logic such that all path finding
is done inside a _single_ database transaction. With this change, we
ensure that we don’t end up possibly creating hundreds of database
transactions slowing down the path finding and payment sending process
all together.
2017-10-10 22:19:27 -07:00
Olaoluwa Osuntokun
5eb3406b19
channeldb: add new Database method to ChannelGraph 2017-10-10 22:19:26 -07:00
Olaoluwa Osuntokun
eb7b5b342e
routing: add a basic test to exercise route pruning in response to errors 2017-10-10 22:19:25 -07:00
Olaoluwa Osuntokun
ce7179a468
routing: add basic route pruning in response to HTLC onion errors
This commit adds basic route pruning in response to HTLC onion errors.
With this new change, the router will now prune routes in response to
HTLC errors, which will reduce the time to payment success, and also
avoid a bunch of unnecessary network traffic.

We now respond to two errors lnwire.FailTemporaryChannelFailure and
lnwire.FailUnknownNextPeer. In response to the first error, we’ll prune
all routes that contain the channel which was unable to be routed over.
In response to the second error we’ll prune all routes that contain the
node which couldn’t be found.
2017-10-10 22:19:25 -07:00
Olaoluwa Osuntokun
8d7f3943bb
routing: add two new methods to filter routes based on node/channel 2017-10-10 22:19:24 -07:00
Olaoluwa Osuntokun
f6ac31281b
routing: also include the source node in the nextHopMap index
In this commit we modify the newRoute function to also add the source
node to the nextHopMap index. With this addition the indexes will now
allow the router to react based on failures that occur during the
_first_ hop, meaning the channel directly attached to the source node.
2017-10-10 22:19:23 -07:00
Olaoluwa Osuntokun
70e114fa6f
routing: add additional indexes to the Route struct to allow for querying
This commit adds three new indexes to the Route struct. These indexes
allow a caller to check if a channel is in the route, check if a node
is in the route, query the next node after a target node, and query the
next channel after a target node. The combination of these new indexes
will allow the ChannelRouter to prune away routes from the available
set in response to any received errors.
2017-10-10 22:19:22 -07:00
Olaoluwa Osuntokun
7ba6b7fa09
routing: add a String() method to the vertex type 2017-10-10 22:19:22 -07:00
Olaoluwa Osuntokun
63838f5764
peer+server: use new sphinx package and htlcswitch package type names 2017-10-10 22:19:21 -07:00
Olaoluwa Osuntokun
a7d846ac40
lnwire: add godoc comments for lnwire.FailureMessage 2017-10-10 22:19:20 -07:00
Olaoluwa Osuntokun
93c008bb46
htlcswitch: update tests to expect new ForwardingError type 2017-10-10 22:19:19 -07:00
Olaoluwa Osuntokun
12ae63101d
Htlcswitch: rename Deobfuscator and Obfuscator interfaces
This commit renames the Deobfuscator interface to ErrorDecrypter and
the Obfuscator interface to ErrorEncrypter. With this rename, the
purpose of these two interfaces are a bit clearer.

Additionally, DecryptError (which was formerly Deobfuscate) now
directly returns an ForwardingError type instead of the
lnwire.FailureMessage.
2017-10-10 22:19:19 -07:00