In this commit, we modify the msgStream struct to ensure that it has a
cap at which it’ll continue to buffer messages. Currently we have two
msgStream structs per peer: the first for the discovery messages, and
the second for any messages that modify channel state. Due to
inefficiencies in the current protocol for reconciling graph state upon
connection (just dump the entire damn thing), when a node first starts
up, this can lead to very high memory usage as all peers will
concurrently send their initial message dump which can be in the
thousands of messages on testate.
Our fix is simple: make the message stream into a _bounded_ message
stream. The newMsgStream function now has a new argument: bufSize.
Internally, we’ll take this bufSize and create more or less an internal
semaphore for the producer. Each time the producer gets a new message,
it’ll try and read an item from the channel. If the queue still has
size, then this will succeed immediately. If not, then we’ll block
until the consumer actually finishes processing a message and then
signals by sending a new item into the channel.
We choose an initial value of 1000. This was chosen as there’s already
a max limit of outstanding adds on the commitment, and a value of 1000
should allow any incoming messages to be safely flushed and processed
by the gossiper.
This commit extends the amount of time we wait
for transaction to enter the mempool from
10 to 30 seconds. The wallet's interface tests
seem to be particularly slow when run with the
race flag, a problem which is only exacerbated
by the slowness of travis.
With 10s and the race flag, I was able to repro
the issues locally fairly consistently.
`lncli` command `walletbalance` apparently no longer supports the `--witness_only` argument. With the deprecated argument:
```
root@8a9ad8528430:/go/src/github.com/lightningnetwork/lnd# lncli walletbalance --witness_only=true
Incorrect Usage: flag provided but not defined: -witness_only
NAME:
lncli walletbalance - Compute and display the wallet's current balance
USAGE:
lncli walletbalance [arguments...]
[lncli] flag provided but not defined: -witness_only
```
After removing that argument:
```
root@8a9ad8528430:/go/src/github.com/lightningnetwork/lnd# lncli walletbalance
{
"total_balance": "1505000000000",
"confirmed_balance": "1505000000000",
"unconfirmed_balance": "0"
}
```
* Update go version for Docker `lnd` build
Per #815, https://github.com/lightningnetwork/lnd/blob/master/htlcswitch/circuit.go#L21 depends on go version >= 1.9
Before this change:
```
Building lnd
Step 1/11 : FROM golang:1.8
...
Step 9/11 : RUN go install . ./cmd/...
---> Running in 7f5d6ac430e5
# github.com/lightningnetwork/lnd/htlcswitch
htlcswitch/circuit.go:21: syntax error: unexpected = in type declaration
ERROR: Service 'lnd' failed to build: The command '/bin/sh -c go install . ./cmd/...' returned a non-zero code: 2
```
After:
```
Building lnd
Step 1/11 : FROM golang:1.9
1.9: Pulling from library/golang
3e731ddb7fc9: Pull complete
47cafa6a79d0: Pull complete
79fcf5a213c7: Pull complete
68e99216b7ad: Pull complete
9fc7abc9c77e: Pull complete
7fefae002ae5: Pull complete
7eef770e7cf3: Pull complete
fea064ba669b: Pull complete
Digest: sha256:caa4d6b6a468ffc4cc1f2d88cd95019bf057eb33bea1407dd55bd79755a26bb4
Status: Downloaded newer image for golang:1.9
---> 8ebf49f75a15
...
Step 9/11 : RUN go install . ./cmd/...
---> Running in ba871e58d517
Removing intermediate container ba871e58d517
---> fe16f8578770
Step 10/11 : COPY "docker/lnd/start-lnd.sh" .
...
---> 1f1d47432540
Successfully built 1f1d47432540
Successfully tagged lnd:latest
...
```
* Use latest go version 1.10
Pre review feedback, use the latest version of go.
This commit alters the behavior of the fundingmanager to
rebroadcast the funding transaction of all pending-open
channels upon restart. This is applied only to single-funder
channels for which we are the initiator, and helps ensure that
funding txns do not get stuck in the face of failures or restarts.
In this commit, we fix a quick in the main autopilot handling. If we’re
unable to establish a channel to a peer for w/e reason, then we’ll
disconnect the peer. With this tactic, we avoid unnecessarily building
up tens of peers for no reason at all.
In this commit, we fix an existing bug that would at times cause us to
spiral out of control and potentially created thousands of outbound
connections. Our fix is simple: limit the total number of outstanding
channel establishment attempts. Without this limit, we have no way to
bound the number of active goroutines.
Fixes#772.
In this commit, we update the pinned versions of lightning-onion as
well as btcwallet.
The newest lightning-onion instance adds some logging statements when
garbage collecting expired shared secret hashes.
The new btcwallet version fixes a recently introduced bug which would
cause us to remove all unman transactions on restart if the full node
new of any of them. Instead, we’ll now only remove these unmanned
transactions if it’s reported that the transactions were double spends,
or orphans.
In this commit we modify the storage location of the sphinx replay
database to be under the precise network, and not only the graph sub
directory. Before this commit, due to the usage of filepath.Dir(), the
db would lie under /graph/, rather than say, /graph/simnet.
// 1. Carol --> Dave --> Alice --> Bob forward payment
// 2. Carol --- Dave X Alice --- Bob disconnect intermediaries
// 3. Carol --- Dave X Alice <-- Bob settle last hop
// 4. Carol --- Dave X X shutdown Bob, restart Alice
// 5. Carol <-- Dave <-- Alice X expect settle to propagate
