Commit Graph

24 Commits

Author SHA1 Message Date
Conner Fromknecht
4599b0eac3
autopilot/agent: track pending connections
This commit modifies the autopilot agent to track
all pending connection requests, and forgo further
attempts if a connection is already present.
Previously, the agent would try and establish
hundreds of requests to a node, especially if the
connections were timing out and not returning.

This resulted in an OOM OMM when cranking up
maxchannels to 200, since there would be close
to 10k pending connections before the program was
terminated. The issue was compounded by periodic
batch timeouts, causing autopilot to try and
process thousands of triggers for failing
connections to the same peer.

With these fixes, autopilot will skip nodes that we
are trying to connect to during heuristic selection.
The CPU and memory utilization have been significantly
reduced as a result.
2018-09-05 20:10:34 -07:00
Johan T. Halseth
0d4df54118
autopilot/agent: signal balanceUpdates on own channel 2018-09-04 10:18:15 +02:00
Johan T. Halseth
a9a9c9aeb4
autopilot/agent: signal chanPendingOpenUpdates on own channel 2018-09-04 10:17:59 +02:00
Johan T. Halseth
186e6d4da4
autopilot/agent: signal chanOpenFailureUpdates on own channel
We do this to avoid a huge amount of goroutines piling up when autopilot
is trying to open many channels, as they will all block trying to send
the update on the stateUpdates channel. Now we instead send them on a
buffered channel, similar to what is done with the nodeUpdates.
2018-09-04 10:17:33 +02:00
Johan T. Halseth
4a88c61a90
autopilot/agent: signal nodeUpdates on own channel
We do this to avoid a huge amount of goroutines piling up on initial
graph sync, as they will all block trying to send the node update on the
stateUpdates channel. Now we instead make a new buffered channel
nodeUpdates, and just return immediately if there is already a signal in
the channel waiting to be processed.
2018-09-04 10:17:09 +02:00
Johan T. Halseth
3e992f094d
autopilot/agent: move signal processing out of select 2018-09-04 10:06:15 +02:00
Wilmer Paulino
e1a376d9f8
autopilot: use updateBalance rather than tracking balance explicitly
In this commit, we modify the balanceUpdate autopilot signal to update
the balance according to what's returned to the WalletBalance callback
rather than explicitly tracking the balance. This gives the agent a
better sense of what the wallet's balance actually is.
2018-08-29 02:06:02 -07:00
Conner Fromknecht
602856750b
autopilot/agent: add OnNodeUpdates signal
Adds a new external signal alerting autopilot that
new nodes have been added to the channel graph or
an existing node has modified its channel
announcment. This allows autopilot to examine its
current state, and attempt to open channels if our
target state is not yet met.
2018-08-23 18:58:12 -07:00
Wilmer Paulino
aa7c2e6d47
autopilot: add OnChannelPendingOpen state update 2018-08-16 20:22:40 -07:00
Wilmer Paulino
dd5b6394d9
autopilot: read quit chan when sending state updates 2018-08-16 20:19:32 -07:00
Wilmer Paulino
454b549c7e
autopilot+pilot: refactor connection logic out of OpenChannel
In this commit, we refactor the existing connection logic outside of the
ChanController's OpenChannel method. We do this as previously it was
possible for peers to stall us while attempting to connect to them. In
order to remedy this, we now attempt to connect the peer before tracking
them in our set of pending opens.
2018-08-16 20:17:03 -07:00
Conner Fromknecht
1ded697e8d
multi: sort import paths with gofmt 2018-08-02 18:20:49 -07:00
Olaoluwa Osuntokun
6f60f139f4 multi: switch over import paths from roasbeef/* to btcsuite/* 2018-07-13 17:05:39 -07:00
Olaoluwa Osuntokun
00ab66469d
autopilot: refresh available funds on channel open/close events 2018-03-10 16:53:02 -08:00
Olaoluwa Osuntokun
ce2d5a2156
autopilot: limit the number of outstanding channel open goroutines
In this commit, we fix an existing bug that would at times cause us to
spiral out of control and potentially created thousands of outbound
connections. Our fix is simple: limit the total number of outstanding
channel establishment attempts. Without this limit, we have no way to
bound the number of active goroutines.

Fixes #772.
2018-03-10 16:53:02 -08:00
Olaoluwa Osuntokun
9f52372cd2
autopilot: modify interfaces to specify *exactly* how many chans to open
In this commit, we fix a regression introduced by a recent change which
would allow the agent to detect a channel as failed, and blacklist the
node, promising faster convergence with the ideal state of the
heuristic.

The source of this bug is that we would use the set of blacklisted
nodes in order to compute how many additional channels we should open.
If 10 failures happened, then we would think that we had opened up 10
channels, and stop much earlier than we actually should.

To fix this, while ensuring we don’t retry to failed peers, the
NeedMoreChans method will now return *how* anymore channels to open,
and the Select method will take in how many channels it should try to
open *exactly*.
2018-02-08 20:08:18 -08:00
MeshCollider
61414df669 multi: Fix two minor typos 2018-02-08 15:21:04 -08:00
Daniel McNally
8543497dcc multi: fixing it's/its typos in comments 2018-02-06 19:13:07 -08:00
Simon Horlick
73468cf06e autopilot: Exclude nodes that previously failed 2018-01-27 18:35:31 -08:00
Simon Horlick
a43d7d3532 autopilot: Make autopilot aware of channel open failures 2018-01-27 18:35:31 -08:00
Olaoluwa Osuntokun
2e2b262999
autopilot: ensure access to pendingOpens in thread-safe when logging 2017-12-10 15:46:49 -08:00
Olaoluwa Osuntokun
804edb8f02
autopilot: ensure all access to pendingOpens is thread-safe
In this commit, we ensure that we grab the mutex for the pending open
channel state when we attempt to merge the pending state with the
committed state.
2017-08-22 00:51:33 -07:00
Olaoluwa Osuntokun
711b695a2f
autopilot: add tracking of pending channel state
This commit adds tracking of the pending channels state within the
autopilot.Agent. This fixes a class of bugs which was discovered during
the latest test net block storm wherein the Agent would attempt to
repeatedly attach to the same node due to rapid closure of other
channels.

In this commit we fix this issue by ensuring that we always factor in
the pending channel state when querying the heuristic w.r.t if we need
more channels, and if so to which nodes should be attached to.
2017-08-15 18:23:58 -07:00
Olaoluwa Osuntokun
306c4aef8e
autopilot: "Look ma no hands!", introducing autopilot mode
This commit introduces the initial implementation of the autopilot
mode. Autopilot is new mode within lnd that enables automatic channel
management. This means that if enabled lnd will attempt to
automatically manage channels according to a set of heuristic defined
within the main configuration for autopilot.Agent instance.

The autopilot.Agent implements a simple closed control loop. It takes
in external signals such as wallet balance updates, new open channel,
and channels that are now closed the updates its internal state. With
each external trigger it will consult the registered
AttachmentHeuristic to decide: if it needs to open any more channels,
and if so how much it should use to open the channels, ultimately
returning a set of recommended AttachmentDirectives. The
autopilot.Agent loop will then take those attempt to establish
connection, and go back in waiting for a new external signal.

With this first implementation the default heuristic is the
ConstrainedPrefAttachment implementation of AttachmentHeuristic. Given
a min and max channel size, a limit on the number of channels, and the
percentage of wallet funds to allocate to channels, it will attempt to
execute a heuristic drive by the Barabási–Albert model model in order
to attempt to drive the global graph towards a scale free topology.

This is commit implements a foundational layer for future simulations,
optimization, and additional heuristics.
2017-08-10 21:14:55 -07:00