Commit Graph

118 Commits

Author SHA1 Message Date
Johan T. Halseth
b71f4632a6
autopilot/agent_test: define testCtx, setup, other helpers
This commit defines a set of helper methods that are used by many of the existing tests.
2019-03-14 08:49:28 +01:00
Johan T. Halseth
4a1e06b204
autopilot/agent: distribute available funds among channels
This commit fixes a regression in how we allocate funds to attempted
channels. We would earlier stay within the channel size limits, but we
wouldn't account for funds consumed by other channels being opened in
parallel.

We fix this by introducing a loop which greadily tries to distribute the
funds among the channels to open, and reduces the number of channels to
open in case not enough funds are available to satisfy the channel size
limits.
2019-03-14 08:49:28 +01:00
Johan T. Halseth
25f1252019
autopilot/externalscoreattach_test: add TestSetNodeScores 2019-02-14 12:37:25 +01:00
Johan T. Halseth
5dabb1ae29
autopilot+lnrpc: wire up SetNodeScores RPC to set scores of agent 2019-02-14 11:41:47 +01:00
Johan T. Halseth
b23e53ea33
autopilot/interface+externalscoreattach: define ScoreSettable
ScoreSettable is an interface that let caller set external scores for
the heuristic. The ExternalScoreAttachment and WeightedCombAttachment
heuristics implement this interface.
2019-02-14 11:41:47 +01:00
Johan T. Halseth
749d9cccca
autopilot/combinedattach: add SetNodeScores
This commit adds a method SetNodesScores to the WeightedCombAttachment
heuristic.

Since the heuristic keeps a list of sub-heuristics, it will attempt to
recursively apply the scores to the sub heuristics.
2019-02-14 11:37:47 +01:00
Johan T. Halseth
dff61facf2
autopilot: add ExternalScoreAttachment heuristic
This commit adds a new autopilot heuristic that is scoring based. It is
a simple heuristic that will keep a list of pubkeys and scores, and will
try opening channels with the nodes with the largest score first.
2019-02-14 11:37:47 +01:00
Johan T. Halseth
49a85b2eb3
autopilot/agent_test: add TestAgentQuitWhenPendingConns
TestAgentQuitWhenPendingConns tests that we are able to stop the autopilot
agent even though there are pending connections to nodes.
2019-02-05 10:45:53 +01:00
Johan T. Halseth
c19763c03f
autopilot/agent: don't block Stop on pending connetion
To ensure a call to ConnectToPeer doesn't block the agent from
shutting down, we'll launch it in a non-waitgrouped goroutine, that
will signal when a result is returned.
2019-02-05 10:45:53 +01:00
Johan T. Halseth
8f54a2bd6f
autopilot/manager: add QueryHeuristics
Proxies a query request to the active agent.
2019-01-23 09:05:43 +01:00
Johan T. Halseth
6c556fd92a
autopilot/agent: add queryHeuristics
This commit adds a method queryHeuristic that will query all simple
heuristics for node scores, in addition to the current active heuristic.
2019-01-23 09:05:43 +01:00
Johan T. Halseth
cf01879d96
autopilot/agent: protect chanState with mutex
Since we want to access the current channel state outside the main
autopilot agent goroutine, we make sure it is protected by a mutex.
2019-01-23 09:05:42 +01:00
Johan T. Halseth
36e7694e8e
config+pilot: define autopilot.heuristic
And validate the config set by the user.
2019-01-23 09:05:42 +01:00
Johan T. Halseth
c0fed861d2
autopilot/interface: add Name() to AttachmentHeuristic interface 2019-01-23 09:05:42 +01:00
Olaoluwa Osuntokun
cebc4d8dba
Merge pull request #2405 from halseth/autopilot-weighted-heuristics-follow-up
[autopilot] Weighted combined attachment heuristic
2019-01-22 19:08:46 -08:00
Valentine Wallace
f8e588e2e3
autopilot/graph: set max htlc in test edge policies
Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:29 +01:00
Valentine Wallace
0fd6004958
multi: partition lnwire.ChanUpdateFlag into ChannelFlags and MessageFlags
In this commit:

* we partition lnwire.ChanUpdateFlag into two (ChanUpdateChanFlags and
ChanUpdateMsgFlags), from a uint16 to a pair of uint8's

* we rename the ChannelUpdate.Flags to ChannelFlags and add an
additional MessageFlags field, which will be used to indicate the
presence of the optional field HtlcMaximumMsat within the ChannelUpdate.

* we partition ChannelEdgePolicy.Flags into message and channel flags.
This change corresponds to the partitioning of the ChannelUpdate's Flags
field into MessageFlags and ChannelFlags.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:26 +01:00
Johan T. Halseth
bdbadeaa75
autopilot: add WeightedCombAttachment
This commit defines a new heuristic WeightedCombAttachment that takes a
set of sub-heuristics, and produces a final node score by querying the
sub-heuristics and combining the scores from them according to their
weights.

This way it will look like a regular, single heuristic to the autopilot
agemnt, but can be a more complex combination of several.
2019-01-21 09:44:38 +01:00
Johan T. Halseth
592ce92c72
autopilot/interface+prefattach: scale node scores to range [0.0, 1.0]
To prepare for combinning scores from multiple heuristics, we require the
scores returned from the NodeSores API to be in the range [0.0, 1.0].

The prefAttach heuristic is altered to scale the returned scores such
that the most connected node in the grpah is given a score of 1.0.
2019-01-21 09:44:37 +01:00
Johan T. Halseth
5b1e72a019
autopilot/interface+agent: return NodeScore from NodeScores
Since NodeScores no longer returns fully populated AttachmentDirectives,
we make this explicit by defining a new type NodeScore that includes a
subset of what the AttachmentDirective does.
2019-01-08 10:23:50 +01:00
Johan T. Halseth
25de66d27b
autopilot/interface+choice: add NodeScore type
We create a new type NodeScore which is a tuple (NodeID, score). The
weightedChoice and chooseN algorithms are altered to expect this type.

This is done in order to simplify the types we are using, since we were
only using a subset of the fields in AttachmentDirective.
2019-01-08 10:23:48 +01:00
Johan T. Halseth
3739c19ef8
autopilot/pref_attachment: rename ConstrainedPrefAttachment->PrefAttachment
Since the ConstrainedPrefAttachment no longers require the heuristic to
be aware of the autopilot constraints, we rename it PrefAttachment.
2019-01-08 10:10:59 +01:00
Johan T. Halseth
ccf4b7feab
autopilot: move address lookup from heuristic to agent
To avoid having the heuristics deal with (possibly conflicting) address
lookups, we let the agent handle them.
2019-01-08 10:10:59 +01:00
Johan T. Halseth
cfd237bf1f
autopilot: move determining chanSize from heuristic to agent
Since we want to combine scores from multiple heuristics, things get
complicated if the heuristics report their own individual channel sizes.
Therefore we change the NodeScores interface slightly, letting the agent
specify the wanted channel size, and let the heuristic score the nodes
accordingly.
2019-01-08 10:10:59 +01:00
Johan T. Halseth
d5f3714f86
autopilot/agent: return early if no funds available
If there are less funds available than the minumum channel size, there's
no reason to score candidates, so we continue early.
2019-01-08 10:10:59 +01:00
Johan T. Halseth
d0c4e253c6
autopilot/interface: remove NeedMoreChans from AttachmentHeuristic
Since the agent is now querying the constraints directly, NeedMoreChans
is no longer needed on the heuristic.
2019-01-08 10:10:59 +01:00
Johan T. Halseth
fbfc9a53af
autopilot: TestConstrainedPrefAttachmentNeedMoreChan->TestConstraintsChannelBudget
Since the constraints are no longer something the heuristic needs to be
aware of, we move the test.
2019-01-08 10:10:58 +01:00
Johan T. Halseth
0e1713956b
autopilot/agent: call ChannelBudget on constrainsts
We let the agent call ChannelBudget on its constraints directly, and
not go through the heuristic. This is needed since when we want to have
multiple active heuristics concurrently, it won't make sense anymore to
ask each of the heuristics.

The mockConstraints are also updated to act as the mockHeuristic did
before, by making it possible to control the responses it gives by
sending them on the contained channels.
2019-01-08 10:10:58 +01:00
Johan T. Halseth
1d82e12fcf
autopilot: define AgentConstraints
To decouple the autopilot heuristic from the constraints, we start by
abstracting them behind an interface to make them easier to mock. We
also rename them HeuristicConstraints->AgentConstraints to make it clear
that they are now constraints the agent must adhere to.
2019-01-08 10:10:58 +01:00
Olaoluwa Osuntokun
21460c9e67
Merge pull request #2293 from halseth/unit-tests-weighted-choice
[autopilot] weighted choice unit tests and optimizations
2019-01-07 19:42:29 -08:00
Johan T. Halseth
dbf7b380a9
autopilot/manager: only set m.pilot if started successfully
This commit fixes a subtle bug within the autopilot manager, that would
cause the active pilot to not be reset in case it wasn't started
successfully.

We also make sure the associated goroutines close over the started
pilot, and not the active pilot.
2018-12-19 19:48:44 +01:00
Johan T. Halseth
902d6edad2
autopilot/choice: add weightedChoice tests 2018-12-19 08:27:35 +01:00
Johan T. Halseth
4ac3c171ec
autopilot/choice: avoid costly map allocations
This commit makes the weightedChoice algorithm take a slice of weights
instead of a map of node scores. This let us avoid costly map allocation
and iteration.

In addition we make the chooseN algorithm keep track of the remaining
nodes by keeping a slice of weights through its entire run, similarly
avoiding costly map allocation and iteration.

In total this brings the runtime of the TestChooseNSample testcase down
from ~73s to ~3.6s.
2018-12-19 08:27:34 +01:00
Johan T. Halseth
3d2a39a18c
autopilot/choice_test: add chooseN tests 2018-12-19 08:27:34 +01:00
Johan T. Halseth
a202860ce1
autopilot: make n uint32 in chooseN
To avoid negative values being input, as it doesn't really make sense.
2018-12-19 08:27:34 +01:00
Johan T. Halseth
6310ed0f1c
autopilot: add autopilot.Manager
This commit adds a new type Manager responsible for managing an
autopilot agent, making it possible to start and stop it at runtime.
2018-12-13 12:26:29 +01:00
Olaoluwa Osuntokun
d24a6d2a3d
Merge pull request #2304 from halseth/weigthed-choice-errnopositive
[autopilot] Don't fail on no more positive nodes
2018-12-12 12:21:06 -08:00
Johan T. Halseth
8e57ed7025
autopilot/agent_test: let agent self-trigger retry
This ensures that it is done updating its pending connections map before
re-querying the heuristic.
2018-12-11 10:21:40 +01:00
Johan T. Halseth
dda7355ad6
autopilot/agent_test: ensure mochHeuristics exits cleanly 2018-12-11 10:20:38 +01:00
Johan T. Halseth
0255db5120
autopilot/prefattach_test: ensure non-connected nodes get 0-score
This addition to the unit tests makes sure nodes that have no channels
in the graph are left out od the scored nodes, implicitly giving them a
score of 0.
2018-12-10 15:05:24 +01:00
Johan T. Halseth
b9c33da3f8
autopilot/prefattach: skip nodes having no channels 2018-12-10 15:01:43 +01:00
Johan T. Halseth
aeeff4714d
autopilot/graph: define addRandNode 2018-12-10 15:01:43 +01:00
Johan T. Halseth
5224c94aaf
autopilot/prefattach_test: assert scored nodes having addresses
This addtion to the test makes sure the scored nodes have their
addresses populated. This would fail without the previous commit.
2018-12-10 15:01:05 +01:00
Johan T. Halseth
d9eef4625e
autopilot/prefattach: populate Addrs field 2018-12-10 15:00:41 +01:00
Johan T. Halseth
55ba5fe654
autopilot/graph: add addresses to memNode 2018-12-10 14:56:54 +01:00
Johan T. Halseth
40db2dd5a5
autopilot/choice: return ErrNoPositive in case no choice can be made 2018-12-10 13:53:36 +01:00
Johan T. Halseth
f1e8c8d5b5
autopilot/agent: move choice algorithms to new file choice.go 2018-12-10 13:53:31 +01:00
Johan T. Halseth
6130189d95
autopilot/interface+agent: remove Select 2018-12-06 14:26:28 +01:00
Johan T. Halseth
b3d315298c
autopilot/agent: use NodeScores to select channel candidates
This commit makes the autopilot agent use the new NodeScores heuristic
API to select channel candiates, instead of the Select API. The result
will be similar, but instead of selecting a set of nodes to open
channels to, we get a score based results which can later be used
together with other heuristics to choose nodes to open channels to.

This commit also makes the existing autopilot agent tests compatible
with the new NodeScores API.
2018-12-06 14:26:26 +01:00
Johan T. Halseth
e84bd29836
autopilot/agent: add weightedChoice and chooseN algorithm
The algorithms will be used to select nodes at random from the weighted
distribution set by the node's scores given by the heuristic.
2018-12-06 14:26:05 +01:00