Commit Graph

151 Commits

Author SHA1 Message Date
Andras Banki-Horvath
608354032c autopilot: parallelize betweenness centrality
This commit parallelizes betweenness centrality calculation, by
distributing the algo to N workers and creating partial results.
2020-03-27 13:56:10 +01:00
Andras Banki-Horvath
3fe9c70722 autopilot: betweenness centrality using Brandes algo on simplifed graph
This commit adds betweenness centrality to the available node metrics.
Betweenness centrality is a per node centrality measure which for an
arbitrary node v equals to the sum of shortest paths going trough v
divided by the number of all shortest paths for for each vertex pair
k, s where k != s != v.
2020-03-27 10:39:50 +01:00
Andras Banki-Horvath
be83d504f8 autopilot: introduce NodeMetric interface for arbitrary graph metrics
This commit adds the NodeMetric interface which will be used for all
graph metrics not directly part of the autopilot but are useful in
composite heuristics to drive autopilot decisions and improve node
scores.
2020-03-27 10:36:28 +01:00
Olaoluwa Osuntokun
21521ff610
autopilot: convert to use new kvdb abstraction 2020-03-18 19:35:20 -07:00
Joost Jager
6e0cfe579b
Merge pull request #3841 from joostjager/check-max-onion-size
routing: check max routing info size
2019-12-20 14:42:47 +01:00
Joost Jager
95ddab57fa
channeldb: add tx argument for FetchLightningNode
To allow execution within an existing tx.
2019-12-20 10:17:01 +01:00
Andras Banki-Horvath
2c3041d8bc autopilot: fix memChannelGraph channel edge addition
This PR fixes an issue that happens when adding a new channel edge
between two nodes in a memChannelGraph. Originally a channel edge held a
node value which made the graph different when iterating from the two
endpoints of an edge. This is simply fixed by holding pointers instead.
2019-12-19 22:42:37 +01:00
Joost Jager
37d9ee302c
channeldb: take serialized key to fetch lightning node
This prevents inefficient key conversions in a follow up commit that
change the inner pathfinding loop.
2019-12-19 12:44:00 +01:00
Conner Fromknecht
5e27b5022c
multi: remove LocalFeatures and GlobalFeatures 2019-11-08 05:32:00 -08:00
Olaoluwa Osuntokun
083b574fd8
autopilot: fix race condition in new test 2019-11-01 19:30:21 -07:00
Johan T. Halseth
5a8ecfcc49
autopilot/agent_test: add TestAgentHeuristicUpdateSignal
TestAgentHeuristicUpdateSignal tests that upon notification about a
heuristic update, the agent reconsults the heuristic.
2019-10-31 10:19:50 +01:00
Johan T. Halseth
7524265ab1
autopilot: trigger agent channel open on new scores
This commit adds a new signal to the autopilot agent, meant to signal
when any of the available heuristics has gotten an update.

We currently use this to trigger a new channel opening after the
external scores have been updated.
2019-10-31 10:17:31 +01:00
Johan T. Halseth
f0ba4be758
autopilot/manager: fix slice modifications
When appending to a slice, there is no guarantee the slice won't be
modified. So instead of appending to the global slice
availableHeuristics, we create a temporary local one.
2019-10-31 10:17:31 +01:00
Johan T. Halseth
df5baa4275
autopilot/agent: trace log reason nodes being skipped 2019-10-31 10:17:31 +01:00
Joost Jager
3d7de2ad39
multi: remove dead code 2019-09-10 17:21:59 +02:00
Johan T. Halseth
2f1aad257c
autopilot/agent: trace log skipped nodes during scoring 2019-09-02 10:41:00 +02:00
Johan T. Halseth
d5ab95546e
autopilot/combinedattach: trace log scoring process from sub heuristics 2019-09-02 10:40:59 +02:00
Johan T. Halseth
b2560b4ad2
autopilot/externalscoreattach: trace log scoring process 2019-09-02 10:40:59 +02:00
Johan T. Halseth
bad9f4b685
autopilot/prefattach: trace log score process 2019-09-02 10:40:58 +02:00
Johan T. Halseth
2dbc4fe973
autopilot/agent: move expensive graph lookup after channel size check 2019-09-02 10:35:14 +02:00
Johan T. Halseth
7838e6e1fa
autopilot/agent_test: add more agent channel allocation testcases
Add tests for allocation when well within and well above channel budget.
2019-07-18 00:58:12 +02:00
Johan T. Halseth
68f7642564
autopilot/agent_test: wait for expected num chans and balance
Previously we waited only for the number of channels to become what we
expected, but this wasn't enough. Sometimes the agent had't yet updated
its internal balance, causing the test to fail with an unexpected
balance.
2019-07-18 00:58:12 +02:00
Johan T. Halseth
bf7f392c11
autopilot: use subtractFees when funding channel
To make the autopilot able to account for fees, we let it use the
subtractFees option when opening channels.

This makes sure that each channel we attempt to open will eat at most
Amt out of our budget. Previously fees would eat into our funds in
addition, causing us to deplete our funds more than expected on each
channel opening.
2019-07-18 00:58:12 +02:00
Olaoluwa Osuntokun
df40a9c1e1
Merge pull request #3177 from halseth/autopilot-targetconf-conf
[autopilot] make confirmation target configurable
2019-06-17 16:05:00 -07:00
Olaoluwa Osuntokun
8e6b903476
autopilot: demote node update log to debug 2019-06-13 16:39:46 -07:00
Johan T. Halseth
9a7f24d2d3
autopilot: extract default fee estimate target conf into constant 2019-06-08 17:22:34 +02:00
Federico Bond
aea52f4bef multi: replace manual CAS with sync.Once in component start/stop
This guarantees callers that the method will not return until it
has executed completely at least once.
2019-05-30 17:14:04 -03:00
Conner Fromknecht
17b2140cb5
multi: fix spelling mistakes 2019-05-04 15:35:37 -07:00
Johan T. Halseth
9d8e67d81e
autopilot/prefattach: count small channels negatively
Decrease scores of nodes having a large number of small channels.
2019-03-27 15:06:30 +01:00
Johan T. Halseth
6860ad9ac6
autopilot: add Median method 2019-03-27 15:06:30 +01:00
Johan T. Halseth
d4813422c9
autopilot+autopilotrpc: ignore local channels if ignore_local_state set 2019-03-18 14:43:40 +01:00
Johan T. Halseth
dd072304b0
autopilot: move queryHeuristic out of agent
This commit moves the logic querying the available heuristics out of the
autopilot agent and into the autopilot manager. This lets us query the
heuristic without the autopilot agent being active.

If called without the agent being active, the current set of channels
will be considered by the heuristics. If the agent is active also the
pending channels will be considered.
2019-03-18 13:32:18 +01:00
Johan T. Halseth
fb5b6ff425
autopilot/agent_test: add TestAgentChannelSizeAllocation
TestAgentChannelSizeAllocation tests that the autopilot agent opens
channel of size that stays within the channel budget and size
restrictions.
2019-03-14 08:49:28 +01:00
Johan T. Halseth
b71f4632a6
autopilot/agent_test: define testCtx, setup, other helpers
This commit defines a set of helper methods that are used by many of the existing tests.
2019-03-14 08:49:28 +01:00
Johan T. Halseth
4a1e06b204
autopilot/agent: distribute available funds among channels
This commit fixes a regression in how we allocate funds to attempted
channels. We would earlier stay within the channel size limits, but we
wouldn't account for funds consumed by other channels being opened in
parallel.

We fix this by introducing a loop which greadily tries to distribute the
funds among the channels to open, and reduces the number of channels to
open in case not enough funds are available to satisfy the channel size
limits.
2019-03-14 08:49:28 +01:00
Johan T. Halseth
25f1252019
autopilot/externalscoreattach_test: add TestSetNodeScores 2019-02-14 12:37:25 +01:00
Johan T. Halseth
5dabb1ae29
autopilot+lnrpc: wire up SetNodeScores RPC to set scores of agent 2019-02-14 11:41:47 +01:00
Johan T. Halseth
b23e53ea33
autopilot/interface+externalscoreattach: define ScoreSettable
ScoreSettable is an interface that let caller set external scores for
the heuristic. The ExternalScoreAttachment and WeightedCombAttachment
heuristics implement this interface.
2019-02-14 11:41:47 +01:00
Johan T. Halseth
749d9cccca
autopilot/combinedattach: add SetNodeScores
This commit adds a method SetNodesScores to the WeightedCombAttachment
heuristic.

Since the heuristic keeps a list of sub-heuristics, it will attempt to
recursively apply the scores to the sub heuristics.
2019-02-14 11:37:47 +01:00
Johan T. Halseth
dff61facf2
autopilot: add ExternalScoreAttachment heuristic
This commit adds a new autopilot heuristic that is scoring based. It is
a simple heuristic that will keep a list of pubkeys and scores, and will
try opening channels with the nodes with the largest score first.
2019-02-14 11:37:47 +01:00
Johan T. Halseth
49a85b2eb3
autopilot/agent_test: add TestAgentQuitWhenPendingConns
TestAgentQuitWhenPendingConns tests that we are able to stop the autopilot
agent even though there are pending connections to nodes.
2019-02-05 10:45:53 +01:00
Johan T. Halseth
c19763c03f
autopilot/agent: don't block Stop on pending connetion
To ensure a call to ConnectToPeer doesn't block the agent from
shutting down, we'll launch it in a non-waitgrouped goroutine, that
will signal when a result is returned.
2019-02-05 10:45:53 +01:00
Johan T. Halseth
8f54a2bd6f
autopilot/manager: add QueryHeuristics
Proxies a query request to the active agent.
2019-01-23 09:05:43 +01:00
Johan T. Halseth
6c556fd92a
autopilot/agent: add queryHeuristics
This commit adds a method queryHeuristic that will query all simple
heuristics for node scores, in addition to the current active heuristic.
2019-01-23 09:05:43 +01:00
Johan T. Halseth
cf01879d96
autopilot/agent: protect chanState with mutex
Since we want to access the current channel state outside the main
autopilot agent goroutine, we make sure it is protected by a mutex.
2019-01-23 09:05:42 +01:00
Johan T. Halseth
36e7694e8e
config+pilot: define autopilot.heuristic
And validate the config set by the user.
2019-01-23 09:05:42 +01:00
Johan T. Halseth
c0fed861d2
autopilot/interface: add Name() to AttachmentHeuristic interface 2019-01-23 09:05:42 +01:00
Olaoluwa Osuntokun
cebc4d8dba
Merge pull request #2405 from halseth/autopilot-weighted-heuristics-follow-up
[autopilot] Weighted combined attachment heuristic
2019-01-22 19:08:46 -08:00
Valentine Wallace
f8e588e2e3
autopilot/graph: set max htlc in test edge policies
Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:29 +01:00
Valentine Wallace
0fd6004958
multi: partition lnwire.ChanUpdateFlag into ChannelFlags and MessageFlags
In this commit:

* we partition lnwire.ChanUpdateFlag into two (ChanUpdateChanFlags and
ChanUpdateMsgFlags), from a uint16 to a pair of uint8's

* we rename the ChannelUpdate.Flags to ChannelFlags and add an
additional MessageFlags field, which will be used to indicate the
presence of the optional field HtlcMaximumMsat within the ChannelUpdate.

* we partition ChannelEdgePolicy.Flags into message and channel flags.
This change corresponds to the partitioning of the ChannelUpdate's Flags
field into MessageFlags and ChannelFlags.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:26 +01:00