Commit Graph

11012 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
8c389d13f9
Merge pull request #3197 from breez/optimize_prune_zombie_channels
Optimize prune zombie channels
2019-07-18 20:56:56 -07:00
Olaoluwa Osuntokun
111cbeaa99
Merge pull request #3165 from halseth/autopilot-spend-coins
[funding+autopilot] Make the funding manager and autopilot fee aware
2019-07-18 20:45:49 -07:00
Olaoluwa Osuntokun
02a5408b15
Merge pull request #3221 from joostjager/queryroutes-mc
routing+routerrpc+lnrpc: add option to use mc in queryroutes
2019-07-18 20:30:29 -07:00
nsa
00814dc7c1 htlcswitch: pipeline settles to switch
This commit makes the outgoing link pipeline the settle to the
switch as soon as it receives it. Previously, it would wait for a
revocation before sending it, which caused increased latency on
payments as well as possibly never settling on the incoming link.
A duplicate settle is still sent to the switch, but it is handled
gracefully. A new AckEventTicker was added to the switch which
acknowledges any pending settle / fail entries in an outgoing
link's fwd pkgs in batch. This was needed in order to reduce the
number of db txn's which would have been incurred by acking whenever
we receive a duplicate settle without batching.
2019-07-18 17:20:40 -04:00
Roei Erez
da9edc876a router: only prune disabled channels when AssumeChannelValid=true. 2019-07-18 17:05:20 +03:00
Joost Jager
541e5f4af7
lncli: add use_mc flag 2019-07-18 15:49:30 +02:00
Joost Jager
fc337cd34f
routing+routerrpc+lnrpc: add option to use mc in queryroutes
A boolean flag is added to the QueryRoutes rpc that allows feeding
mission control probabilities into path finding.
2019-07-18 15:49:29 +02:00
Joost Jager
7f4c42847c
lnrpc/routerrpc: create mission control interface 2019-07-18 15:49:27 +02:00
Joost Jager
34b264a3d8
routing: create PathFindingConfig for global parameters 2019-07-18 15:49:25 +02:00
Conner Fromknecht
377b7bf3ce
Merge pull request #3316 from cfromknecht/num-zombies-rpc
rpcserver+channeldb: return num zombies in NetworkInfo response
2019-07-17 16:27:48 -07:00
Wilmer Paulino
1730f1ea72
Merge pull request #3325 from wpaulino/lnd-log-error
lnd: log error if any throughout initialization
2019-07-17 16:05:06 -07:00
Johan T. Halseth
7838e6e1fa
autopilot/agent_test: add more agent channel allocation testcases
Add tests for allocation when well within and well above channel budget.
2019-07-18 00:58:12 +02:00
Johan T. Halseth
68f7642564
autopilot/agent_test: wait for expected num chans and balance
Previously we waited only for the number of channels to become what we
expected, but this wasn't enough. Sometimes the agent had't yet updated
its internal balance, causing the test to fail with an unexpected
balance.
2019-07-18 00:58:12 +02:00
Johan T. Halseth
bf7f392c11
autopilot: use subtractFees when funding channel
To make the autopilot able to account for fees, we let it use the
subtractFees option when opening channels.

This makes sure that each channel we attempt to open will eat at most
Amt out of our budget. Previously fees would eat into our funds in
addition, causing us to deplete our funds more than expected on each
channel opening.
2019-07-18 00:58:12 +02:00
Johan T. Halseth
0405703019
fundingmanager test: add TestFundingManagerFundAll
TestFundingManagerFundAll tests that we can initiate a funding request to
use the funds remaining in the wallet. This should produce a funding tx with
no change output.
2019-07-18 00:58:12 +02:00
Johan T. Halseth
b9816259cb
fundingmanager+server: define subtractFees
Let one initiate a funding request with the remaining funds in the
wallet.
2019-07-18 00:58:12 +02:00
Johan T. Halseth
87c8165f01
mock: make it possible to set list of returned utxos 2019-07-18 00:58:12 +02:00
Johan T. Halseth
2cd7d5a2a4
fundingmanager test: extract publishing logic into fundChannel, set funding fee during unit tests 2019-07-18 00:58:12 +02:00
Johan T. Halseth
f15d81426c
lnwallet/wallet: define SubtractFees for InitFundingReserveMsg
This commit adds a SubtractFees option to the funding request, letting
the caller specify that the fees should be deducted from the funding
amount. This paves the way for letting the funding manager spend up to a
given amount when creating a channel, like the rest of the funds in the
wallet.
2019-07-18 00:33:15 +02:00
Johan T. Halseth
4239f7d600
lnwallet/wallet: add coinSelectSubtractFees 2019-07-18 00:33:15 +02:00
Johan T. Halseth
1b2297c2b5
lnwallet/wallet: extract contribution setup into initOurContribution
This lets us easily call unlock() in case contribution setup fails.
2019-07-18 00:33:15 +02:00
Johan T. Halseth
44384a1b5b
lnwallet: move coin selection before ChannelReservation
Now that coin selection is independent of creating the channel
reservation, we can move it first, in preparation for doing custom coin
selection.
2019-07-18 00:33:14 +02:00
Johan T. Halseth
98a3d04ba3
lnwallet: make selectCoinsAndChange return selected coins
This makes the method independent of the ChannelContribution struct.

We also add a function closure to the return of selectCoinsAndChange,
that let is unlock the selected output in case of error.
2019-07-18 00:33:14 +02:00
Johan T. Halseth
fcf74debe6
lnwallet+funding: rename Capacity and FundingAmt
Instead use LocalFundingAmt and RemoteFundingAmt to make it clear who is
contributing funds.
2019-07-18 00:33:14 +02:00
Johan T. Halseth
e716251805
server+funding: remove unused remoteFundingAmt 2019-07-18 00:33:14 +02:00
Johan T. Halseth
2f5c1a69fb
lnwallet test: add TestCoinSelect 2019-07-18 00:33:14 +02:00
Johan T. Halseth
78da62c6f7
fundingmanager tests: run tests in parallel 2019-07-18 00:33:14 +02:00
Johan T. Halseth
452ee6aad4
fundingmanager+server: define MaxPendingChans. RejectPush in funding config
This commit makes the funding manager access the MaxPendingChannels and
RejectPush values from the fundingConfig instead of the global config
struct.

Done to avoid sharing state between tests.
2019-07-18 00:33:14 +02:00
Wilmer Paulino
0d3ef43c92
lnd: log error if any throughout initialization
In this commit, we establish a new pattern to always log errors before
returning them to the higher level caller, which then prints the error
to stdout/stderr. Errors returned are usually lowercase, but we decide
not to apply this rule here as these errors should not be chained
forward.
2019-07-17 14:24:21 -07:00
Olaoluwa Osuntokun
98274f63dc
Merge pull request #3307 from Roasbeef/better-aezeed-error
aezeed: publicly export the `word` field  in `ErrUnknownMnenomicWord`
2019-07-16 20:15:48 -07:00
Olaoluwa Osuntokun
d5122b7f04
aezeed: publicly export the word field in ErrUnknownMnenomicWord
In this commit, we publicly export the `word` field as it makes it
easier to programmatically interact with the package when attempting to
re-derive proper `cipherseed` instances. We also add a new `Index` field
as well to provide additional context for programmatic manipulating of
seeds.
2019-07-16 19:31:35 -07:00
Olaoluwa Osuntokun
92e14af72a
Merge pull request #3298 from joostjager/globalize-mc
routing: globalize mc
2019-07-16 17:39:36 -07:00
Olaoluwa Osuntokun
3faece15be
Merge pull request #3314 from cfromknecht/remove-initial-graph-sync
peer: don't reply to requests for `initial_graph_sync`
2019-07-16 16:45:38 -07:00
Conner Fromknecht
97d8be7f6a
rpcserver: query and set num_zombie_chans in NetworkInfo resp 2019-07-16 16:32:27 -07:00
Conner Fromknecht
0223039fb4
channeldb/graph: add NumZombies method and tests 2019-07-16 16:32:27 -07:00
Conner Fromknecht
4b70d6b987
lnrpc: add num_zombie_chans to NetworkInfo response 2019-07-16 16:32:27 -07:00
Roei Erez
9781ea0082 router: Optimize pruneZombieChannels.
The current approach iterates all channels in the graph in order to
filter those in need. This approach is time consuming, several seconds
on my mobile device for ~40,000 channels, while during this time the
db is locked in a transaction.

The proposed change is to use an existing functionality that utilize the
fact that channel update are saved indexed by date. This method enables
us to go over only a small subset of the channels, only those that
were updated before the "channel expiry" time and further filter
them for our need.
The same graph that took several seconds to prune was pruned, after
the change, in several milliseconds.

In addition for testing purposes I added Initiator field to the
testChannel structure to reflect the channeldEdgePolicy direction.
2019-07-16 17:48:07 +03:00
Roei Erez
7300f33fe2 graph: implement index for disabled channels.
This commit adds an index bucket, disabledEdgePolicyBucket, for those
ChannelEdgePolicy with disabled bit on.
The main purpose is to be able to iterate over these fast when prune is
needed without the need for iterating the whole graph.

The entry points for accessing this index are:
1. When updating ChannelEdgePolicy - insert an entry.
2. When deleting ChannelEdge - delete the associated entries.
3. When querying for disabled channels - implemented DisabledChannelIDs
function
2019-07-16 17:48:07 +03:00
Olaoluwa Osuntokun
a7152efca4
Merge pull request #3313 from wpaulino/neutrino-cleanup-panic
lnd: prevent panic on nil neutrino cleanup
2019-07-15 19:41:29 -07:00
Conner Fromknecht
a4097c113a
discovery/gossiper: remove unused SynchronizeNode method 2019-07-15 14:11:18 -07:00
Conner Fromknecht
9eda94a40e
peer: don't reply if peer's request initial_graph_sync
We stopped requesting this from other nodes as it is very expensive as
the graph continues to grow. In this commit we will also stop
responding, as nodes are recommended to begin using the `gossip_queries`
and upcoming `extended_gossip_queries` to reconcile missing graph data.
2019-07-15 14:09:42 -07:00
Wilmer Paulino
62d716d3c3
chainregistry: properly close neutrino database upon errors 2019-07-15 13:42:01 -07:00
Wilmer Paulino
e147445c08
lnd: prevent panic on nil neutrino cleanup
A cleanup closure is not included when an error is returned, causing the
defer to execute and triggering the following panic:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x105da38]

goroutine 1 [running]:
github.com/lightningnetwork/lnd.Main(0x2083e40, 0xc0004f6db0)
	/home/user/lnd/lnd.go:208 +0x2bfa
main.main()
	/home/user/lnd/cmd/lnd/main.go:14 +0x26
2019-07-15 13:32:37 -07:00
Joost Jager
8055bcf2e0
routing: report failures to mission control directly
As there is no more state kept in the payment session, failure reporting
can go straight to mission control.
2019-07-13 22:38:25 +02:00
Joost Jager
dc13da5abb
routing: move second chance logic into mission control
If nodes return a channel policy related failure, they may get a second
chance. Our graph may not be up to date. Previously this logic was
contained in the payment session.

This commit moves that into global mission control and thereby removes
the last mission control state that was kept on the payment level.

Because mission control is not aware of the relation between payment
attempts and payments, the second chance logic is no longer based
tracking second chances given per payment.

Instead a time based approach is used. If a node reports a policy
failure that prevents forwarding to its peer, it will get a second
chance. But it will get it only if the previous second chance was
long enough ago.

Also those second chances are no longer dependent on whether an
associated channel update is valid. It will get the second chance
regardless, to prevent creating a dependency between mission control and
the graph. This would interfer with (future) replay of history, because
the graph may not be the same anymore at that point.
2019-07-13 22:38:23 +02:00
Joost Jager
d31efddf1b
routing: move generateBandwidthHints 2019-07-13 22:38:21 +02:00
Joost Jager
37e2751695
routing+routerrpc: isolate payment session source from mission control 2019-07-13 22:38:19 +02:00
Joost Jager
cf3dd3fb94
routing/test: create mission control test context 2019-07-13 22:38:17 +02:00
Olaoluwa Osuntokun
616750184e
Merge pull request #2926 from Crypt-iQ/visited_nodes_0410
routing: tracking visited nodes in findPath
2019-07-12 17:29:11 -07:00
Conner Fromknecht
0c8ff3021c
Merge pull request #2682 from valentinewallace/fix-subscribe-chans-flake
lnd_test: fix channel event subscription test flake.
2019-07-12 16:08:18 -07:00