Commit Graph

767 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
7b589e5811
routing: add strict zombie pruning as a config level param
In this commit, we add strict zombie pruning as a config level param.
This allow us to add the option for those that want a tighter graph, and
not change the default composition of the channel graph for most users
over night.

In addition, we expand the test case slightly by testing that the self
node won't be pruned, but also that if there's a node with only a single
known stale edge, then both variants will prune that edge.
2021-04-21 13:56:27 -05:00
Olaoluwa Osuntokun
916059da48
routing: update chan pruning test w/ new zombie logic 2021-04-21 13:56:24 -05:00
Conner Fromknecht
e3a8b3b0c4
routing/router: prune zombies when either end is stale 2021-04-21 13:56:21 -05:00
yyforyongyu
37aa49c7aa
routing: fix pathfind edge features being nil 2021-04-13 15:53:18 +08:00
Wilmer Paulino
82fe5d9bba
build: update btcwallet dependency introducing pruned bitcoind support
This is achieved by some recent work within the BitcoindClient enabling
it to retrieve pruned blocks from its server's peers.
2021-04-06 14:55:14 -07:00
Olaoluwa Osuntokun
ca96e66b43
Merge pull request #5116 from joostjager/mc-deadlock
routing: fix mission control deadlock
2021-04-05 20:02:57 -07:00
Wilmer Paulino
a620ce3682
build: update btcd and btcwallet dependencies 2021-04-05 15:41:04 -07:00
Olaoluwa Osuntokun
a329c80612
Merge pull request #5133 from wpaulino/routing-validation-cancel-deps
discovery+routing: cancel dependent jobs if parent validation fails
2021-04-01 18:32:58 -07:00
Johan T. Halseth
1231c90a19
routing: avoid open DB transaction if no zombies to prune
We add a simple length check to the channels to be pruned to avoid
opening the DB if there is nothing to be done.
2021-03-30 11:04:13 +02:00
Johan T. Halseth
a0f3624303
routing: delay initial zombie prune by 30 sec
Since zombie pruning can be very slow on some devices (e.g. mobile) it
would stall lnd startup. Since it is not essential for pruning to be
finished for lnd to be functional, we instead delay the initial prune by
30 seconds.

Note that we could also wait for the graphPruneInterval to tick, but
since this is by default 2 hours, it is unlikely that a mobile app will
ever be open that long.
2021-03-30 11:04:13 +02:00
Wilmer Paulino
393111cea9
discovery+routing: cancel dependent jobs if parent validation fails
Previously, we would always allow dependent jobs to be processed,
regardless of the result of its parent job's validation. This isn't
correct, as a parent job contains actions necessary to successfully
process a dependent job. A prime example of this can be found within the
AuthenticatedGossiper, where an incoming channel announcement and update
are both processed, but if the channel announcement job fails to
complete, then the gossiper is unable to properly validate the update.
This commit aims to address this by preventing the dependent jobs to
run.
2021-03-23 11:56:51 -07:00
carla
22491ad567
routing: add mission control import functionality 2021-03-18 10:46:45 +02:00
Joost Jager
56238ebc60 routing: remove unnecessary lock in mission control init
The init code is part of the instantiation, so there is no need for
locking yet.
2021-03-17 12:06:12 +01:00
Joost Jager
89751f869f routing: fix mission control deadlock
This commit fixes the following potential deadlock situation:
* Pathfinding holds a database lock and tries to obtain a mission control lock
via GetProbability
* ReportPaymentSuccess/ReportPaymentFail holds a mission control lock
and tries to obtain a database lock to store the payment result.
2021-03-17 12:03:32 +01:00
yyforyongyu
541fbbb054
routing: check payment.DestFeatures against nil 2021-03-17 00:14:20 +08:00
Conner Fromknecht
250bc8560e
routing: avoid modifying AssumeChannelValid in unit tests
This produces a race condition when reading AssumeChannelValid from a
different goroutine. Instead we isolate the test cases and initial
AssumeChannelValid properly.
2021-02-17 22:43:24 -08:00
Conner Fromknecht
f7c5236bf6
routing: dial back max concurrent block fetches
This commit reduces the number of concurrent validation operations the
router will perform when fully validating the channel graph. Reports
from several users indicate that GetInfo would hang for several minutes,
which is believed to be caused by attempting to validate massive amounts
of channels in parallel. This commit returns the limit back to its
original state before adding the batched gossip improvements.

We keep the 1000 concurrent validation request limit for
AssumeChannelValid, since we don't fetch blocks in that case. This
allows us to still keep the performance benefits on mobile/low-resource
devices.
2021-02-17 18:17:09 -08:00
Olaoluwa Osuntokun
b73a6e2c61
routing: if MaxShardAmt is set, then use that as a ceiling for our splits
In this commit, we thread through the necessary state to allow users to
set a max shard amount. If this value is set, then this'll effectively
serve as a ceiling for all our split attempts. If we need to split,
we'll first try to use `paymentAmt/2`, if that's bigger than
`MaxShardAmt, then we'll use the latter instead.

Ideally in the future we have a dynamic way to automatically set both
the `MaxShardAmt` as well as `MaxParts` for users. Until then exposing
these two new fields will allow us to experiment with setting them
automatically using the RPC interface, and also give users a bit more
control over how we attempt to route payments, akin to coin control for
on-chain payments.

Fixes #4730
2021-02-15 19:31:52 -08:00
Olaoluwa Osuntokun
7398e59927
lnrpc/routerrpc+routing: add new MaxShardAmt field to LightningPayment 2021-02-15 19:31:49 -08:00
Olaoluwa Osuntokun
7b0ea3c029
Merge pull request #4909 from carlaKC/mc-paramsapi
routing: allow runtime updates to mission control config
2021-02-10 18:51:53 -08:00
Johan T. Halseth
7e34132c53
routing: let graph methods take scheduler option 2021-02-10 23:54:03 +01:00
Olaoluwa Osuntokun
107f19a049
routing: add new TestPaymentAddrOnlyNoSplit test case
This test case ensures that we won't try to split payment if the dest
has the payment addr bit, but NOT the mpp optional/require bit.
2021-02-03 17:53:40 -08:00
Olaoluwa Osuntokun
27c1779757
routing: allow custom dest feature bits in integratedRoutingContext.testPayment
This is a preparatory commit for a new test to ensure that if a node
only has the TLV and payment addr feature bits, we don't try to split a
payment.
2021-02-03 17:53:32 -08:00
Olaoluwa Osuntokun
301f1a870e
Merge pull request #4924 from champo/check_payreq_multipart
routerrpc,routing: limit max parts if the invoice doesn't declare MPP support
2021-02-03 16:53:49 -08:00
Oliver Gugger
02267565fe
multi: unify code blocks in READMEs 2021-01-22 09:14:11 +01:00
carla
64dad77e2e
multi: add get and set mission control to routerrpc 2021-01-19 10:57:15 +02:00
carla
edac5bb868
routing: add getter and setter for mission control config 2021-01-19 10:57:14 +02:00
carla
e10e8f11de
routing: extract probability estimator cfg and add validation
In preparation for allowing live update of mc config, we extract our
probability estimator cfg for easy update and add validation.
2021-01-19 10:57:13 +02:00
carla
7b24b586a0
routing: move locking for ReportPaymentSuccess and ReportPaymentFailure
All of the other mission control exported functions acquire their locks
immediately, and do not lock in the subsequent unexported functions.
This commit moves the lock up for the report payment functions so that
mission control's config values are covered by this lock, in preparation
for allowing config to be updated at runtime. Moving this lock means
that we will hold the lock for the additional time it takes to store a
single result, AddResult, to the store.
2021-01-19 10:57:12 +02:00
carla
97442da8f7
routing: add string method for cfg 2021-01-19 10:57:11 +02:00
carla
0735d359b9
router: move self node out of config
We are going to use the config struct to allow getting and setting
of the mission control config in the commits that follow. Self node
is not something we want to change, so we move it out for better
separation.
2021-01-19 10:57:10 +02:00
Juan Pablo Civile
f574c150aa routing: limit max parts if the invoice doesn't declare MPP support 2021-01-15 14:56:32 -03:00
András Bánki-Horváth
669663bfc2
Merge pull request #4822 from bhandras/add_globalfeatures
routing: add missing features to NetworkNodeUpdate (and also depreacte GlobalFeatures)
2020-12-08 10:16:50 +01:00
Yong
582b164c46
kvdb: add timeout options for bbolt (#4787)
* mod: bump btcwallet version to accept db timeout

* btcwallet: add DBTimeOut in config

* kvdb: add database timeout option for bbolt

This commit adds a DBTimeout option in bbolt config. The relevant
functions walletdb.Open/Create are updated to use this config. In
addition, the bolt compacter also applies the new timeout option.

* channeldb: add DBTimeout in db options

This commit adds the DBTimeout option for channeldb. A new unit
test file is created to test the default options. In addition,
the params used in kvdb.Create inside channeldb_test is updated
with a DefaultDBTimeout value.

* contractcourt+routing: use DBTimeout in kvdb

This commit touches multiple test files in contractcourt and routing.
The call of function kvdb.Create and kvdb.Open are now updated with
the new param DBTimeout, using the default value kvdb.DefaultDBTimeout.

* lncfg: add DBTimeout option in db config

The DBTimeout option is added to db config. A new unit test is
added to check the default DB config is created as expected.

* migration: add DBTimeout param in kvdb.Create/kvdb.Open

* keychain: update tests to use DBTimeout param

* htlcswitch+chainreg: add DBTimeout option

* macaroons: support DBTimeout config in creation

This commit adds the DBTimeout during the creation of macaroons.db.
The usage of kvdb.Create and kvdb.Open in its tests are updated with
a timeout value using kvdb.DefaultDBTimeout.

* walletunlocker: add dbTimeout option in UnlockerService

This commit adds a new param, dbTimeout, during the creation of
UnlockerService. This param is then passed to wallet.NewLoader
inside various service calls, specifying a timeout value to be
used when opening the bbolt. In addition, the macaroonService
is also called with this dbTimeout param.

* watchtower/wtdb: add dbTimeout param during creation

This commit adds the dbTimeout param for the creation of both
watchtower.db and wtclient.db.

* multi: add db timeout param for walletdb.Create

This commit adds the db timeout param for the function call
walletdb.Create. It touches only the test files found in chainntnfs,
lnwallet, and routing.

* lnd: pass DBTimeout config to relevant services

This commit enables lnd to pass the DBTimeout config to the following
services/config/functions,
  - chainControlConfig
  - walletunlocker
  - wallet.NewLoader
  - macaroons
  - watchtower
In addition, the usage of wallet.Create is updated too.

* sample-config: add dbtimeout option
2020-12-07 15:31:49 -08:00
Andras Banki-Horvath
98c61be342
routing: add missing GlobalFeatures to NetworkNodeUpdate 2020-12-07 15:47:09 +01:00
Oliver Gugger
2edc3cf98f
multi: update to latest btcd version
With this commit we update btcd to the latest version which allows us to
specify the btcd binary we want to use when spinning up a new node.
2020-12-03 11:30:22 +01:00
Olaoluwa Osuntokun
7e298f1434
Merge pull request #3367 from cfromknecht/batched-graph-updates
Batched graph updates
2020-11-25 18:40:40 -08:00
Olaoluwa Osuntokun
d996607470
routing+lnrpc: extend BuildRoute to accept raw payAddr
In this commit, we extend the `BuildRoute` method and RPC on the router
sub-server to accept a raw payment address which will be included as
part of an MPP payload for the finla hop. This change actually also
allows users to craft their own MPP paths using BuildRoute+SendToRoute.
Our primary goal however, was to fix some broken itests since we now
require the payAddr to be present for ALL payments other than key send
payments.
2020-11-25 16:32:02 -08:00
Conner Fromknecht
e8c545e909
routing/router: increase validation barrier size to 1000
This allows for a 1000 different persistent operations to proceed
concurrently. Now that we are batching operations at the db level, the
average number of outstanding requests will be higher since the commit
latency has increased. To compensate, we allow for more outstanding
requests to keep the router busy while batches are constructed.
2020-11-24 16:40:10 -08:00
Johan T. Halseth
df049ad755
routing: promote shard logs on resume and terminal failure to Info 2020-11-24 14:59:39 +01:00
Johan T. Halseth
f8493a7522
routing: define errShardHandlerExiting 2020-11-24 14:59:39 +01:00
Andras Banki-Horvath
d89f51d1d0
multi: add reset closure to kvdb.Update
Similarly as with kvdb.View this commits adds a reset closure to the
kvdb.Update call in order to be able to reset external state if the
underlying db backend needs to retry the transaction.
2020-11-05 17:57:12 +01:00
Andras Banki-Horvath
2a358327f4
multi: add reset closure to kvdb.View
This commit adds a reset() closure to the kvdb.View function which will
be called before each retry (including the first) of the view
transaction. The reset() closure can be used to reset external state
(eg slices or maps) where the view closure puts intermediate results.
2020-11-05 17:57:12 +01:00
carla
c92b030652
multi: remove experimential build flag for assumechanvalid 2020-10-21 14:13:50 +02:00
Johan T. Halseth
90a59fe70f
router: call CleanStore on startup 2020-10-06 10:46:04 +02:00
Johan T. Halseth
d1a118388d
payment_control: remove unused Attempts field 2020-10-06 10:41:25 +02:00
Joost Jager
bbfeeec8b1
routing: add proportional attempt cost 2020-09-08 15:47:08 +02:00
Joost Jager
60a22ff09d
routing/test: parametrize test amount 2020-09-08 15:28:49 +02:00
Joost Jager
17a6175e8b
routing+routerrpc: rename PaymentAttemptPenalty to AttemptCost
Make field names consistent with the command line flag.
2020-09-08 14:07:56 +02:00
Conner Fromknecht
6622c4814e
multi: enforce routing.MinCLTVDelta=18 for invoices + chanupd
This commit clamps all user-chosen CLTVs in LND to be at least 18, which
is the new conservative value used in the sepc. This minimum is applied
uniformly to forwarding CLTV deltas (via channel updates) as well as
final CLTV deltas for new invoices.
2020-07-24 13:13:56 -07:00