Commit Graph

722 Commits

Author SHA1 Message Date
Conner Fromknecht
cbf71b5452
channeldb+invoices: use payment addr as primary index 2020-05-26 18:55:27 -07:00
Conner Fromknecht
3522f09a08
channeldb+invoices: track invoices by InvoiceRef 2020-05-26 18:55:27 -07:00
Conner Fromknecht
e80e21d1a8
channeldb/migration: add generic CreateTLB migration
This commit creates a generic migration for creating top-level buckets.
2020-05-26 18:55:27 -07:00
Conner Fromknecht
24cce7a6ec
channeldb: consolidate top-level bucket create/wipe 2020-05-26 18:55:27 -07:00
Conner Fromknecht
d343575104
channeldb/migtest: log migration failure instead of failing
This line was incorrectly moved when the migtest package was created for
migration 12. This PR introduces a negative test for CreateTLB which
surfaced this.
2020-05-26 18:55:22 -07:00
Andras Banki-Horvath
c3fcfd1530 etcd: add namespace support to separate key spaces
This commit extends etcd db with namespaces without additional storage
space requirements. This is simply done by instead of using an all zero
root bucket id, we use the sha256 hash of the name space as our root
bucket id.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
bce0597643 channeldb: silence linter 2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
85aee9b064 kvdb+lncfg: fully move etcd behind build tag
This commit separates all etcd related sources (sans a few stubs and
config) from the rest of the source tree and makes compilation conditional
depending on whether the kvdb_etcd build tag is specified.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
3ef331e016 lncfg+etcd: add TLS config
This commit extends lncfg with etcd TLS config and passes these
parameters to the etcd client upon construction.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
b54f8a1b97 kvdb+channeldb: create ExtendedBackend interface to allow rich tx logic
This commit adds the ExtendedBackend interface which is an extension to
the walletdb.DB interface. This paves the way to using etcd.db.View and
etcd.db.Update in the global View and Update functions without much code
rewrite.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
23fcb59a2b channeldb: convert payment tests to use test backend 2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
02f9d91e49 kvdb+etcd: only lock on bucket path mod revision for reads and writes
This commit reduces the compare set size the STM will submit in
transactions by adding only the bucket keys along the bucket path to a
specific lock set. This lock set then used to filter the read set,
effectively removing all read only keys from the transaction predicate
that are not bucket keys.
By tracking if a read-write tx actually changes something, we can also
"bump" the mod revision of the bucket keys.
With this trick we essentially implement a read-write lock for our
bucket structure greatly reducing transaction processing time.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
859a457e48 kvdb+etcd: extend kvdb and STM with stats
The commit itslef adds stats to the transactions such that we can see
how LND behaves with and etcd backend.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
248a00f211 kvdb+etcd: enable channeldb testing with both bdb/etcd 2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
3b7525659c channeldb+kvdb: add walletdb interface test
This commit adds walletdb interface test suite to the existing test set.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
6a24a03cec channeldb+kvdb: walletdb/kvdb interface etcd implementation
This commit adds a full interface implementation of the walletdb/kvdb
interface with detailed tests.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
137dee04e8 channeldb+kvdb: an extended STM on top of etcd clientv3
This commit adds an extended STM, similar to what available in etcd's
clientv3 module. This incarnation of said STM supports additional
features, like positioning in key intervals while taking into account
deletes and writes as well. This is a preliminary work to support all
features of the kvdb interface.
2020-05-22 11:26:24 +02:00
Oliver Gugger
a17ddc5dd1
channeldb+htlcswitch: don't use fastsha256 in tests
The btcsuite/fastsha256 registers itself in the crypto package of
golang as a replacement for sha256. This causes problems in TLS1.3
connections that require the hash implementations to be serializable
and results in the "tls: internal error: failed to clone hash" error.
By removing all uses of the library we fix that error.
2020-05-20 11:07:38 +02:00
Conner Fromknecht
2f84d1a819
Merge pull request #4280 from cfromknecht/read-for-each-node
channeldb: remove Tx arg from ForEachNode
2020-05-13 16:37:10 -07:00
Conner Fromknecht
90d3bf532d
channeldb+routing: remove tx arg for ForEachNode 2020-05-12 17:53:20 -07:00
Joost Jager
cc37485432
routing: return full htlc attempt from shard handler 2020-05-12 19:56:50 +02:00
Conner Fromknecht
c775819372
channeldb/db: adds optional dry-run abort for migrations 2020-05-11 15:38:45 -07:00
Olaoluwa Osuntokun
cc1dbb5677
channeldb: eliminate extra copy in QueryPayments
In this commit, we eliminate an extraneous copy in the `QueryPayments`
method. Before this commit, we would copy each payment from the initial
FetchPayments call into a new slice. However, pointers to payments are
return from `FetchPayments`, so we can just maintain that same reference
rather than copying again when we want to limit our response.
2020-05-04 12:13:47 -07:00
Andras Banki-Horvath
556e3525ea misc: fix error formatting in multiple files 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath
e636364ccf channeldb: remove dead code from payments test 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath
3628e65378 channeldb: (nit) remove size argument from map make 2020-04-24 19:15:08 +02:00
Joost Jager
7fc1938f10
routing: payment splitting pre-check
This commit reverts cb4cd49dc8d3b0255afe9ff29af9c46c2dbb2c98 to bring
back the insufficient local balance failure.

Distinguishing betweeen this failure and a regular "no route" failure
prevents meaningless htlcs from being sent out.
2020-04-18 08:30:26 +02:00
Conner Fromknecht
93089aaaf8
channeldb/channel: don't print ChanStatusDefault when it's not
This commit removes ChanStatusDefault from the list of
orderedChanStatusFlags since it is not flag. As with the prior commit,
the logic around these flags assumes everything in the list is a flag,
but ChanStatusDefault is not.

It turns out we properly special case that if the channel is in
ChanStatusDefault that we only return ChanStatusDefault. However, if any
of the bits are set we would always report ChanStatusDefault since
status&0 == 0. This fixed simply by removing ChanStatusDefault from the
list since we only need the list to express non-default status flags.
2020-04-10 16:11:44 -07:00
Conner Fromknecht
f71cc951fd
channeldb/channel: fix HasChanStatus for ChanStatusDefault
This commit resovles a lingering issue w/in the codebase wrt how the
ChannelStatus flags are defined. Currently ChannelStatus is improperly
used to define a bit field and the individual flags themselves. As a
result, HasChanStatus accepts queries on particular status (combinations
of flags) and individual flags themselves.

This is an issue because the way HasChanStatus computes whether the
channel has a particular status assumes the provided inputs are all
flags (or at least combinations of flags).

However, ChanStatusDefault is simply the absence of any other flag.
Hence, HasChanStatus will always return true when querying for
ChanStatusDefault because status&0 == 0 is always true.

Longer term we should should consider splitting these definitions into
flags and particular states, and change the way construct or operate on
them, but for now I've just special-cased this one value. Fortunately,
we don't query HasChannelStatus w/ ChanStatusDefault anywhere in the
codebase so we dodge a bullet here, but it'd be nice to have some
greater assurances moving forward.
2020-04-10 16:05:12 -07:00
Joost Jager
cb4cd49dc8
routing: map insufficient local bandwidth error to no path
With mpp it isn't possible anymore for findPath to determine that there
isn't enough local bandwidth. The full payment amount isn't known at
that point.

In a follow-up, this payment outcome can be reintroduced on a higher
level (payment lifecycle).
2020-04-09 08:20:35 +02:00
Joost Jager
278915e598
channeldb: return updated payment on attempt update
Similar to what is done for SettleAttempt.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2020-04-08 08:54:01 +02:00
Olaoluwa Osuntokun
7e6f3ece23
Merge pull request #3960 from bitromortac/listpayments-pagination
channeldb+lnrpc+lncli: listpayments pagination support
2020-04-07 17:06:37 -07:00
Conner Fromknecht
310e20f8b7
invoices/invoiceregistry: remove debug spew 2020-04-07 11:54:29 -07:00
bitromortac
d5dd48fa71 channeldb/test: unit tests for payments query
Adds tests for the payments query, where different edge cases for
the index offsets and normal and reversed orders are tested.
2020-04-07 07:03:21 +02:00
bitromortac
4c5e8aec41 channeldb: add payments query
Adds a PaymentsQuery struct, which contains parameters to restrict the
response of QueryPayments, returning a PaymentsQuerySlice with the
payments query result. The behavior of this api is the same as
the QueryInvoices one.
2020-04-07 07:03:21 +02:00
bitromortac
4800a84122 channeldb: export sequenceNum in MPPayment
Exports sequenceNum in MPPayment for later use
in the rpcserver.
2020-04-07 07:03:21 +02:00
Olaoluwa Osuntokun
92ee49767f
Merge pull request #3970 from halseth/amp-router-mvp-2020
MPP: Enable MultiPathPayments for payment lifecycle
2020-04-03 13:53:19 -07:00
Conner Fromknecht
9385b8cdc6
channeldb+lnwallet: move ActiveHtlcs calc to OpenChannel 2020-04-02 17:39:10 -07:00
Johan T. Halseth
864e64e725
channeldb: validate MPP options when registering attempts
We add validation making sure we are not trying to register MPP shards
for non-MPP payments, and vice versa. We also add validtion of total
sent amount against payment value, and matching MPP options.

We also add methods for copying Route/Hop, since it is useful to use
for modifying the route amount in the test.
2020-04-02 19:31:23 +02:00
Johan T. Halseth
9a1ec950bd
channeldb/payments: extract common info fetch into fetchCreationInfo 2020-04-02 19:29:15 +02:00
Johan T. Halseth
7b318a4be7
routing/payment_lifecycle+channeldb: enable multi shard send
This commit finally enables MP payments within the payment lifecycle
(used for SendPayment). This is done by letting the loop launch shards
as long as there is value remaining to send, inspecting the outcomes for
the sent shards when the full payment amount has been filled.

The method channeldb.MPPayment.SentAmt() is added to easily look up how
much value we have sent for the payment.
2020-04-02 19:29:15 +02:00
Johan T. Halseth
3610824abd
channeldb/payment_control_test: add TestPaymentControlMultiShard 2020-04-02 19:29:14 +02:00
Johan T. Halseth
70202be580
channeldb: make database logic MPP compatible
This commit redefines how the control tower handles shard and payment
level settles and failures. We now consider the payment in flight as
long it has active shards, or it has no active shards but has not
reached a terminal condition (settle of one of the shards, or a payment
level failure has been encountered).

We also make it possible to settle/fail shards regardless of the payment
level status (since we must allow late shards recording their status
even though we have already settled/failed the payment).

Finally, we make it possible to Fail the payment when it is already
failed. This is to allow multiple concurrent shards that reach terminal
errors to mark the payment failed, without havinng to synchronize.
2020-04-02 19:29:14 +02:00
Johan T. Halseth
f6c97daf0c
channeldb/payment_control_test: simplify HTLC status assertion 2020-04-02 10:24:36 +02:00
Johan T. Halseth
7b5c10814b
routing/payment_lifecycle+channeldb: collect existing outcome first
To move towards how we will handle existing attempt in case of MPP
(collecting their outcome will be done in separate goroutines separate
from the payment loop), we move to collect their outcome first.

To easily fetch HTLCs that are still not resolved, we add the utility
method InFlightHTLCs to channeldb.MPPayment.
2020-04-02 10:24:35 +02:00
Johan T. Halseth
9bcf41d401
channeldb: make FailureReason Error() 2020-04-02 10:24:34 +02:00
nsa
f757bf48bd channeldb: use RemoteCommitment in NextLocalHtlcIndex
This commit changes the fallback in NextLocalHtlcIndex to
RemoteCommitment since the LocalHtlcIndex field lags behind
on the LocalCommitment. Without this bug fix, open circuits
would get prematurely trimmed, resulting in more erroneous
logs. A test case is included to check that the fix works.
2020-03-25 11:49:16 -04:00
Olaoluwa Osuntokun
421d73b72e
channeldb: add new frozen channel type bit and thaw height
In this commit, we add a new channel type bit: a frozen channel. A
frozen channel is one that can only be cooperatively closed by the
responder, but not the initiator. This channel type is useful for
certain classes of channel factory like protocols. We then add a new key
on the channel bucket level to store the height after which this
restriction no longer applies.
2020-03-23 16:56:48 -07:00
carla
41355756a1
channeldb: add balance at height lookup
Add a balance at height lookup function which can be used to
obtain local/remote balance at a given height. The current in memory
commits and revocation log are used to source this information.
2020-03-19 13:20:17 +02:00
Olaoluwa Osuntokun
a4e39906b1
build: silence new linter errors, tidy modules
The explicit `bbolt` dep is gone, as we depend on `kvdb`, which is
actually `walletdb`, which has its own module that defines the proper
`bbolt` version.
2020-03-18 19:35:29 -07:00