Commit Graph

709 Commits

Author SHA1 Message Date
Andras Banki-Horvath
248a00f211 kvdb+etcd: enable channeldb testing with both bdb/etcd 2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
3b7525659c channeldb+kvdb: add walletdb interface test
This commit adds walletdb interface test suite to the existing test set.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
6a24a03cec channeldb+kvdb: walletdb/kvdb interface etcd implementation
This commit adds a full interface implementation of the walletdb/kvdb
interface with detailed tests.
2020-05-22 11:26:25 +02:00
Andras Banki-Horvath
137dee04e8 channeldb+kvdb: an extended STM on top of etcd clientv3
This commit adds an extended STM, similar to what available in etcd's
clientv3 module. This incarnation of said STM supports additional
features, like positioning in key intervals while taking into account
deletes and writes as well. This is a preliminary work to support all
features of the kvdb interface.
2020-05-22 11:26:24 +02:00
Oliver Gugger
a17ddc5dd1
channeldb+htlcswitch: don't use fastsha256 in tests
The btcsuite/fastsha256 registers itself in the crypto package of
golang as a replacement for sha256. This causes problems in TLS1.3
connections that require the hash implementations to be serializable
and results in the "tls: internal error: failed to clone hash" error.
By removing all uses of the library we fix that error.
2020-05-20 11:07:38 +02:00
Conner Fromknecht
2f84d1a819
Merge pull request #4280 from cfromknecht/read-for-each-node
channeldb: remove Tx arg from ForEachNode
2020-05-13 16:37:10 -07:00
Conner Fromknecht
90d3bf532d
channeldb+routing: remove tx arg for ForEachNode 2020-05-12 17:53:20 -07:00
Joost Jager
cc37485432
routing: return full htlc attempt from shard handler 2020-05-12 19:56:50 +02:00
Conner Fromknecht
c775819372
channeldb/db: adds optional dry-run abort for migrations 2020-05-11 15:38:45 -07:00
Olaoluwa Osuntokun
cc1dbb5677
channeldb: eliminate extra copy in QueryPayments
In this commit, we eliminate an extraneous copy in the `QueryPayments`
method. Before this commit, we would copy each payment from the initial
FetchPayments call into a new slice. However, pointers to payments are
return from `FetchPayments`, so we can just maintain that same reference
rather than copying again when we want to limit our response.
2020-05-04 12:13:47 -07:00
Andras Banki-Horvath
556e3525ea misc: fix error formatting in multiple files 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath
e636364ccf channeldb: remove dead code from payments test 2020-04-24 19:15:08 +02:00
Andras Banki-Horvath
3628e65378 channeldb: (nit) remove size argument from map make 2020-04-24 19:15:08 +02:00
Joost Jager
7fc1938f10
routing: payment splitting pre-check
This commit reverts cb4cd49dc8d3b0255afe9ff29af9c46c2dbb2c98 to bring
back the insufficient local balance failure.

Distinguishing betweeen this failure and a regular "no route" failure
prevents meaningless htlcs from being sent out.
2020-04-18 08:30:26 +02:00
Conner Fromknecht
93089aaaf8
channeldb/channel: don't print ChanStatusDefault when it's not
This commit removes ChanStatusDefault from the list of
orderedChanStatusFlags since it is not flag. As with the prior commit,
the logic around these flags assumes everything in the list is a flag,
but ChanStatusDefault is not.

It turns out we properly special case that if the channel is in
ChanStatusDefault that we only return ChanStatusDefault. However, if any
of the bits are set we would always report ChanStatusDefault since
status&0 == 0. This fixed simply by removing ChanStatusDefault from the
list since we only need the list to express non-default status flags.
2020-04-10 16:11:44 -07:00
Conner Fromknecht
f71cc951fd
channeldb/channel: fix HasChanStatus for ChanStatusDefault
This commit resovles a lingering issue w/in the codebase wrt how the
ChannelStatus flags are defined. Currently ChannelStatus is improperly
used to define a bit field and the individual flags themselves. As a
result, HasChanStatus accepts queries on particular status (combinations
of flags) and individual flags themselves.

This is an issue because the way HasChanStatus computes whether the
channel has a particular status assumes the provided inputs are all
flags (or at least combinations of flags).

However, ChanStatusDefault is simply the absence of any other flag.
Hence, HasChanStatus will always return true when querying for
ChanStatusDefault because status&0 == 0 is always true.

Longer term we should should consider splitting these definitions into
flags and particular states, and change the way construct or operate on
them, but for now I've just special-cased this one value. Fortunately,
we don't query HasChannelStatus w/ ChanStatusDefault anywhere in the
codebase so we dodge a bullet here, but it'd be nice to have some
greater assurances moving forward.
2020-04-10 16:05:12 -07:00
Joost Jager
cb4cd49dc8
routing: map insufficient local bandwidth error to no path
With mpp it isn't possible anymore for findPath to determine that there
isn't enough local bandwidth. The full payment amount isn't known at
that point.

In a follow-up, this payment outcome can be reintroduced on a higher
level (payment lifecycle).
2020-04-09 08:20:35 +02:00
Joost Jager
278915e598
channeldb: return updated payment on attempt update
Similar to what is done for SettleAttempt.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2020-04-08 08:54:01 +02:00
Olaoluwa Osuntokun
7e6f3ece23
Merge pull request #3960 from bitromortac/listpayments-pagination
channeldb+lnrpc+lncli: listpayments pagination support
2020-04-07 17:06:37 -07:00
Conner Fromknecht
310e20f8b7
invoices/invoiceregistry: remove debug spew 2020-04-07 11:54:29 -07:00
bitromortac
d5dd48fa71 channeldb/test: unit tests for payments query
Adds tests for the payments query, where different edge cases for
the index offsets and normal and reversed orders are tested.
2020-04-07 07:03:21 +02:00
bitromortac
4c5e8aec41 channeldb: add payments query
Adds a PaymentsQuery struct, which contains parameters to restrict the
response of QueryPayments, returning a PaymentsQuerySlice with the
payments query result. The behavior of this api is the same as
the QueryInvoices one.
2020-04-07 07:03:21 +02:00
bitromortac
4800a84122 channeldb: export sequenceNum in MPPayment
Exports sequenceNum in MPPayment for later use
in the rpcserver.
2020-04-07 07:03:21 +02:00
Olaoluwa Osuntokun
92ee49767f
Merge pull request #3970 from halseth/amp-router-mvp-2020
MPP: Enable MultiPathPayments for payment lifecycle
2020-04-03 13:53:19 -07:00
Conner Fromknecht
9385b8cdc6
channeldb+lnwallet: move ActiveHtlcs calc to OpenChannel 2020-04-02 17:39:10 -07:00
Johan T. Halseth
864e64e725
channeldb: validate MPP options when registering attempts
We add validation making sure we are not trying to register MPP shards
for non-MPP payments, and vice versa. We also add validtion of total
sent amount against payment value, and matching MPP options.

We also add methods for copying Route/Hop, since it is useful to use
for modifying the route amount in the test.
2020-04-02 19:31:23 +02:00
Johan T. Halseth
9a1ec950bd
channeldb/payments: extract common info fetch into fetchCreationInfo 2020-04-02 19:29:15 +02:00
Johan T. Halseth
7b318a4be7
routing/payment_lifecycle+channeldb: enable multi shard send
This commit finally enables MP payments within the payment lifecycle
(used for SendPayment). This is done by letting the loop launch shards
as long as there is value remaining to send, inspecting the outcomes for
the sent shards when the full payment amount has been filled.

The method channeldb.MPPayment.SentAmt() is added to easily look up how
much value we have sent for the payment.
2020-04-02 19:29:15 +02:00
Johan T. Halseth
3610824abd
channeldb/payment_control_test: add TestPaymentControlMultiShard 2020-04-02 19:29:14 +02:00
Johan T. Halseth
70202be580
channeldb: make database logic MPP compatible
This commit redefines how the control tower handles shard and payment
level settles and failures. We now consider the payment in flight as
long it has active shards, or it has no active shards but has not
reached a terminal condition (settle of one of the shards, or a payment
level failure has been encountered).

We also make it possible to settle/fail shards regardless of the payment
level status (since we must allow late shards recording their status
even though we have already settled/failed the payment).

Finally, we make it possible to Fail the payment when it is already
failed. This is to allow multiple concurrent shards that reach terminal
errors to mark the payment failed, without havinng to synchronize.
2020-04-02 19:29:14 +02:00
Johan T. Halseth
f6c97daf0c
channeldb/payment_control_test: simplify HTLC status assertion 2020-04-02 10:24:36 +02:00
Johan T. Halseth
7b5c10814b
routing/payment_lifecycle+channeldb: collect existing outcome first
To move towards how we will handle existing attempt in case of MPP
(collecting their outcome will be done in separate goroutines separate
from the payment loop), we move to collect their outcome first.

To easily fetch HTLCs that are still not resolved, we add the utility
method InFlightHTLCs to channeldb.MPPayment.
2020-04-02 10:24:35 +02:00
Johan T. Halseth
9bcf41d401
channeldb: make FailureReason Error() 2020-04-02 10:24:34 +02:00
nsa
f757bf48bd channeldb: use RemoteCommitment in NextLocalHtlcIndex
This commit changes the fallback in NextLocalHtlcIndex to
RemoteCommitment since the LocalHtlcIndex field lags behind
on the LocalCommitment. Without this bug fix, open circuits
would get prematurely trimmed, resulting in more erroneous
logs. A test case is included to check that the fix works.
2020-03-25 11:49:16 -04:00
Olaoluwa Osuntokun
421d73b72e
channeldb: add new frozen channel type bit and thaw height
In this commit, we add a new channel type bit: a frozen channel. A
frozen channel is one that can only be cooperatively closed by the
responder, but not the initiator. This channel type is useful for
certain classes of channel factory like protocols. We then add a new key
on the channel bucket level to store the height after which this
restriction no longer applies.
2020-03-23 16:56:48 -07:00
carla
41355756a1
channeldb: add balance at height lookup
Add a balance at height lookup function which can be used to
obtain local/remote balance at a given height. The current in memory
commits and revocation log are used to source this information.
2020-03-19 13:20:17 +02:00
Olaoluwa Osuntokun
a4e39906b1
build: silence new linter errors, tidy modules
The explicit `bbolt` dep is gone, as we depend on `kvdb`, which is
actually `walletdb`, which has its own module that defines the proper
`bbolt` version.
2020-03-18 19:35:29 -07:00
Olaoluwa Osuntokun
f0911765af
channeldb: convert to uniformly use new kvdb abstractions
In this commit, we migrate all the code in `channeldb` to only reference
the new `kvdb` package rather than `bbolt` directly.

In many instances, we need to add two version to fetch a bucket as both
read and write when needed. As an example, we add a new
`fetchChanBucketRw` function. This function is identical to
`fetchChanBucket`, but it will be used to fetch the main channel bucket
for all _write_ transactions. We need a new method as you can pass a
write transaction where a read is accepted, but not the other way around
due to the stronger typing of the new `kvdb` package.
2020-03-18 19:34:49 -07:00
Olaoluwa Osuntokun
fc808ac538
channeldb/kvdb: create new package for an abstract kv store
In this commit, we create a new package `kvdb`, which is meant to serve
as the basis for any future database abstractions within `lnd`. Rather
than directly use the `walletdb` package (which we base off of), we
instead use a series of type-aliases to re-type the fundamental
types/interfaces of the `walletdb` package. This lets us type
`kvdb.RwTx` instead of `walletdb.ReadWriteTransaction` everywhere.
Additionally, our usage of type-aliases is also intended to create an
easy pathway in the future wherein we can gradually re-defined or
re-implement these types to wean off of the `walletdb` package.
2020-03-18 19:34:45 -07:00
Olaoluwa Osuntokun
3dda93e30d
Merge pull request #3821 from halseth/pluggable-anchors-lnwallet
[anchor] pluggable anchor commitments
2020-03-09 19:49:05 -07:00
Joost Jager
866623e84b
channeldb/migration13: migrate to mpp structure
This commit migrates the payments in the database to a new structure
that allows for multiple htlcs per payments. The migration introduces a
new sub-bucket that contains a list of htlcs and moves the old single
htlc into that.
2020-03-09 18:31:39 +01:00
Joost Jager
4cea2d5213
channeldb/migtest: add migration test tools
This commit adds test helper code to dump, restore and verify the
low-level bbolt database structure.
2020-03-09 18:31:37 +01:00
Joost Jager
f86e68a1a2
channeldb+routing: store full htlc failure reason
This commit extends the htlc fail info with the full failure reason that
was received over the wire. In a later commit, this info will also be
exposed on the rpc interface. Furthermore it serves as a building block
to make SendToRoute reliable across restarts.
2020-03-09 18:31:35 +01:00
Joost Jager
48c0e42c26
channeldb+routing: store all payment htlcs
This commit converts the database structure of a payment so that it can
not just store the last htlc attempt, but all attempts that have been
made. This is a preparation for mpp sending.

In addition to that, we now also persist the fail time of an htlc. In a
later commit, the full failure reason will be added as well.

A key change is made to the control tower interface. Previously the
control tower wasn't aware of individual htlc outcomes. The payment
remained in-flight with the latest attempt recorded, but an outcome was
only set when the payment finished. With this commit, the outcome of
every htlc is expected by the control tower and recorded in the
database.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2020-03-09 18:31:33 +01:00
Johan T. Halseth
af68ff1640
lnwallet: add anchor commitmenttype
With this commitment type, we'll add extra anchor outputs to the
commitment transaction if the anchor channel type is active.
2020-03-09 12:10:59 +01:00
Johan T. Halseth
b25f2fa94b
channeldb: define AnchorOutputsBit channel type 2020-03-09 12:10:58 +01:00
Joost Jager
c357511051
channeldb/migtest: remove channeldb dependency
Removes this unnecessary dependency allowing migration code to use
utility functions from channeldb/migtest.
2020-03-09 11:43:42 +01:00
Joost Jager
4c74c0817e
channeldb/test: extend payment control test with failed attempt 2020-03-09 11:43:40 +01:00
Joost Jager
fa3a762a2c
channeldb: add error return value to fetchPaymentStatus
Preparation for when we need to return errors in a next commit.
2020-03-09 11:43:38 +01:00
Joost Jager
cc5e18c487
channeldb: isolate duplicate payments
Duplicate payments is legacy that we keep alive for accounting purposes.
This commit isolates the deserialization logic for duplicate payments in
its own file, so that regular payment logic and db structure can evolve
without needing to handle/migrate the legacy data.
2020-03-09 11:43:30 +01:00