Commit Graph

717 Commits

Author SHA1 Message Date
Joost Jager
f86e68a1a2
channeldb+routing: store full htlc failure reason
This commit extends the htlc fail info with the full failure reason that
was received over the wire. In a later commit, this info will also be
exposed on the rpc interface. Furthermore it serves as a building block
to make SendToRoute reliable across restarts.
2020-03-09 18:31:35 +01:00
Joost Jager
48c0e42c26
channeldb+routing: store all payment htlcs
This commit converts the database structure of a payment so that it can
not just store the last htlc attempt, but all attempts that have been
made. This is a preparation for mpp sending.

In addition to that, we now also persist the fail time of an htlc. In a
later commit, the full failure reason will be added as well.

A key change is made to the control tower interface. Previously the
control tower wasn't aware of individual htlc outcomes. The payment
remained in-flight with the latest attempt recorded, but an outcome was
only set when the payment finished. With this commit, the outcome of
every htlc is expected by the control tower and recorded in the
database.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2020-03-09 18:31:33 +01:00
Johan T. Halseth
af68ff1640
lnwallet: add anchor commitmenttype
With this commitment type, we'll add extra anchor outputs to the
commitment transaction if the anchor channel type is active.
2020-03-09 12:10:59 +01:00
Johan T. Halseth
b25f2fa94b
channeldb: define AnchorOutputsBit channel type 2020-03-09 12:10:58 +01:00
Joost Jager
c357511051
channeldb/migtest: remove channeldb dependency
Removes this unnecessary dependency allowing migration code to use
utility functions from channeldb/migtest.
2020-03-09 11:43:42 +01:00
Joost Jager
4c74c0817e
channeldb/test: extend payment control test with failed attempt 2020-03-09 11:43:40 +01:00
Joost Jager
fa3a762a2c
channeldb: add error return value to fetchPaymentStatus
Preparation for when we need to return errors in a next commit.
2020-03-09 11:43:38 +01:00
Joost Jager
cc5e18c487
channeldb: isolate duplicate payments
Duplicate payments is legacy that we keep alive for accounting purposes.
This commit isolates the deserialization logic for duplicate payments in
its own file, so that regular payment logic and db structure can evolve
without needing to handle/migrate the legacy data.
2020-03-09 11:43:30 +01:00
Joost Jager
c29b74168f
channeldb/test: refactor payment control test
Previously this was tested as a white box. Database access methods were
duplicated as test code and compared to the return value of the code
under test. This approaches leads to brittle test because it relies
heavily on implementation details. This commit changes this and prepares
for additional test coverage being added in later commits.
2020-03-09 11:43:28 +01:00
Johan T. Halseth
bee2380441
channeldb: rename PaymentAttemptInfo to HTLCAttemptInfo
To better distinguish payments from HTLCs, we rename the attempt info
struct to HTLCAttemptInfo. We also embed it into the HTLCAttempt struct,
to avoid having to duplicate this information.

The paymentID term is renamed to attemptID.
2020-03-09 11:43:26 +01:00
Joost Jager
967b4b2dc3
channeldb: remove redundant MPPaymentCreationInfo struct
MPPaymentCreationInfo and PaymentCreationInfo are identical except for
the naming of CreationTime/CreationDate.
2020-03-09 11:43:24 +01:00
carla
11d975bd13
channeldb: save channel status on channel close
Add an optional channel status CloseChannel which will be stored on the
hitsorical channel which is persisted at channel close. This status is
used to set the close initiator for channels that do not complete the
funding flow or we abandon. In follow up commits, this status will be
used to record force and breach closes. The value is written to the
historical channel bucket for diplay over rpc.
2020-02-21 13:33:53 +02:00
carla
d3cb6ad869
channeldb: store close initiator status
This commit adds two new channel statuses which indicate the party that
initatited closing the channel. These statuses are set in conjunction
with the existing commit broadcast status so that we do not need to
migrate existing logic to handle multiple types of closes. This status
is set for locally initiated force closes in this commit because they
follow a similar pattern to cooparative closes, marking the commitment
broadcast then proceeding with tx broadcast. Remote force closes are
added in the following commit, as they are handled differently.
2020-02-21 13:33:49 +02:00
carla
8d632b8022
channeldb: add historical channel bucket lookup 2020-02-21 13:23:30 +02:00
Johan T. Halseth
cdf3a49c54
channeldb: remove unused payments status serialization 2020-02-18 13:01:57 +01:00
carla
ed81c88239
channeldb: replace fetch channels booleans with optional filters
This changes replaces the pending an waiting booleans in fetchChannels
with optional filters which can be more flexibly used. This change
allows filtering of channels without having to reason about the matrix
of possible boolean combinations. A test is added to ensure that the
combinations of these filters act as expected.
2020-02-06 21:39:54 +02:00
carla
c5c2fc27f9
channeldb/test: replace test channel boilerplate createTestChannel
This change replaces test channel creation boilerplate with a
createTestChannel function which can be customized using functional
options.
2020-02-06 21:28:25 +02:00
Joost Jager
1413995ab7
Merge pull request #3872 from joostjager/invalid-sig-fix
htlcswitch+lnwallet+channeldb: invalid sig fix
2020-01-23 21:08:07 +01:00
Joost Jager
b600ecda86
channeldb: inject clock into database
Use our standard clock mock for database time queries.
2020-01-20 11:08:30 +01:00
Conner Fromknecht
51dbdd3b38
multi: rename key_send, key-send and key send to keysend 2020-01-16 18:37:16 -08:00
Joost Jager
82579400b3
lnwallet: restore unsigned acked remote updates
This commit updates the channel state machine to
persistently store remote updates that we have received a
signature for, but that we haven't yet included in a commit
signature of our own.

Previously those updates were only stored in memory and
dropped across restarts. This lead to the production of
an invalid signature and channel force closure. The remote
party expects us to include those updates.
2020-01-15 13:09:35 +01:00
Joost Jager
2d37d341a1
channeldb: extract log update serialization
Extract functionality to methods as a preparation for serializing remote
log updates.
2020-01-15 12:37:24 +01:00
Joost Jager
ea892fce17
channeldb: update UpdateCommitment description
This method is only used to update the local commitment transaction.
Updated comment accordingly.
2020-01-15 12:37:22 +01:00
Andras Banki-Horvath
8af5d8bc73 channeldb: remove unused, test only FetchAllInvoices function
This commit removes channeldb.FetchAllInvoices and changes tests such
that expectation sets are prepared in the test case instead of selected
from the DB.
2020-01-08 16:51:51 +01:00
Andras Banki-Horvath
4136b18e3d channeldb: remove time.Now() from tests 2020-01-08 16:51:51 +01:00
Andras Banki-Horvath
ec6c0689ef channeldb: fix channeldb.InvoiceHTLC deep copy
This commit fixes deep copy of chaneldb.InvoiceHTLC, where previously
the map holding the custom record set wasn't properly copied.
2020-01-08 16:51:51 +01:00
Johan T. Halseth
2c46640dd0
channeldb+lnwallet: note that balance is after subtracting commit fee
It was incorrectly stated that the commitment balance was before
subctracting the commit fee, which led to some confusion.
2020-01-06 11:42:02 +01:00
Joost Jager
1d5844c196
invoices: jit insert key send invoices
This commit adds handling code for the key send custom record. If this
record is present and its hash matches the payment hash, invoice
registry will insert a new invoice into the database "just in time". The
subsequent settle flow is unchanged. The newly inserted invoice is
picked up and settled. Notifications will be broadcast as usual.
2019-12-23 21:51:19 +01:00
Joost Jager
af1b8a549d
channeldb: update payment request comment
This field isn't optional. It was introduced in
5ed31b1030176374d4184fee0b80ae7679e0e355 which was part of release
0.4.0. This release contained breaking database changes, so it is safe
to assume that from there on the field is always populated.

If there would still be empty payment request fields in the wild, users
would also experience issues with ListInvoices. ListInvoices decodes the
payment request.
2019-12-23 21:51:11 +01:00
Joost Jager
6e0cfe579b
Merge pull request #3841 from joostjager/check-max-onion-size
routing: check max routing info size
2019-12-20 14:42:47 +01:00
Joost Jager
95ddab57fa
channeldb: add tx argument for FetchLightningNode
To allow execution within an existing tx.
2019-12-20 10:17:01 +01:00
Joost Jager
5ea63158e0
Merge pull request #3838 from bhandras/i1404
channeldb+invoices: fix what pending invoice means in ChannelDB
2019-12-19 21:02:57 +01:00
Andras Banki-Horvath
fa14ff67ad channeldb+invoices: fix what pending invoice means in ChannelDB
This commits builds on top of PR #3694 to further clarify invoice
state by defining pending invoices as the ones which are not
settled or canceled. Automatic cancellation of expired invoices
makes this possbile. While this change only directly affects
ChannelDB, users of the listinvoices RPC will receive actual
pending invoices when pending_only flag is set.
2019-12-19 17:44:32 +01:00
Andras Banki-Horvath
9b60685961 channeldb: remove math/rand from tests
This commit removes random invoice state selection when testing
FetchAllInvoicesWithPaymentHash.
2019-12-19 17:44:31 +01:00
Joost Jager
37d9ee302c
channeldb: take serialized key to fetch lightning node
This prevents inefficient key conversions in a follow up commit that
change the inner pathfinding loop.
2019-12-19 12:44:00 +01:00
Conner Fromknecht
80802d8e84
channeldb/graph: always populate LightningNode features
Previously we would return nil features when we didn't have a node
announcement or a given node. With this change, we can always assume the
feature vector is populated during pathfinding.
2019-12-18 23:53:05 -08:00
Joost Jager
2ce22d912b
Merge pull request #3763 from joostjager/close-later
cnct: keep open channel data after channel commitment tx confirms
2019-12-17 16:00:20 +01:00
Andras Banki-Horvath
0758b9310e channeldb: adding a unit test for FetchAllInvoicesWithPaymentHash
This commit changes how FetchAllInvoicesWithPaymentHash behaves
when the DB is empty and also adds a unit test to test that
case as well as normal expected behavior.
2019-12-13 17:03:19 +01:00
Andras Banki-Horvath
44f13d1d60 invoices: adding InvoiceExpryWatcher to cancel expired invoices
This commit adds InvoiceExpryWatcher which is a separate class that
receives new invoices (and existing ones upon restart) from InvoiceRegistry
and actively watches their expiry. When an invoice is expired
InvoiceExpiryWatcher will call into InvoiceRegistry to cancel the
invoice and by that notify all subscribers about the state change.
2019-12-13 17:03:08 +01:00
Joost Jager
de2b7b78de
channeldb: custom records sanity check 2019-12-12 17:49:36 +01:00
Joost Jager
d02de70d20
multi: do not use tlv.Record outside wire format handling
This commit prepares for more manipulation of custom records. A list of
tlv.Record types is more difficult to use than the more basic
map[uint64][]byte.

Furthermore fields and variables are renamed to make them more
consistent.
2019-12-12 00:14:58 +01:00
Joost Jager
8b5bb0ac63
record: move CustomRecordSet 2019-12-12 00:12:19 +01:00
Joost Jager
7aa4a7c7fc
channeldb/migration_01_to_11: isolate route structure
Before we change the Hop struct, isolate the code that is used in older
migrations to prevent breaking them.

route.go was taken from commit 6e463c1634061d595953f20813860207e5d485ce
2019-12-12 00:12:17 +01:00
Joost Jager
b2f43858c3
invoices: accept mpp payments 2019-12-11 16:14:49 +01:00
Joost Jager
d8fd6fae23
config+channeldb: describe min_htlc fields more accurately 2019-12-11 00:16:55 +01:00
Joost Jager
5f4bd136cd
invoices: store custom records in invoice database 2019-12-10 06:54:24 +01:00
Conner Fromknecht
f3398c0c0e
Merge pull request #3789 from cfromknecht/coop-close-rpc-status
republish force and coop closes on startup
2019-12-05 10:35:53 -08:00
Joost Jager
91d716be1f
Merge pull request #3770 from joostjager/mpp-prep-registry
channeldb+invoices: prepare invoice db for mpp
2019-12-05 11:38:56 +01:00
Joost Jager
00ef493aa0
channeldb: keep open channel data in historical channel bucket 2019-12-05 09:19:39 +01:00
Olaoluwa Osuntokun
183797f102
Merge pull request #3655 from carlaKC/fundingmgr-optionupfrontshutdown
Add Option Upfront Shutdown
2019-12-04 21:05:24 -08:00
Conner Fromknecht
46990c412c
channeldb/channel: allow storing empty closes
This is preparation for the subsequent commit, allowing us to fix a race
condition in the integration test assertions.
2019-12-04 14:25:29 -08:00
Conner Fromknecht
1c0dc98a7c
channeldb: differentiate force vs coop close 2019-12-04 14:25:28 -08:00
Joost Jager
6c07902a63
channeldb/test: add single htlc cancel test 2019-12-04 14:51:44 +01:00
Joost Jager
00d93ed87b
channeldb: stricter validation of invoice updates 2019-12-04 14:51:42 +01:00
Joost Jager
a4a3c41924
channeldb: split cancel and add htlc updates
Previously the cancel and add actions were combined in a single map.
Nil values implictly signaled cancel actions. This wasn't very obvious.
Furthermore this split prepares for processing the adds and cancels
separately, which is more efficient if there are already two maps.
2019-12-04 14:51:40 +01:00
Joost Jager
915867e90f
invoiceregistry: promote update closure to method
This commit moves the update code into its own function as a preparation
for extending the logic further for mpp.

In order to make this change cleanly, structured result codes are
introduced. This also prepares for a future htlc notifier rpc hook that
reports htlc settle decisions to external applications.

Furthermore the awkward use of errNoUpdate as a way to signal no update
is removed.
2019-12-04 14:51:36 +01:00
Joost Jager
e234f88b82
channeldb: export now function
To allow mocking when testing other packages.
2019-12-04 14:51:31 +01:00
Joost Jager
5d4ceca038
tlv: remove unused error return value
This commit also modifies a previous migration. Because the change is so
limited, we don't consider this a significant risk.
2019-12-04 13:27:29 +01:00
Joost Jager
883f9e5f9a
Merge pull request #3749 from joostjager/extended-routing-failures
routing: local balance check before path finding
2019-12-04 12:17:17 +01:00
Joost Jager
97344af8f3
routing: local balance check 2019-12-04 09:45:07 +01:00
carla
77222d8b69
channeldb: Add upfront shutdown script to OpenChannel
This commit adds fields for upfront shutdown scripts set
by the local and remote peer to the OpenChannel struct.
These values are optional, so they are added with their
own keys in the chanBucket in the DB.
2019-12-03 11:38:29 +02:00
Olaoluwa Osuntokun
6753a02439
channeldb: add new NoFundingTxBit modifier to ChannelType
In this commit, we add a new bit to the existing ChannelType bitfield.
If this bit is set, then it signals that we have the funding transaction
stored on disk. A future change will enable lnd to have the funding
transaction be constructed externally, allowing for things like funding
from a hardware wallet, or a channel created as a sub-branch within an
existing channel factory.
2019-12-02 17:11:16 -06:00
Olaoluwa Osuntokun
d59aba35a0 Merge branch 'refresh-chan-id' 2019-11-27 15:21:42 -06:00
Olaoluwa Osuntokun
5bdb0d3d66
channeldb+lntest: code style fixes 2019-11-27 15:21:28 -06:00
Roei Erez
8b3dd9415e channeldb: refresh channel state within RefreshShortChanID
Refresh channel memory state whenever the short channel id is refreshed.
This is to make the in-memory channel consistent with the disk data.

Fixes #3765.
2019-11-27 15:10:00 -06:00
Conner Fromknecht
3b253e05f6
multi: restructure invoice Terms field
This commit restructures an invoice's ContractTerms to better encompass
the restrictions placed on settling. For instance, the final ctlv delta
and invoice expiry are moved from the main invoice body (where
additional metadata is stored). Additionally, it moves the State field
outside of the terms since it is rather metadata about the invoice
instead of any terms offered to the sender in the payment request.
2019-11-22 02:25:02 -08:00
Conner Fromknecht
6cabea563e
channeldb/invoices: optimize deserializeHtlcs
Instead of allocating a byte slice to read in each htlc's raw TLV
stream, use a LimitReader to truncate the stream to the proper length.
2019-11-22 02:24:44 -08:00
Conner Fromknecht
4c872c438b
channeldb: complete migration 12 for TLV invoices 2019-11-22 02:24:28 -08:00
Conner Fromknecht
76682ad820
channeldb/migration12: add tlv invoice migration tests 2019-11-22 02:24:11 -08:00
Conner Fromknecht
4b358aa2ad
channeldb/migration12: write out invoices using TLV encoding 2019-11-22 02:23:59 -08:00
Conner Fromknecht
0f8048d336
channeldb/migration12: copy invoice deserialization 2019-11-22 02:23:47 -08:00
Olaoluwa Osuntokun
6f9fcfaccc
Merge pull request #3499 from cfromknecht/mpp-payments-rpc
channeldb+rpcserver: expose legacy payments as multi-path payments
2019-11-21 18:38:53 -08:00
Olaoluwa Osuntokun
f6614dd435
channeldb: add new AbandonChannel method 2019-11-20 17:42:49 -08:00
Conner Fromknecht
063f24f2ed
channeldb+routing: expose HTLCs in payment subscriptions
This commit modifies the FetchPayment method to return MPPayment structs
converted from the legacy on-disk format. This allows us to attach the
HTLCs to the events given to clients subscribing to the outcome of an
HTLC.

This commit also bubbles up to the routerrpc/router_server, by
populating HTLCAttempts in the response and extracting the legacy route
field from the HTLCAttempts.
2019-11-19 20:43:38 -08:00
Conner Fromknecht
01cecb1f27
multi: rename to FailureReasonPaymentDetails
Prior name is too long XD
2019-11-19 20:42:21 -08:00
Conner Fromknecht
77602451b8
channeldb: add MPPayment from FetchPayments 2019-11-19 20:41:08 -08:00
Conner Fromknecht
fbd599a2cb
multi: rename Payment.PaymentPreimage to Payment.Preimage
Preliminary step to exposing PaymentPreimage() as a method that can be
shared between legacy payments and mutli-path payments.
2019-11-19 20:40:29 -08:00
Conner Fromknecht
5e27b5022c
multi: remove LocalFeatures and GlobalFeatures 2019-11-08 05:32:00 -08:00
Conner Fromknecht
6d971e5113
routing/route/route: add optional MPP field to Hop
This commit also modifies the Router serialization to persist the MPP
struct when present, and properly restore it when loading from disk.
2019-11-04 14:11:28 -08:00
Olaoluwa Osuntokun
38e313a869
Merge pull request #3633 from joostjager/safe-migrations
channeldb: isolate migrations
2019-11-01 18:28:56 -07:00
Olaoluwa Osuntokun
a73ee28e46
multi: convert the existing channeldb.ChannelType uint8 into a bit field
In this commit, we convert the existing `channeldb.ChannelType` type
into a _bit field_. This doesn't require us to change the current
serialization or interpretation or the type as it is, since all the
current defined values us a distinct bit. This PR lays the ground work
for any future changes that may introduce new channel types (like anchor
outputs), and also any changes that may modify the existing invariants
around channels (if we're the initiator, we always have the funding
transaction).
2019-10-31 16:34:37 -07:00
Joost Jager
4486a06b1a
migration_01_to_11: remove version checking for migration tests 2019-10-31 11:25:07 +01:00
Joost Jager
60503d6c44
channeldb/migration_01_to_11: remove unused code 2019-10-31 07:18:36 +01:00
Joost Jager
f5191440c5
channeldb: initialize migrations logger 2019-10-30 08:56:18 +01:00
Joost Jager
43449ca7a7
channeldb/migration_01_to_11: add references to untested migrations 2019-10-29 11:49:33 +01:00
Joost Jager
6913cd64b6
channeldb: reference migrations in package
This commit removes the migrations from channeldb and references those
in the migrations_01_to_11 package. This creates a one-way dependency on
the migrations. Future changes to channeldb won't be able to break
migrations anymore.
2019-10-24 12:49:40 +02:00
Joost Jager
6e463c1634
channeldb: make copy for migrations
This commit is a direct copy of the complete channeldb package. It only
changes the package declaration at the top of every file. We make this
full copy so that review can be focused on the actual changes made.
Otherwise changes may drown in all the file moves.

Linting for the new package is disabled, as it contains lots of
pre-existing issues.
2019-10-24 12:49:38 +02:00
Olaoluwa Osuntokun
e9d2ad5d73 Revert "channeldb: require minimum db upgrade version"
This reverts commit f1942a4c33dbea7144f756d9cd0139ab1fe8197f.
2019-10-12 14:30:26 -07:00
Olaoluwa Osuntokun
0386e0c297 Revert "channeldb: remove unsupported migrations"
This reverts commit 3ab4c749c6da4c2693b3978c0a0863787c9c3495.
2019-10-12 00:47:45 -07:00
Wilmer Paulino
fa96450db8
channeldb: add freelist sync option modifier 2019-10-04 12:19:41 -04:00
Joost Jager
b58dbb2d70
multi: fix canceled spelling 2019-10-03 17:27:36 +02:00
Joost Jager
ac7c93f544
channeldb: fail migration for accepted hodl invoices 2019-09-28 08:30:33 +02:00
Joost Jager
31f72f6c7d
channeldb: extract before migration func 2019-09-28 08:30:31 +02:00
Joost Jager
f47de09850
channeldb: log error for migration test failures 2019-09-28 08:23:49 +02:00
Olaoluwa Osuntokun
b399203e71
lnwallet: update channel state machine to be aware of tweakless commits
In this commit, we update the channel state machine to be aware of
tweakless commits. In several areas, we'll now check the channel's type
to see if it's `SingleFunderTweakless`. If so, then we'll opt to use the
remote party's non-delay based point directly in the script, skipping
any additional cryptographic operations. Along the way we move the
`validateCommitmentSanity` method to be defined _before_ it's used as is
cutomary within the codebase.

Notably, within the `NewUnilateralCloseSummary` method, we'll now _blank
out_ the `SingleTweak` value if the commitment is tweakless. This
indicates to callers the witness type they should map to, as the value
isn't needed at all any longer when sweeping a non-delay output.

We also update the signing+verification tests to also test that we're
able to properly generate a valid witness for the new tweakless
commitment format.
2019-09-25 18:25:43 -07:00
Olaoluwa Osuntokun
21133e1856
channeldb: define new channel type, SingleFunderTweakless
In this commit, we define a new channel type: SingleFunderTweakless.
We'll use this channel type to denote channels with commitments that
don't tweak the remote party's key in their non-delay output.
2019-09-25 18:25:40 -07:00
Johan T. Halseth
2a6ad6e634
channeldb+lnwallet: don't pass isRestoredChan to ChanSyncMsg
Since we have access to the internal state of the channel, we can
instead get it directly instead of passing it in as a parameter.
2019-09-25 14:04:44 +02:00
Johan T. Halseth
ac0e9b6016
channeldb/channel: add BroadcastedCommitment 2019-09-25 14:04:43 +02:00
Johan T. Halseth
02b2787e44
multi: make MarkCommitmentBroadcasted take closeTx 2019-09-25 14:04:43 +02:00
Johan T. Halseth
a810092e53
channeldb/channel: make putChanStatus take optional extra closures 2019-09-25 14:04:43 +02:00