Commit Graph

11033 Commits

Author SHA1 Message Date
Conner Fromknecht
fc9af92dd9
channeldb: only accept/settle non-empty HTLC sets
Prior to AMP, there could only be one HTLC set. Now that there can be
multiple, we introduce the inherent risk that we might be persisting a
settle/accept of the wrong HTLC set. To migitigate this risk, we add a
check to assert that an invoice can only be transitioned into an
Accepted or Settled state if the specified HTLC set is non-empty, i.e.
has accepted HTLCs.

To do this check, we unroll the loops in UpdateInvoice to first process
any added or canceled HTLCs. This ensures that the set of accepted HTLCs
is up-to-date when we got to transition the invoice into a new state, at
which point we can accurately check that the HTLC set is non-empty.
Previously this sort of check wasn't possible because a newly added HTLC
wouldn't be populated on the invoice when going to call
updateInvoiceState.

Once the invoice-level transition checks have completed, we complete a
final pass to move the HTLCs into their final states and recompute the
amount paid.

With this refactor, it is now possible to make stronger assertions
about the invoice and the state transitions being made by the
invoiceregistry before those changes are ultimately persisted.
2021-03-03 16:28:02 -08:00
Conner Fromknecht
e1b0fe5e98
channeldb/invoices: update AMP invoice htlcs 2021-03-03 16:28:02 -08:00
Conner Fromknecht
fad25f3f26
channeldb/invoices: add invoices to set id index on update 2021-03-03 16:28:01 -08:00
Conner Fromknecht
7c0d1e0093
channeldb/invoice: map zero-value timestamps to 0
Mainly affects ResolveTime which can be 0 before its settled.
2021-03-03 16:26:43 -08:00
Conner Fromknecht
da049ebcf7
channeldb: ensure invoices can't be added with HTLCs 2021-03-03 16:26:43 -08:00
Conner Fromknecht
b2234703f6
invoices+channeldb: add HTLCSet method to invoice 2021-03-03 16:26:43 -08:00
Conner Fromknecht
7f05c9d3bb
channeldb/invoice: persist optional AMP fields on InvoiceHTLC 2021-03-03 16:26:43 -08:00
Conner Fromknecht
464dff09ac
invoices: expose AMPRecord() method on Payload 2021-03-03 16:26:39 -08:00
Olaoluwa Osuntokun
055db6b9ac
Merge pull request #5069 from Roasbeef/v12-99
build: bump version to v0.12.99-beta
2021-03-03 16:24:51 -08:00
Olaoluwa Osuntokun
75eca3a595
Merge pull request #4923 from halseth/the-great-tlv-wire-migration
The great tlv wire migration of 2021
2021-03-03 16:24:31 -08:00
Olaoluwa Osuntokun
6c3d3711f6
build: bump version to v0.12.99-beta
In this commit we bump the version to v0.12.99-beta to reflect that the
master branch is a super set of the recently released v0.12.1-beta.
2021-03-03 15:49:22 -08:00
Olaoluwa Osuntokun
660d37a335
Merge pull request #5049 from Crypt-iQ/commitFee_check_0222
lnd+lnwallet: make capacity check stricter by adding fee
2021-02-24 19:01:34 -08:00
Johan T. Halseth
6842c0ba8c
lnwire: convert the delivery addr in [Open+Accept]Channel to a TLV type in ExtraData
In this commit, we convert the delivery address in the open and accept
channel methods to be a TLV type. This works as an "empty" delivery
address is encoded using a two zero bytes (uint16 length zero), and a
tlv type of 0 is encoded in the same manner (byte for type, byte for
zero length). This change allows us to easily extend these messages in
the future, in a uniform manner.

When decoding the message we snip the bytes from the read TLV data.
Similarly, when encoding we concatenate the TLV record for the shutdown
script with the rest of the TLV data.
2021-02-24 17:40:08 +01:00
Olaoluwa Osuntokun
9a6bb19770
lnwire: prep wire messages for TLV extensions
Messages:
- UpdateFulfillHTLC
- UpdateFee
- UpdateFailMalformedHTLC
- UpdateFailHTLC
- UpdateAddHTLC
- Shutdown
- RevokeAndAck
- ReplyShortChanIDsEnd
- ReplyChannelRange
- QueryShortChanIDs
- QueryChannelRange
- NodeAnnouncement
- Init
- GossipTimestampRange
- FundingSigned
- FundingLocked
- FundingCreated
- CommitSig
- ClosingSigned
- ChannelUpdate
- ChannelReestablish
- ChannelAnnouncement
- AnnounceSignatures

lnwire: update quickcheck tests, use constant for Error

multi: update unit tests to pass deep equal assertions with messages

In this commit, we update a series of unit tests in the code base to now
pass due to the new wire message encode/decode logic. In many instances,
we'll now manually set the extra bytes to an empty byte slice to avoid
comparisons that fail due to one message having an empty byte slice and
the other having a nil pointer.
2021-02-24 17:31:55 +01:00
Olaoluwa Osuntokun
a603ac4938
lnwire: create new ExtraOpaqueData type for parsing TLV extensions
In this commit, we create a new `ExtraOpaqueData` based on the field
with the same name that's present in all the announcement related
messages. In later commits, we'll embed this new type in each message,
so we'll have a generic way to add/parse TLV extensions from messages.
2021-02-24 17:31:55 +01:00
Olaoluwa Osuntokun
dd6f0ba931
discovery+lnwire: remove embedding within ReplyChannelRange
In order to prep for allowing TLV extensions for the `ReplyChannelRange`
and `QueryChannelRange` messages, we'll need to remove the struct
embedding as is. If we don't remove this, then we'll attempt to decode
TLV extensions from both the embedded and outer struct.

All relevant call sites have been updated to reflect this minor change.
2021-02-24 17:31:55 +01:00
Johan T. Halseth
466c079bbe
channeldb/migration21: add body and tests for migration 21
In this commit, we migrate all wire messages in the database from the
`legacy` to the `current` encoding.

This affects the way we write the `CommitDiff` and `LogUpdates` struct
to disk. We also need to migrate the network results bucket in the
switch as it includes a wire message without a length prefix.
2021-02-24 17:31:55 +01:00
Johan T. Halseth
d4e90eae83
channeldb/migration21: add new serialization to package 'current'
We copy the new serialization togic to a new package 'current' and
export the methods we will need from the migration. This ensures that we
have isolation between the legacy and current serialization methods,
cleanly inidcating what type we are using during the migration.
2021-02-24 17:31:49 +01:00
Johan T. Halseth
db66fef6cc
channeldb+htlcswitch: write wire messages using length prefix
In this commit, we modify the way we write wire messages across the
entire database. We'll now ensure that we always write wire messages
with a length prefix. We update the `codec.go` file to always write a 2
byte length prefix, this affects the way we write the `CommitDiff` and
`LogUpdates` struct to disk, and the network results bucket in the
switch as it includes a wire message.
2021-02-24 17:15:27 +01:00
Johan T. Halseth
4133b4d04e
channeldb/migration21: copy over legacy types+codec for migration21
The legacy encoding depends on the lnwire21 version of lnwire, so it will
let us change lnwire after the migration. To make sure it is separated
from the new encoding, we add it to a new package 'legacy'.

We also put common types in a new package 'common', which will house
types that won't change during the migration, and can be used by both
legacy and current serialization code.
2021-02-24 17:15:21 +01:00
Johan T. Halseth
90d36dbdd4
channeldb: rename Encode/Decode to serializeLogUpdate/deserializeLogUpdate
To more easily use different version of it post-/pre-migration, we
rename the method and make it take the LogUpdate as an argument.
2021-02-24 14:34:57 +01:00
Johan T. Halseth
7569cca19b
channeldb/migration: copy current lnwire to migration dir
To avoid code changing underneath the static migrations, we copy the
lnwire dependency in its current form into the migration directory.

Ideally the migrations doesn't depend on any code that might change,
this takes us a step closer.
2021-02-24 14:34:57 +01:00
Olaoluwa Osuntokun
c6a8816043
channeldb: mark ApplyMigration as a test helper
With this change, errors from migrations will have the proper local line
number.
2021-02-24 14:34:57 +01:00
Johan T. Halseth
db23e34f74
Merge pull request #3191 from halseth/deleteallpayments-failedonly
Add option for deleting failed-only payments.
2021-02-22 21:40:16 +01:00
eugene
bc238ee84c
lnd+lnwallet: make capacity check stricter by adding fee
Also patches breacharbiter tests by using the correct commitment
fee and balances.
2021-02-22 12:07:21 -05:00
Johan T. Halseth
59f41e518d
channeldb: add failedHtlcsOnly option to DeletePayments 2021-02-22 12:50:25 +01:00
Johan T. Halseth
6eb42aec0a
channeldb/payments: use only Read bucket when iterating 2021-02-22 12:50:25 +01:00
Johan T. Halseth
7e3738d773
rpcserver+channeldb: delete only failed payments if requested 2021-02-22 12:50:25 +01:00
Johan T. Halseth
0ef2ca06c1
lnrpc: add failed_payments_only and failed_htlcs_only option to DeleteAllPayments 2021-02-22 12:50:24 +01:00
Johan T. Halseth
82fe6f5f59
channeldb/payments_test: handle that dupe payments will always have StatusSuccess
Legacy duplicate payments would only be migrated over if they had
succeeded. Alter the test to reflect this.
2021-02-22 12:50:24 +01:00
Conner Fromknecht
c9ed5927f6
Merge pull request #5038 from cfromknecht/debug-unit
make: add unit-debug target
2021-02-19 14:01:31 -08:00
Olaoluwa Osuntokun
203a669578
Merge pull request #5043 from cfromknecht/reduce-concurrent-router-validation
routing: dial back max concurrent block fetches
2021-02-18 16:36:57 -08:00
Olaoluwa Osuntokun
7f68e961e6
Merge pull request #5046 from cfromknecht/ws-close-whitelist
lntest/itest: add websocket close to error whitelist
2021-02-18 16:27:35 -08:00
Conner Fromknecht
17489ba6a3
lntest/itest: add websocket close to error whitelist 2021-02-18 12:14:50 -08:00
Conner Fromknecht
250bc8560e
routing: avoid modifying AssumeChannelValid in unit tests
This produces a race condition when reading AssumeChannelValid from a
different goroutine. Instead we isolate the test cases and initial
AssumeChannelValid properly.
2021-02-17 22:43:24 -08:00
Conner Fromknecht
f7c5236bf6
routing: dial back max concurrent block fetches
This commit reduces the number of concurrent validation operations the
router will perform when fully validating the channel graph. Reports
from several users indicate that GetInfo would hang for several minutes,
which is believed to be caused by attempting to validate massive amounts
of channels in parallel. This commit returns the limit back to its
original state before adding the batched gossip improvements.

We keep the 1000 concurrent validation request limit for
AssumeChannelValid, since we don't fetch blocks in that case. This
allows us to still keep the performance benefits on mobile/low-resource
devices.
2021-02-17 18:17:09 -08:00
Conner Fromknecht
1c75d92929
Merge pull request #5037 from guggero/make-docker-release
release: fix golang version issue in docker-release and shasum issue in verification script
2021-02-17 10:45:47 -08:00
Oliver Gugger
591954ff61
scripts: detect whether sha256sum or shasum is available
The shasum command isn't available in Alpine linux while the sha256sum
command isn't available on MacOS. We add a simple switch that tries to
detect which one is available.
2021-02-17 18:11:42 +01:00
Oliver Gugger
fdbd4da771
make: use make for docker-release
To fix an issue where the golang version would be picked up from the
host system if the docker-release command was used, we switch over to
using make inside of the container as well instead of feeding the
parameters into the release script manually.
We only pass in the flags that we might actually want to overwrite.
2021-02-17 18:11:42 +01:00
Conner Fromknecht
7f9dcc143f
make: add unit-debug target 2021-02-17 08:47:59 -08:00
Conner Fromknecht
db5af6fc14
Merge pull request #5017 from Roasbeef/default-max-parts
routing: if MaxShardAmt is set, then use that as a ceiling for our splits, use default of 16 for MaxParts
2021-02-16 09:43:34 -08:00
Olaoluwa Osuntokun
db69331db9
cmd/lncli: add max shard size parsing for payment commands 2021-02-15 19:31:58 -08:00
Olaoluwa Osuntokun
b73a6e2c61
routing: if MaxShardAmt is set, then use that as a ceiling for our splits
In this commit, we thread through the necessary state to allow users to
set a max shard amount. If this value is set, then this'll effectively
serve as a ceiling for all our split attempts. If we need to split,
we'll first try to use `paymentAmt/2`, if that's bigger than
`MaxShardAmt, then we'll use the latter instead.

Ideally in the future we have a dynamic way to automatically set both
the `MaxShardAmt` as well as `MaxParts` for users. Until then exposing
these two new fields will allow us to experiment with setting them
automatically using the RPC interface, and also give users a bit more
control over how we attempt to route payments, akin to coin control for
on-chain payments.

Fixes #4730
2021-02-15 19:31:52 -08:00
Olaoluwa Osuntokun
7398e59927
lnrpc/routerrpc+routing: add new MaxShardAmt field to LightningPayment 2021-02-15 19:31:49 -08:00
Olaoluwa Osuntokun
c70e53c791
lnrpc/routerrpc: add new max_shard_size_msat field to SendPaymentRequest 2021-02-15 19:31:46 -08:00
Olaoluwa Osuntokun
d5cf6043e6
lnrpc/routerrpc: use new default value of 16 for maxparts
In this commit, we raise the default value for the `MaxParts` field from
1 to 16. This change was motivated by the fact that many users either
forget, or don't even know this field is there in the first place. A
value of 16 was chosen rather arbitraliy (other than power of 2). In
the future, we should tune this value based on the expected number of
payment attempts for a given payment amount.
2021-02-15 19:31:43 -08:00
Conner Fromknecht
8c06eb57c5
Merge pull request #5026 from cfromknecht/init-mock-fee-estimator
htlcswitch: init mockFeeEstimator in other LinkChannelConfigs
2021-02-15 13:03:02 -08:00
Conner Fromknecht
4fcc82251e
Merge pull request #5016 from cfromknecht/extract-channel-whitelist
lntest: update error log whitelist
2021-02-15 12:50:33 -08:00
Conner Fromknecht
fd3dd2cfff
Merge pull request #5023 from cfromknecht/min-required-sigs-five
scripts/verify-install: require 5 of 7 signatures before accepting
2021-02-15 12:17:36 -08:00
Conner Fromknecht
774d29d890
htlcswitch: init mockFeeEstimator in other LinkChannelConfigs
If the tests don't execute quick enough, the link will try to sample the
network fee and cause a panic. This happens semi-regularly on travis.
2021-02-15 12:14:32 -08:00