Commit Graph

11015 Commits

Author SHA1 Message Date
Johan T. Halseth
d4e90eae83
channeldb/migration21: add new serialization to package 'current'
We copy the new serialization togic to a new package 'current' and
export the methods we will need from the migration. This ensures that we
have isolation between the legacy and current serialization methods,
cleanly inidcating what type we are using during the migration.
2021-02-24 17:31:49 +01:00
Johan T. Halseth
db66fef6cc
channeldb+htlcswitch: write wire messages using length prefix
In this commit, we modify the way we write wire messages across the
entire database. We'll now ensure that we always write wire messages
with a length prefix. We update the `codec.go` file to always write a 2
byte length prefix, this affects the way we write the `CommitDiff` and
`LogUpdates` struct to disk, and the network results bucket in the
switch as it includes a wire message.
2021-02-24 17:15:27 +01:00
Johan T. Halseth
4133b4d04e
channeldb/migration21: copy over legacy types+codec for migration21
The legacy encoding depends on the lnwire21 version of lnwire, so it will
let us change lnwire after the migration. To make sure it is separated
from the new encoding, we add it to a new package 'legacy'.

We also put common types in a new package 'common', which will house
types that won't change during the migration, and can be used by both
legacy and current serialization code.
2021-02-24 17:15:21 +01:00
Johan T. Halseth
90d36dbdd4
channeldb: rename Encode/Decode to serializeLogUpdate/deserializeLogUpdate
To more easily use different version of it post-/pre-migration, we
rename the method and make it take the LogUpdate as an argument.
2021-02-24 14:34:57 +01:00
Johan T. Halseth
7569cca19b
channeldb/migration: copy current lnwire to migration dir
To avoid code changing underneath the static migrations, we copy the
lnwire dependency in its current form into the migration directory.

Ideally the migrations doesn't depend on any code that might change,
this takes us a step closer.
2021-02-24 14:34:57 +01:00
Olaoluwa Osuntokun
c6a8816043
channeldb: mark ApplyMigration as a test helper
With this change, errors from migrations will have the proper local line
number.
2021-02-24 14:34:57 +01:00
Johan T. Halseth
db23e34f74
Merge pull request #3191 from halseth/deleteallpayments-failedonly
Add option for deleting failed-only payments.
2021-02-22 21:40:16 +01:00
Johan T. Halseth
59f41e518d
channeldb: add failedHtlcsOnly option to DeletePayments 2021-02-22 12:50:25 +01:00
Johan T. Halseth
6eb42aec0a
channeldb/payments: use only Read bucket when iterating 2021-02-22 12:50:25 +01:00
Johan T. Halseth
7e3738d773
rpcserver+channeldb: delete only failed payments if requested 2021-02-22 12:50:25 +01:00
Johan T. Halseth
0ef2ca06c1
lnrpc: add failed_payments_only and failed_htlcs_only option to DeleteAllPayments 2021-02-22 12:50:24 +01:00
Johan T. Halseth
82fe6f5f59
channeldb/payments_test: handle that dupe payments will always have StatusSuccess
Legacy duplicate payments would only be migrated over if they had
succeeded. Alter the test to reflect this.
2021-02-22 12:50:24 +01:00
Conner Fromknecht
c9ed5927f6
Merge pull request #5038 from cfromknecht/debug-unit
make: add unit-debug target
2021-02-19 14:01:31 -08:00
Olaoluwa Osuntokun
203a669578
Merge pull request #5043 from cfromknecht/reduce-concurrent-router-validation
routing: dial back max concurrent block fetches
2021-02-18 16:36:57 -08:00
Olaoluwa Osuntokun
7f68e961e6
Merge pull request #5046 from cfromknecht/ws-close-whitelist
lntest/itest: add websocket close to error whitelist
2021-02-18 16:27:35 -08:00
Conner Fromknecht
17489ba6a3
lntest/itest: add websocket close to error whitelist 2021-02-18 12:14:50 -08:00
Conner Fromknecht
250bc8560e
routing: avoid modifying AssumeChannelValid in unit tests
This produces a race condition when reading AssumeChannelValid from a
different goroutine. Instead we isolate the test cases and initial
AssumeChannelValid properly.
2021-02-17 22:43:24 -08:00
Conner Fromknecht
f7c5236bf6
routing: dial back max concurrent block fetches
This commit reduces the number of concurrent validation operations the
router will perform when fully validating the channel graph. Reports
from several users indicate that GetInfo would hang for several minutes,
which is believed to be caused by attempting to validate massive amounts
of channels in parallel. This commit returns the limit back to its
original state before adding the batched gossip improvements.

We keep the 1000 concurrent validation request limit for
AssumeChannelValid, since we don't fetch blocks in that case. This
allows us to still keep the performance benefits on mobile/low-resource
devices.
2021-02-17 18:17:09 -08:00
Conner Fromknecht
1c75d92929
Merge pull request #5037 from guggero/make-docker-release
release: fix golang version issue in docker-release and shasum issue in verification script
2021-02-17 10:45:47 -08:00
Oliver Gugger
591954ff61
scripts: detect whether sha256sum or shasum is available
The shasum command isn't available in Alpine linux while the sha256sum
command isn't available on MacOS. We add a simple switch that tries to
detect which one is available.
2021-02-17 18:11:42 +01:00
Oliver Gugger
fdbd4da771
make: use make for docker-release
To fix an issue where the golang version would be picked up from the
host system if the docker-release command was used, we switch over to
using make inside of the container as well instead of feeding the
parameters into the release script manually.
We only pass in the flags that we might actually want to overwrite.
2021-02-17 18:11:42 +01:00
Conner Fromknecht
7f9dcc143f
make: add unit-debug target 2021-02-17 08:47:59 -08:00
Conner Fromknecht
db5af6fc14
Merge pull request #5017 from Roasbeef/default-max-parts
routing: if MaxShardAmt is set, then use that as a ceiling for our splits, use default of 16 for MaxParts
2021-02-16 09:43:34 -08:00
Olaoluwa Osuntokun
db69331db9
cmd/lncli: add max shard size parsing for payment commands 2021-02-15 19:31:58 -08:00
Olaoluwa Osuntokun
b73a6e2c61
routing: if MaxShardAmt is set, then use that as a ceiling for our splits
In this commit, we thread through the necessary state to allow users to
set a max shard amount. If this value is set, then this'll effectively
serve as a ceiling for all our split attempts. If we need to split,
we'll first try to use `paymentAmt/2`, if that's bigger than
`MaxShardAmt, then we'll use the latter instead.

Ideally in the future we have a dynamic way to automatically set both
the `MaxShardAmt` as well as `MaxParts` for users. Until then exposing
these two new fields will allow us to experiment with setting them
automatically using the RPC interface, and also give users a bit more
control over how we attempt to route payments, akin to coin control for
on-chain payments.

Fixes #4730
2021-02-15 19:31:52 -08:00
Olaoluwa Osuntokun
7398e59927
lnrpc/routerrpc+routing: add new MaxShardAmt field to LightningPayment 2021-02-15 19:31:49 -08:00
Olaoluwa Osuntokun
c70e53c791
lnrpc/routerrpc: add new max_shard_size_msat field to SendPaymentRequest 2021-02-15 19:31:46 -08:00
Olaoluwa Osuntokun
d5cf6043e6
lnrpc/routerrpc: use new default value of 16 for maxparts
In this commit, we raise the default value for the `MaxParts` field from
1 to 16. This change was motivated by the fact that many users either
forget, or don't even know this field is there in the first place. A
value of 16 was chosen rather arbitraliy (other than power of 2). In
the future, we should tune this value based on the expected number of
payment attempts for a given payment amount.
2021-02-15 19:31:43 -08:00
Conner Fromknecht
8c06eb57c5
Merge pull request #5026 from cfromknecht/init-mock-fee-estimator
htlcswitch: init mockFeeEstimator in other LinkChannelConfigs
2021-02-15 13:03:02 -08:00
Conner Fromknecht
4fcc82251e
Merge pull request #5016 from cfromknecht/extract-channel-whitelist
lntest: update error log whitelist
2021-02-15 12:50:33 -08:00
Conner Fromknecht
fd3dd2cfff
Merge pull request #5023 from cfromknecht/min-required-sigs-five
scripts/verify-install: require 5 of 7 signatures before accepting
2021-02-15 12:17:36 -08:00
Conner Fromknecht
774d29d890
htlcswitch: init mockFeeEstimator in other LinkChannelConfigs
If the tests don't execute quick enough, the link will try to sample the
network fee and cause a panic. This happens semi-regularly on travis.
2021-02-15 12:14:32 -08:00
Conner Fromknecht
95eadfee2f
scripts/verify-install.sh: combine final SUCCESS logs 2021-02-15 10:00:42 -08:00
Conner Fromknecht
c03f95a63b
scripts/verify-install: bump min required signatures to 5 2021-02-15 09:59:59 -08:00
Conner Fromknecht
12182d0fc9
Merge pull request #5019 from guggero/detach-sign
release: create and verify detached signatures, fix hashing command on MacOS
2021-02-15 09:48:17 -08:00
Oliver Gugger
85c42b0b79
scripts: add more verbose error messages to verification
We want to be more precise in what exactly went wrong and what the cause
could be.
2021-02-15 10:47:46 +01:00
Oliver Gugger
99ba272822
docs+scripts: switch to detached signatures
Due to a misunderstanding of how the gpg command line options work, we
didn't actually create detached signatures because the --clear-sign
flag would overwrite that. We update our verification script to now only
download the detached signatures and verify them against the main
manifest file.
We also update the signing instructions.
2021-02-15 10:33:20 +01:00
Oliver Gugger
132d23c964
scripts: verify hash length
To make sure we've actually calculated the hash correctly, we make sure
it's 64 characters long.
2021-02-15 10:26:17 +01:00
Oliver Gugger
644424296b
scripts: use shasum instead of sha256sum
Because the sha256sum binary isn't available on MacOS we instead use the
shasum -a 256 command that was used before.
2021-02-15 10:26:15 +01:00
Conner Fromknecht
4a96c2e55a
Merge pull request #5021 from guggero/custom-binary-verify
scripts: allow verification of custom binary
2021-02-12 12:04:49 -08:00
Conner Fromknecht
32507bc835
Merge pull request #5020 from guggero/expand-path
config: clean and expand backup file path
2021-02-12 09:19:39 -08:00
Oliver Gugger
aca93199cf
scripts: allow verification of custom binary
Instead of only allowing the installed versions of lnd and lncli to be
verified, we now also support specifying explicit paths to binaries that
we want to verify.
2021-02-12 13:22:13 +01:00
Oliver Gugger
1a2cb9b975
config: clean and expand backup file path
Because the CleanAndExpandPath function wasn't applied to the backup
file path, any path that contained a tilde for the home directory didn't
work.
2021-02-12 11:14:36 +01:00
Conner Fromknecht
91a3350dd1
lntest: add Block height out of range to whitelist 2021-02-11 17:51:07 -08:00
Conner Fromknecht
cc34af4e57
lntest: add unable to extract ChannelUpdate to whitelist 2021-02-11 17:48:41 -08:00
Conner Fromknecht
76e0388368
Merge pull request #5013 from lightningnetwork/release-pin-go-version
github/workflows: pin exact docker release
2021-02-11 17:24:07 -08:00
Conner Fromknecht
a6b977e760
github/workflows: pin exact docker release 2021-02-11 16:05:32 -08:00
Johan T. Halseth
b52e872878
Merge pull request #4899 from halseth/ignore-local-ann-from-remote
[gossiper] Ignore received ChannelAnnouncements for own channels
2021-02-11 18:39:15 +01:00
Johan T. Halseth
926005aefa
discovery/gossiper: ignore remote ChannelAnnouncement for own channel
To avoid learning about our own channel we have already closed and
removed from our graph, we ignore all ChannelAnns for channels we are
involved in.
2021-02-11 15:13:51 +01:00
Johan T. Halseth
e8f7a11470
gossiper_test: split keys into self/remote
To make it more clear what is local and remote messages, we change to
use `selfKey` only for local messages.
2021-02-11 15:13:51 +01:00