Commit Graph
7770 Commits
Author SHA1 Message Date
Olaoluwa Osuntokun 7eca7b02a6 Merge pull request #3390 from joostjager/invoice-circuits
channeldb+cnct+invoices: track invoice htlcs
2019-09-04 19:51:37 -07:00
Wilmer Paulino 38c64b2030 Merge pull request #3433 from halseth/atpl-nodescore-tracelogs
[autopilot] Add trace logging during node scoring
2019-09-04 13:04:10 -07:00
Wilmer Paulino d1a4fe7e2e Merge pull request #2088 from wpaulino/lower-mempool-spend-check-timeout
chainntnfs/interface_test: lower mempool spend check timeout
2019-09-04 12:08:06 -07:00
Joost Jager 45cd76e9eb lnrpc+invoicesrpc: report invoice htlcs 2019-09-04 19:20:33 +02:00
Joost Jager d6d9ec6aa5 invoices: replay awareness
Previously the invoice registry wasn't aware of replayed htlcs. This was
dealt with by keeping the invoice accept/settle logic idempotent, so
that a replay wouldn't have an effect.

This mechanism has two limitations:

1. No accurate tracking of the total amount paid to an invoice. The total
amount couldn't just be increased with every htlc received, because it
could be a replay which would lead to counting the htlc amount multiple
times. Therefore the total amount was set to the amount of the first
htlc that was received, even though there may have been multiple htlcs
paying to the invoice.

2. Impossible to check htlc expiry consistently for hodl invoices. When
an htlc is new, its expiry needs to be checked against the invoice cltv
delta. But for a replay, that check must be skipped. The htlc was
accepted in time, the invoice was moved to the accepted state and a
replay some blocks later shouldn't lead to that htlc being cancelled.
Because the invoice registry couldn't recognize replays, it stopped
checking htlc expiry heights when the invoice reached the accepted
state. This prevents hold htlcs from being cancelled after a restart.
But unfortunately this also caused additional htlcs to be accepted on an
already accepted invoice without their expiry being checked.

In this commit, the invoice registry starts to persistently track htlcs
so that replays can be recognized. For replays, an htlc resolution
action is returned early. This fixes both limitations mentioned above.
2019-09-04 19:20:31 +02:00
Joost Jager 53eea09b63 channeldb: add now function
Needed for time control in unit tests.
2019-09-04 19:20:29 +02:00
Joost Jager c8fa51f865 invoices: refactor invoice update callback
This commit refactors the invoice registry accept/settle logic so that
it doesn't rely anymore on a set of error values to indirectly
communicate from the update callback to the main function what action is
required on the htlc.
2019-09-04 19:20:27 +02:00
Joost Jager 144856757d channeldb+invoices: move invoice cancel logic into registry
This commit is a continuation of the centralization of invoice state
transition logic in the invoice registry.
2019-09-04 19:20:25 +02:00
Joost Jager 416bc8c68c channeldb+invoices: move hold invoice settle logic into registry
This commit is a continuation of the centralization of invoice state
transition logic in the invoice registry.
2019-09-04 19:20:23 +02:00
Joost Jager ad3522f1a6 channeldb+invoices: move invoice accept or settle logic into registry
As the logic around invoice mutations gets more complex, the friction
caused by having this logic split between invoice registry and channeldb
becomes more apparent. This commit brings a clearer separation of
concerns by centralizing the accept/settle logic in the invoice
registry.

The original AcceptOrSettle method is renamed to UpdateInvoice because
the update to perform is controlled by the callback.
2019-09-04 19:20:21 +02:00
Joost Jager c1345a4117 multi: use separate cltv expiry field from invoice
Now that the Invoice struct contains the decoded final cltv delta value,
the decoding of payment requests can be removed from the invoice
registry.
2019-09-04 19:20:19 +02:00
Joost Jager 4105142c96 channeldb+invoices: add invoice htlcs
This commit adds a set of htlcs to the Invoice struct and
serializes/deserializes this set to/from disk. It is a preparation for
accurate invoice accounting across restarts of lnd.

A migration is added for the invoice htlcs.

In addition to these changes, separate final cltv delta and expiry
invoice fields are created and populated. Previously it was required
to decode this from the stored payment request. The reason to create
a combined commit is to prevent multiple migrations.
2019-09-04 19:20:17 +02:00
Joost Jager 061b34b924 channeldb: add int64 to codec 2019-09-04 19:20:15 +02:00
Joost Jager 05e6b62cb2 cnct+htlcswitch+invoices: report circuit key to invoice registry
Currently the invoice registry cannot tell apart the htlcs that pay to
an invoice. Because htlcs may also be replayed on startup, it isn't
possible to determine the total amount paid to an invoice.

This commit is a first step towards fixing that. It reports the circuit
keys of htlcs to the invoice registry, which forms the basis for
accurate invoice accounting.
2019-09-04 19:20:13 +02:00
Joost Jager 5871d69bde cnct: convert supplement functions to methods 2019-09-04 19:20:11 +02:00
Joost Jager 4fe846af6e invoices/test: extend hodl invoice test
Add logic to specifically exercise the replay behavior of invoice
registry for hodl invoices.
2019-09-04 19:20:09 +02:00
Joost Jager 43bad4af9f invoices: always check htlc amt with invoice amount
Previously a check was made for accepted and settled invoices against
the paid amount. This opens up a probe vector where an attacker can pay
to an invoice with an amt that is higher than the invoice amount and
find out if the invoice is already paid or not.
2019-09-04 19:20:07 +02:00
Joost Jager 762609a169 channeldb: fix suppressed error 2019-09-04 19:20:05 +02:00
Joost Jager d98a22879f invoices/test: clarify test parameters 2019-09-04 19:20:04 +02:00
Olaoluwa Osuntokun c397a2ea78 Merge pull request #3438 from alrs/fix-cmd-error
cmd/lncli: Fix dropped error
2019-09-03 19:59:30 -07:00
Olaoluwa Osuntokun 78c1c4930e Merge pull request #3455 from wpaulino/cli-missing-chan-backup
cmd/lncli: return error upon missing channel backup when parsing
2019-09-03 19:39:12 -07:00
Olaoluwa Osuntokun b592760eee Merge pull request #3456 from wpaulino/pending-waiting-close-channel-fields
rpc: populate missing fields for waiting close channels
2019-09-03 19:35:49 -07:00
Wilmer Paulino d7364d0f28 rpc: populate missing fields for waiting close channels 2019-09-03 14:25:03 -07:00
Wilmer Paulino 06f544a78c cmd/lncli: return error upon missing channel backup when parsing 2019-09-03 14:04:38 -07:00
Lars Lehtonen 686fee57ee cmd/lncli: Fix dropped error 2019-09-03 12:29:26 -07:00
Johan T. Halseth 2f1aad257c autopilot/agent: trace log skipped nodes during scoring 2019-09-02 10:41:00 +02:00
Johan T. Halseth d5ab95546e autopilot/combinedattach: trace log scoring process from sub heuristics 2019-09-02 10:40:59 +02:00
Johan T. Halseth b2560b4ad2 autopilot/externalscoreattach: trace log scoring process 2019-09-02 10:40:59 +02:00
Johan T. Halseth bad9f4b685 autopilot/prefattach: trace log score process 2019-09-02 10:40:58 +02:00
Johan T. Halseth 2dbc4fe973 autopilot/agent: move expensive graph lookup after channel size check 2019-09-02 10:35:14 +02:00
Johan T. Halseth 81cb3cb739 Merge pull request #3434 from wpaulino/multi-hop-htlc-local-timeout-mempool-flake
lntest/itest: resolve mempool flake in multi-hop htlc local timeout test
2019-09-02 09:35:03 +02:00
Wilmer Paulino 578df81fb2 lntest/itest: resolve mempool flake in multi-hop htlc local timeout test
The test assumed that transactions would be broadcast and confirmed at
incorrect heights. Due to timing issues, it was possible for the test to
still succeed, resulting in a flake.

The test assumes that Bob will sweep a pending outgoing HTLC and commit
output back to their wallet. This commit ensures that these operations
are done when expected, i.e.:

1. Bob force closes the channel due to the HTLC timing out.
2. Once the channel is confirmed, Bob broadcasts their HTLC timeout
transaction.
3. Bob broadcasts their commit output sweep transaction once its CSV
expires.
4. Bob broadcasts their second layer sweep transaction for the timed out
HTLC once its CSV expires.
2019-08-30 12:15:14 -07:00
Wilmer Paulino 06206f09b5 Merge pull request #3010 from wpaulino/node-ann-flake
lntest/itest: prevent direct connection within testNodeAnnouncement
2019-08-30 11:55:05 -07:00
Wilmer Paulino 2f6ec77b3c chainntnfs/interface_test: lower mempool spend check timeout
In this commit, we lower the mempool spend check timeout to be twice as
long as the trickle interval of the miner node, which will greatly
improve the execution time of this specific test. We're able to do this
now since we can specify custom trickle intervals for our test
harnesses.
2019-08-29 18:05:46 -07:00
Wilmer Paulino 47a8cd36f7 lntest/itest: prevent direct connection within testNodeAnnouncement
Alice and Dave don't need to be connected in order to receive the node
announcement as we assume that she can receive it from Bob because they
are connected at the beginning of every test.
2019-08-29 18:04:13 -07:00
Johan T. Halseth 281c535b49 Merge pull request #3427 from cfromknecht/node-info-update-order
rpcserver: order updates in getnodeinfo by increasing pubkey
2019-08-29 09:13:48 +02:00
Conner Fromknecht d08721b32d rpcserver: correctly compute edge-level lastUpdate 2019-08-28 13:31:12 -07:00
Conner Fromknecht 2ebd76bbbb rpcserver: marshal db edges by increasing pubkey 2019-08-28 13:30:06 -07:00
Johan T. Halseth 2026d3b45e Merge pull request #3432 from cfromknecht/bolt11-features
zpay32: BOLT 11 Feature Bits
2019-08-28 15:23:13 +02:00
Conner Fromknecht 1311baf51f zpay32: add BOLT 11 feature bits and test vectors 2019-08-27 15:51:24 -07:00
Conner Fromknecht 8c2176fbf8 lnwire/features: add EncodeBase32 and DecodeBase32 w/ generic helpers 2019-08-27 15:51:24 -07:00
Conner Fromknecht 2953f3532a lnwire/features: add SerializeSize32 for base32 encodings 2019-08-27 15:51:24 -07:00
Olaoluwa Osuntokun 3868bdc490 Merge pull request #3405 from wpaulino/chainrpc-sane-defaults
chainntnfs: validate conf/spend ntfn registration parameters
2019-08-26 16:25:06 -07:00
Johan T. Halseth 9ef66f568f Merge pull request #2203 from ccdle12/reject-htlc-option
htlcswitch+config+server: adding RejectHTLC flag
2019-08-26 14:53:32 +02:00
Johan T. Halseth d2435a4dc3 Merge pull request #3059 from champo/rpc_drop
lnd: close the wallet unlock grpc server
2019-08-26 14:18:33 +02:00
Olaoluwa Osuntokun ebb3e987b1 lnrpc/routerrpc: complete JSON naming for mc responses 2019-08-23 16:36:35 -07:00
Olaoluwa Osuntokun 557083c41b Merge pull request #3372 from joostjager/mc-successes
routing: process payment successes in mission control
2019-08-23 16:33:18 -07:00
Olaoluwa Osuntokun 441b15055b channeldb: properly compare payment attempts in serialization test 2019-08-23 12:50:25 -07:00
Joost Jager ff0c5a0d5e routing: process successes in mission control
This commit modifies paymentLifecycle so that it not only feeds
failures into mission control, but successes as well.
This allows for more accurate probability estimates. Previously,
the success probability for a successful pair and a pair with
no history was equal. There was no force that pushed towards
previously successful routes.
2019-08-23 09:15:41 +02:00
Joost Jager 2594abfba1 routing/test: remove unused parameter in mission control test 2019-08-23 09:15:39 +02:00