Commit Graph

4528 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
7917fd2ab0
invoiceregistry: re-work logic to support delivering notification backlog
In this commit, we re-work the existing invoiceRegistry struct to
support delivering backlog notifications to subscription clients if
needed. Rather than using 1 goroutine per-client per-event, each client
now gains a concurrent notification queue. This queue will then be used
to ensure in-order delivery of all notifications from the
invoiceEventNotifier.

The SubscribeNotifications method now takes two params: addIndex, and
settleIndex. These should be the values of the last add index and settle
index the caller knows of. If specified (not zero), then we'll look up
all the notifications that the caller has missed, and then deliver those
before sending out any new notifications. In order to do this without
losing ordering of events, we've added a new central goroutine which
will ensure that all events are properly serialized.
2018-07-06 12:21:59 -07:00
Olaoluwa Osuntokun
e4c5840472
rpc: ensure new invoice fields are populated in RPC responses 2018-07-06 12:21:58 -07:00
Olaoluwa Osuntokun
2097564307
channeldb: add new migration to upgrade old databases to new invoice time series index 2018-07-06 12:21:58 -07:00
Olaoluwa Osuntokun
24d97b5ee5
channeldb: add new test to exercise invoice time series index 2018-07-06 12:21:58 -07:00
Olaoluwa Osuntokun
532975588d
channeldb: update existing invoice tests to due to recent field additions 2018-07-06 12:21:57 -07:00
Olaoluwa Osuntokun
24ae13d3a5
channeldb: add two methods to allow seeking into the invoice time series
In this commit, we add two new methods: InvoicesAddedSince and
InvoicesSettledSince. These methods will be used by higher level
sub-systems that implement notifications to deliver any notifications
backlog based on the last add index, and last settle index that the
client knows of.

It's important to note that care has been taken to ensure that this new
API can be used in a backwards compatible manner. If a client specifies
and index of 0 for either of the methods, then no backlog will be sent.
This is due to the fact that current users of the API don't expect any
backlog notifications to be sent. Additionally, the index actually
starts at 1, instead of 0.
2018-07-06 12:21:57 -07:00
Olaoluwa Osuntokun
5d20c02ea8
channeldb: add new add+settle index to invoice database
In this commit, we add two new indexes to the invoice database: the add
index, and the settle index. These to indexes essentially form a time
series index on top of the existing primary index bucket. Each time an
invoice is added, we'll advance the addIndex seqno, and then create a
mapping from seqNo -> invoiceNum. Each time an invoice is settled, we'll
do the same, but within the settle index.

This change is required in order to allow callers to effectively seek
into the current invoice database in order to obtain notifications for
any invoices they may have missed out on while they were disconnected.
This will allow us to implement robust streaming invoice notifications
within lnd to ensure that clients never miss an event.
2018-07-06 12:21:56 -07:00
Olaoluwa Osuntokun
fc0f0d33b2
channeldb: add new AmtPaid field to the Invoice struct
In this commit, in order to allow the caller to specify the amount that
was ultimately accepted for an invoice, the SettleInvoice method has
gained a new parameter: amtPaid. SettleInvoice will now populate the
final amount paid in the database upon db commit.
2018-07-06 12:21:56 -07:00
Olaoluwa Osuntokun
a32f2b79da
htlcswitch: modify the InvoiceDatabase interface to allow specifying final payment amt
In this commit, we modify the InvoiceDatabase slightly to allow the link
to record what the final payment about for an invoice was. It may be the
case that the invoice actually had no specified value, or that the payer
paid more than necessary. As a result, it's important that our on-disk
records properly reflect this.

To fix this issue, the SettleInvoice method now also accepts the final
amount paid.

Fixes #856.
2018-07-06 12:21:55 -07:00
Olaoluwa Osuntokun
edbdcddea1
lnrpc: extend invoice subscriptions to allow callers to receive backlog notifications
In this commit, we extend the current SubscribeInvoice streaming RPC
call. We add two new values to the InvoiceSubscription message:
add_index and settle_index. These fields have also been added to the
current Invoice message. Each time a new invoice is added, the add index
will be incremented. Each time a new invoice is settled the settle index
will be incremented. The new field on the InvoiceSubscription message
allow callers to specify the last add index and the last settle index
they know of. With this new addition, callers will now be able to
reliably receive notifications for new received payments.

Care has been taken to ensure that these changes are backwards
compatible. If callers don't specify either of the new fields, then they
won't receive any notification backlog at all.

Fixes #862.
2018-07-06 12:21:55 -07:00
Olaoluwa Osuntokun
29a27bbc3a
test: ensure we always cancel active streaming notifications 2018-07-06 12:21:54 -07:00
Olaoluwa Osuntokun
dbf7e4525a
lntest: ensure we always cancel active streaming notifications 2018-07-06 12:21:54 -07:00
Suriyaa ✌️️
92029370ec README: use HTTPS instead of HTTP 2018-07-06 14:20:51 -05:00
Olaoluwa Osuntokun
a0b2fadea3
Merge pull request #1458 from cfromknecht/add-addrs-to-codec
channeldb: expand codec to include net.Addr types
2018-07-03 22:50:22 -05:00
Olaoluwa Osuntokun
d83f4fbb85
Merge pull request #1498 from yaslama/fix-1488
Fix the ability to disable p2p listening
2018-07-03 22:40:02 -05:00
Olaoluwa Osuntokun
0a045f8651
Merge pull request #1482 from cfromknecht/uniform-fee-update-timeout
htlcswitch/link: correct bias in fee update backoff
2018-07-03 22:27:22 -05:00
Olaoluwa Osuntokun
552a371f09
Merge pull request #1475 from cfromknecht/disable-mainnet-debug-htlc
config: disable debug-htlc mode on mainnet
2018-07-03 22:26:45 -05:00
Olaoluwa Osuntokun
9815142922
Merge pull request #1497 from hackerrdave/invoiceregistry-comment-updates
update comment for AddInvoice
2018-07-03 22:05:24 -05:00
Olaoluwa Osuntokun
1778a3c38e
Merge pull request #1480 from cfromknecht/fix-server-log-msgs
server: log pubkey hex in connection callbacks
2018-07-03 22:00:25 -05:00
Olaoluwa Osuntokun
5626fd6c9b
Merge pull request #1478 from wpaulino/resolve-tor-dns-host
config: only normalize and resolve tor DNS host if it has changed
2018-07-03 21:59:49 -05:00
Olaoluwa Osuntokun
bc04bfdde2
Merge pull request #1471 from cfromknecht/lnwire-query-sids-corpus
[lnwire]: align behavior of query sids corpus for zero-length slices
2018-07-03 21:58:00 -05:00
Olaoluwa Osuntokun
184f1e41fc
Merge pull request #1469 from cfromknecht/expose-lnwallet-commit-scripts
lnwallet: export commit script helper methods
2018-07-03 21:53:17 -05:00
Olaoluwa Osuntokun
85526dfa46
Merge pull request #1459 from wpaulino/identify-peer-failed-conn
brontide: identify remote address of failed connection
2018-07-03 21:46:03 -05:00
Olaoluwa Osuntokun
3c1d06d54a
Merge pull request #1486 from Roasbeef/minor-forwarding-perf-improvements
htlcswitch+routing: minor forwarding perf improvements
2018-07-03 21:44:16 -05:00
Conner Fromknecht
5be6993a56
server: log pubkey hex in connection callbacks 2018-07-03 17:09:10 -07:00
Conner Fromknecht
7d116abedb
config: disable debug-htlc mode on mainnet 2018-07-03 17:08:53 -07:00
Conner Fromknecht
61c2493b7d
lnwire/query_short_chan_ids: short circuit if 0 sids
In this commit, we alter the behavior of the regular
short channel id encoding, such that it returns a nil
slice if the decoded number of elements is 0. This is
done so that it matches the behavior of the zlib
decompression, allowing us to test both in using the
same corpus.
2018-07-03 17:08:40 -07:00
Conner Fromknecht
8cc217b526
lnwire/lnwire_test: use nil slice when 0 sids
Modifies the behavior of the quick test for
MsgQueryShortChanIDs, such that the generated
slice of expected short chan ids is always nil
if no elements are returned. This mimics the
behavior of the zlib decompression, where
elements are appended to the slice, instead of
assigning to preallocated slice.
2018-07-03 17:08:39 -07:00
Conner Fromknecht
92cbfbccca
lnd_test: remove itest race caused by mining extra block 2018-07-03 17:08:21 -07:00
Conner Fromknecht
2843df88c0
lnwallet/script_utils_test: use public commit script helpers 2018-07-03 17:07:48 -07:00
Conner Fromknecht
a66c488546
lnwallet/wallet: use public witness script hash helper 2018-07-03 17:07:48 -07:00
Conner Fromknecht
bb808a1937
lnwallet/channel: use public variant of script helpers 2018-07-03 17:07:48 -07:00
Conner Fromknecht
b507ea0ca3
lnwallet/script_utils: expose commit script helpers
Makes the helper methods for constructing witness script
hash and to-local outputs. This will allow watchtowers to
import and reuse this logic when sweeping outputs.
2018-07-03 17:07:48 -07:00
Conner Fromknecht
75d7436f7a
channledb/codec: add net.Addr and []net.Addr to codec 2018-07-03 17:07:10 -07:00
Conner Fromknecht
4cb7953f66
channledb/addr: remove silent encoding failure for unknown types 2018-07-03 17:07:10 -07:00
Conner Fromknecht
c6c442f6b6
channledb/addr_test: add negative test for unknown types 2018-07-03 17:07:10 -07:00
Conner Fromknecht
21674c05e5
channeldb/codec: add concrete error for unknown types 2018-07-03 17:07:09 -07:00
Conner Fromknecht
1d5189bd25
channeldb/forwarding_log: use public Read/WriteElements 2018-07-03 17:07:09 -07:00
Conner Fromknecht
30ff91913d
channeldb/channel: use public Read/WriteElements 2018-07-03 17:07:09 -07:00
Conner Fromknecht
ec029ac7fd
channeldb/codec: exposes Read/WriteElements 2018-07-03 17:07:09 -07:00
Yaacov Akiba Slama
c9c47b3a51 Fix the ability to disable p2p listening 2018-07-03 07:54:17 +03:00
hackerrdave
d98507abcf update comment for AddInvoice 2018-07-02 23:20:17 -04:00
Olaoluwa Osuntokun
d889644f52
htlcswitch: use Batch for calls within ackSettleFail 2018-06-30 17:15:31 -05:00
Olaoluwa Osuntokun
3ad15c5f42
hltcswitch: use a logClosure to delay spew evaluation until log print time 2018-06-30 17:15:04 -05:00
Olaoluwa Osuntokun
39c1f46f85
routing: new newLogClosure to delay spew evaluation until log print time
In this commit, we update the generateSphinxPacket to use newLogClosure
to delay the spew evaluation until log print time. Before this commit,
even if we weren't on the trace logging level, the spew call would
always be evaluated.
2018-06-30 17:14:28 -05:00
Olaoluwa Osuntokun
b3bb65ea43
rpc: restore proper debug HTLC usage
In this commit, we fix a slight regression that happened when the
sendtoroute RPC call was implemented. As a result of the refactoring,
the debughtlc mode silently broke. In this commit, we fix this by
checking against the zero hash, rather than the length of the payment
hash, as since we use a fixed sized byte slice, it will always be zero.
2018-06-30 17:13:19 -05:00
Conner Fromknecht
12f74f762d
htlcswitch/test_utils: use realistic link timeouts 2018-06-30 11:51:47 -07:00
Conner Fromknecht
10ecf2307f
htlcswitch/link_test: use realistic link timeouts 2018-06-30 11:42:36 -07:00
Conner Fromknecht
5fa8124aa1
htlcswitch/link: correct bias in fee update backoff
This commit corrects the distribution used to
schedule a link's randomized backoff for fee
updates. Currently, our algorithm biases the
lowest value in the range, with probability
equal to lower/upper, or the ratio of the lower
bound to the upper. This distribution is skewed
more heavily as lower approaches upper.

The solution is to sample a random value in the
range upper-lower, then add this to our lower
bound. The effect is a uniformly distributed
timeout in [lower, upper).
2018-06-30 10:22:38 -07:00
Olaoluwa Osuntokun
9205720bea
Merge pull request #1477 from Roasbeef/proper-exit-hop-cltv-fix
channeldb+htlcswitch: parse out min cltv delta from payreq rather than modifying the db
2018-06-29 18:11:19 -07:00