Commit Graph

497 Commits

Author SHA1 Message Date
Conner Fromknecht
807012f960
Merge pull request #3143 from Crypt-iQ/pipelining_settle_0525
htlcswitch: pipeline settles to switch
2019-07-19 18:12:59 -07:00
Olaoluwa Osuntokun
111cbeaa99
Merge pull request #3165 from halseth/autopilot-spend-coins
[funding+autopilot] Make the funding manager and autopilot fee aware
2019-07-18 20:45:49 -07:00
nsa
00814dc7c1 htlcswitch: pipeline settles to switch
This commit makes the outgoing link pipeline the settle to the
switch as soon as it receives it. Previously, it would wait for a
revocation before sending it, which caused increased latency on
payments as well as possibly never settling on the incoming link.
A duplicate settle is still sent to the switch, but it is handled
gracefully. A new AckEventTicker was added to the switch which
acknowledges any pending settle / fail entries in an outgoing
link's fwd pkgs in batch. This was needed in order to reduce the
number of db txn's which would have been incurred by acking whenever
we receive a duplicate settle without batching.
2019-07-18 17:20:40 -04:00
Joost Jager
34b264a3d8
routing: create PathFindingConfig for global parameters 2019-07-18 15:49:25 +02:00
Johan T. Halseth
b9816259cb
fundingmanager+server: define subtractFees
Let one initiate a funding request with the remaining funds in the
wallet.
2019-07-18 00:58:12 +02:00
Johan T. Halseth
e716251805
server+funding: remove unused remoteFundingAmt 2019-07-18 00:33:14 +02:00
Johan T. Halseth
452ee6aad4
fundingmanager+server: define MaxPendingChans. RejectPush in funding config
This commit makes the funding manager access the MaxPendingChannels and
RejectPush values from the fundingConfig instead of the global config
struct.

Done to avoid sharing state between tests.
2019-07-18 00:33:14 +02:00
Joost Jager
37e2751695
routing+routerrpc: isolate payment session source from mission control 2019-07-13 22:38:19 +02:00
Joost Jager
2b47632b26
htlcswitch+routing+routerrpc: return error source as index 2019-07-11 10:37:26 +02:00
Olaoluwa Osuntokun
0470d27603
server: add an exponential back off to initialPeerBootstrap
In this commit we add exponential back off to the `initialPeerBootstrap`
method. Before this change, if the DNS seed was down, we would hammer it
in an attempt to get our initial set of peers. This makes this section a
bit less aggressive, but saves log spam and also will hit the DNS
servers less frequently.
2019-06-28 16:10:50 -07:00
Olaoluwa Osuntokun
efab9cb584
server: extract backOffCeiling into bootstrapBackOffCeiling as constant 2019-06-28 16:10:49 -07:00
Olaoluwa Osuntokun
abe73ca6c1
server: don't re-use existing wait group for loops within initialPeerBootstrap 2019-06-28 16:10:48 -07:00
Wilmer Paulino
dff7fba40a
server: properly set stopping atomic var upon shutdown 2019-06-18 19:38:20 -07:00
Conner Fromknecht
a543c781fe
watchtower/wtpolicy: add basic validity constraints 2019-06-13 19:54:22 -07:00
Conner Fromknecht
7c0a03c7c8
server+utxonursery: generate pkscript closures
This commit moves the newSweepPkScript function
previously in the nursery to be a helper function
within the server. Additionally, the function now
returns a closure that satisfies the configuration
interfaces of several other subsystems.

As a result, the configuration sites contain much
less boilerplate, as it's now encapsulated in
the newSweepPkScriptGen helper.
2019-06-13 19:54:19 -07:00
Conner Fromknecht
d2f3d5ef6c
server: add watchtower client 2019-06-13 19:54:18 -07:00
Johan T. Halseth
00a86696a8
Merge pull request #3145 from federicobond/avoid-pubkey-serialization
discovery,fundingmanager: avoid serialization in NotifyWhenOnline
2019-06-12 10:50:35 +02:00
Joost Jager
f03533c67a
routerrpc: convert sendpayment to async
Modify the routerrpc SendPayment api to asynchronous. This allows
callers to pick up a payment after the rpc connection was lost or lnd
was restarted.
2019-06-05 12:41:53 +02:00
Joost Jager
87d3207baf
channeldb+routing: move control tower interface to routing
This commit creates an empty shall for control tower in the routing
package. It is a preparation for adding event notification.
2019-06-05 12:41:47 +02:00
Federico Bond
9bd3055fb8 discovery,fundingmanager: avoid serialization in NotifyWhenOnline 2019-06-04 16:36:21 -03:00
Joost Jager
054e42f680
routing+routerrpc: expose mission control parameters in lnd config
This commit exposes the three main parameters that influence mission
control and path finding to the user as command line or config file
flags. It allows for fine-tuning for optimal results.
2019-06-04 13:22:50 +02:00
Joost Jager
b6102ad191
routing: remove querybandwidth self node check
This function is only ever called for channels connected to self.
2019-06-04 10:00:19 +02:00
Joost Jager
68f2a04f42
routerrpc: expose mission control reset rpc 2019-06-04 10:00:17 +02:00
Conner Fromknecht
f802ebddba
Merge pull request #3087 from joostjager/move-itest
lntest: move itest into package
2019-05-29 16:29:06 -07:00
Conner Fromknecht
f8287b0080
Merge pull request #2985 from johng/sub-batch
Broadcast gossip announcements in sub batches
2019-05-28 17:05:06 -07:00
Joost Jager
dff0387363
lntest: move itest into package 2019-05-28 21:43:23 +02:00
Olaoluwa Osuntokun
19fafd7a9a
Merge pull request #2761 from halseth/reliable-payments-router-state-machine
[reliable payments] router payment state machine
2019-05-27 16:46:19 -07:00
Johan T. Halseth
3f76bc0629
routing+server: define PaymentSessionSource 2019-05-27 20:18:59 +02:00
Johan T. Halseth
de1bf8a518
routing/router: persist payment state machine
This commit makes the router use the ControlTower to drive the payment
life cycle state machine, to keep track of active payments across
restarts.  This lets the router resume payments on startup, such that
their final results can be handled and stored when ready.
2019-05-27 20:18:58 +02:00
Wilmer Paulino
5172a5e255
multi: support arbitrary client fee preferences to UtxoSweeper
In this commit, we introduce support for arbitrary client fee
preferences when accepting input sweep requests. This is possible with
the addition of fee rate buckets. Fee rate buckets are buckets that
contain inputs with similar fee rates within a specific range, e.g.,
1-10 sat/vbyte, 11-20 sat/vbyte, etc. Having these buckets allows us to
batch and sweep inputs from different clients with similar fee rates
within a single transaction, allowing us to save on chain fees.

With this addition, we can now get rid of the UtxoSweeper's default fee
preference. As of this commit, any clients using the it to sweep inputs
specify the same fee preference to not change their behavior. Each of
these can be fine-tuned later on given their use cases.
2019-05-24 15:30:38 -07:00
John Griffith
f6058b4117 server: added new variables to gossiper config 2019-05-23 10:51:25 +01:00
Johan T. Halseth
c9e8ff6a34
switch+router+server: move NextPaymentID to router
This commit moves the responsibility of generating a unique payment ID
from the switch to the router. This will make it easier for the router
to keep track of which HTLCs were successfully forwarded onto the
network, as it can query the switch for existing HTLCs as long as the
paymentIDs are kept.

The router is expected to maintain a map from paymentHash->paymentID,
such that they can be replayed on restart. This also lets the router
check the status of a sent payment after a restart, by querying the
switch for the paymentID in question.
2019-05-16 23:56:06 +02:00
Johan T. Halseth
f1cb54f943
routing/router: define PaymentAttemptDispatcher interface
The switch satisfies this interface, and makes it easy to mock the send
method from the router.
2019-05-16 23:53:08 +02:00
Joost Jager
064e8492de
cnct+htlcswitch+invoices: move invoice parameter check out of link
This commit is the final step in making the link unaware of invoices. It
now purely offers the htlc to the invoice registry and follows
instructions from the invoice registry about how and when to respond to
the htlc.

The change also fixes a bug where upon restart, hodl htlcs were
subjected to the invoice minimum cltv delta requirement again. If the
block height has increased in the mean while, the htlc would be canceled
back.

Furthermore the invoice registry interaction is aligned between link and
contract resolvers.
2019-05-15 14:42:12 +02:00
Joost Jager
1a80a1e540
witness_beacon: do not look up invoice preimages
This commit isolates preimages of forwarded htlcs from invoice
preimages. The reason to do this is to prevent the incoming contest
resolver from settling exit hop htlcs for which the invoice isn't marked
as settled.
2019-05-15 14:41:58 +02:00
Johan T. Halseth
ee257fd0eb
multi: move Route to sub-pkg routing/route 2019-04-29 14:52:33 +02:00
Olaoluwa Osuntokun
a783f35ad4
Merge pull request #2932 from wpaulino/sync-manager-improvements
discovery: SyncManager improvements
2019-04-26 15:59:38 -07:00
Wilmer Paulino
5db2cf6273
discovery+server: remove roundRobinHandler and related code
Since ActiveSync GossipSyncers no longer synchronize our state with the
remote peers, none of the logic surrounding the round-robin is required
within the SyncManager.
2019-04-24 13:19:07 -07:00
Johan T. Halseth
b53899c43c
lnd: rename package main->lnd 2019-04-23 20:57:33 +02:00
Wilmer Paulino
760f38736e
fundingmanager: populate additional edge info after adding to graph
This commit serves as another stop-gap for light clients since they are
unable to obtain the capacity and channel point of graph edges. Since
they're aware of these things for their own channels, they can populate
the information within the graph themselves once each channel has been
successfully added to the graph.
2019-04-18 21:57:40 -07:00
Wilmer Paulino
5173ef6f85
Merge pull request #2014 from frennkie/re-create-port-mappings
Re-create forwardings after externalIP change
2019-04-18 17:38:02 -07:00
Robert Habermann
a27ac66eed server: add periodic renewal of port forwarding
The check prevented the creation of port forwardings which were assumed
to be present already. After this change the port forwardings which
might have been removed from the NAT device can be re-created.
2019-04-13 13:49:31 +00:00
Olaoluwa Osuntokun
d817b8c9db
server: remove duplicate HistoricalSyncTicker param 2019-04-11 16:01:17 -07:00
Wilmer Paulino
be727ab0be config+server: expose HistoricalSyncInterval flag
This allows us to set how often the daemon should routinely perform
historical graph syncs.
2019-04-11 15:45:19 -07:00
Olaoluwa Osuntokun
4b57fb199f server: add new errPeerAlreadyConnected error type 2019-04-10 17:27:58 -07:00
Conner Fromknecht
25d2b1b537
Merge pull request #2885 from cfromknecht/stagger-initial-reconnect
server: stagger initial reconnects
2019-04-05 15:46:12 -07:00
Joost Jager
ab4da0f53d
cnct: define separate broadcast delta for outgoing htlcs
This commits exposes the various parameters around going to chain and
accepting htlcs in a clear way.

In addition to this, it reverts those parameters to what they were
before the merge of commit d1076271456bdab1625ea6b52b93ca3e1bd9aed9.
2019-04-05 11:36:07 +02:00
Conner Fromknecht
cf80476e01
server: stagger initial reconnects
This commit adds optional jitter to our initial reconnection to our
persistent peers. Currently we will attempt reconnections to all peers
simultaneously, which results in large amount of contention as the
number of channels a node has grows.

We resolve this by adding a randomized delay between 0 and 30 seconds
for all persistent peers. This spreads out the load and contention to
resources such as the database, read/write pools, and memory
allocations. On my node, this allows to start up with about 80% of the
memory burst compared to the all-at-once approach.

This also has a second-order effect in better distributing messages sent
at constant intervals, such as pings. This reduces the concurrent jobs
submitted to the read and write pools at any given time, resulting in
better reuse of read/write buffers and fewer bursty allocation and
garbage collection cycles.
2019-04-05 02:31:58 -07:00
Wilmer Paulino
70be812747
discovery+server: use new gossiper's SyncManager subsystem 2019-04-03 15:44:43 -07:00
Johan T. Halseth
96f4cd1df9
server: stop chansubswapper on shutdown instead of start 2019-04-02 09:58:13 +02:00
Olaoluwa Osuntokun
893cff60c4
Merge pull request #2839 from joostjager/htlc-expiry-check
link: increase expiry grace delta for exit hop and forwarded htlcs
2019-04-01 18:15:56 -07:00
Olaoluwa Osuntokun
26c52892cb
server: convert Start/Stop methods to use sync.Once
In this commit, we convert the server's Start/Stop methods to use the
sync.Once. We do this in order to fix concurrency issues that would
allow certain queries to be sent to the server before it has actually
fully start up. Before this commit, we would set started to 1 at the
very top of the method, allowing certain queries to pass before the rest
of the daemon was had started up.

In order to fix this issue, we've converted the server to using a
sync.Once, and two new atomic variables for clients to query to see if
the server has fully started up, or is in the process of stopping.
2019-03-28 17:54:05 -07:00
Olaoluwa Osuntokun
6bc20b1a14
server: assemble and start chanbackup.SubSwapper on startup 2019-03-28 17:53:36 -07:00
Olaoluwa Osuntokun
c5933d45fb
server: feed through any SCBs on start up to be restored
In this commit, we modify the server to serve the role as the agent
which will carry out the SCB restoration protocol if the Init/Unlock
methods include a set of channels to be recovered.
2019-03-28 17:53:26 -07:00
Wilmer Paulino
174645fcba
routing+server: expose DefaultChannelPruneExpiry 2019-03-27 13:07:13 -07:00
Conner Fromknecht
c5365160fb
server: wrap peer connection tiebreaking to 80 chars 2019-03-26 16:41:30 -07:00
Conner Fromknecht
7358535725
peer+server: log pubkey@addr 2019-03-26 16:41:13 -07:00
Olaoluwa Osuntokun
1a8e4b0316
Merge pull request #2058 from roeierez/fix_reconnect
Reconnect to peer when switching networks
2019-03-26 16:13:21 -07:00
Joost Jager
cd535b9401
link: increase expiry grace delta
This commit increase the expiry grace delta to a value above the
broadcast delta. This prevents htlcs from being accepted that would
immediately trigger a channel force close.

A correct delta is generated in server.go where there is access to
the broadcast delta and passed via the peer to the links.

Co-authored-by: Jim Posen <jim.posen@gmail.com>
2019-03-26 18:42:00 +01:00
Olaoluwa Osuntokun
71161848aa
Merge pull request #2736 from cfromknecht/config-num-workers
lncfg: add CLI worker configuration
2019-03-19 15:47:38 -07:00
Joost Jager
3b5c2f44c6
invoices: extract invoice decoding from registry
Previously it was difficult to use the invoice registry in unit tests,
because it used zpay32 to decode the invoice. For that to succeed, a
valid signature is required on the payment request.

This commit injects the decode dependency on a different level so that
it is easier to mock.
2019-03-15 10:09:06 +01:00
Joost Jager
449c3d533e
contractcourt: add access to full invoice registry from resolvers
Previously a function pointer was passed to chain arbitrator to avoid a
circular dependency. Now that the routetypes package exists, we can pass
the full invoice registry to chain arbitrator.

This is a preparation to be able to use other invoice registry methods
in contract resolvers.
2019-03-15 10:08:35 +01:00
Conner Fromknecht
76116f0d1a
config: use CLI worker configuration instead of NumCPU() 2019-03-13 20:32:39 -07:00
Conner Fromknecht
935ea7d450
server: broadcast using SendMessageLazy, remove unused sendPeerMessages 2019-03-05 17:09:14 -08:00
Conner Fromknecht
b50fd33915
server: remove unused SendToPeer method 2019-03-05 17:09:01 -08:00
Conner Fromknecht
0ae06c8189
discovery+server: send lazy gossip msgs 2019-03-05 17:08:48 -08:00
Olaoluwa Osuntokun
a6ba965bc4
Merge pull request #2474 from cfromknecht/read-and-write-pools
lnpeer+brontide: reduce memory footprint using read/write pools for message encode/decode
2019-02-24 16:39:32 -03:00
Conner Fromknecht
603601a4c8
peer+server: use peer-level readPool 2019-02-21 20:11:33 -08:00
Conner Fromknecht
9a3c0b8bca
peer+server: switch to pool.Write from pool.WriteBuffer 2019-02-21 20:10:51 -08:00
Olaoluwa Osuntokun
cbe0bf6a22
Merge pull request #2501 from cfromknecht/batch-preimage-writes
htlcswitch: batch preimage writes/consistency fix
2019-02-21 17:00:00 -08:00
Olaoluwa Osuntokun
2bf22617d4
Merge pull request #1595 from wpaulino/send-channel-update-reliably
discovery/gossiper: reliably send channel update msg to remote peer
2019-02-19 21:09:04 -08:00
Conner Fromknecht
2d8bc99d9e
lntypes/preimage: add MakePreimage initializer 2019-02-19 17:05:45 -08:00
Olaoluwa Osuntokun
9d23d382fc
Merge pull request #2419 from cfromknecht/brontide-buffer-pool
brontide: read buffer pool
2019-02-18 17:51:17 -08:00
Roei Erez
6c256aea30 server: fix the algo to prefer connections when multiple exist
This commit modified the condition to whether drop an existing
connection to a peer when a new connection to this peer is
established.
The previous algorithm used public keys comparison for this decision
which determines that between every two nodes only one of them will
ever drop the connection in such cases.
The problematic case is when a node disconnects and reconnects in a
short interval which is the case of mobile devices.
In such case it takes as much as the "timeout" configured value for
the remote node to detect the "disconnection" (and try to reconnect
if this connection is persistent). In the case this node is also the
one that has the "smaller" public key the reconnect attempts of the
other node will be rejected causing it impossible to fast reconnect.

The solution is to only drop the connection if if we already have a
connected peer that is of the opposite direction from the this new
connection. By doing so the "initiator" will be enabled to replace
the connection and recconnect immediately.
2019-02-18 19:00:56 +02:00
Conner Fromknecht
6f96d04b72
multi: add buffer.Write and pool.WriteBuffer, make GCQueue generic 2019-02-15 19:31:24 -08:00
Wilmer Paulino
12168f022e
server+discovery: send channel updates to remote peers reliably
In this commit, we also allow channel updates for our channels to be
sent reliably to our channel counterparty. This is especially crucial
for private channels, since they're not announced, in order to ensure
each party can receive funds from the other side.
2019-02-14 18:33:27 -08:00
Wilmer Paulino
4996d49118
server+discovery: use reliableSender to replace existing resend logic 2019-02-14 18:33:27 -08:00
Wilmer Paulino
73b4bc4b68
server+discovery: remove channeldb.DB reference within the gossiper
Now that we've replaced the built-in messageStore with the
channeldb.GossipMessageStore, the reference to channeldb.DB is no longer
needed.
2019-02-14 18:29:39 -08:00
Wilmer Paulino
2277535e6b
server+discovery: replace gossiper message store with MessageStore 2019-02-14 18:29:39 -08:00
Conner Fromknecht
de282172a1
peer+server+test_utlils: use new ChanStatusManager
This commit hooks up the new netann.ChanStatusManager,
replacing the prior method which used the
watchChannelStatus goroutine.
2019-02-14 17:13:44 -08:00
Conner Fromknecht
4686db4196
config+server+lnd_test: expose chan status manager config options
Exposes the three parameters that dictate
the behavior of the channel status manager:
 * --chan-enable-timeout
 * --chan-disable-timeout
 * --chan-status-sample-interval
2019-02-14 17:13:11 -08:00
Conner Fromknecht
1a7fd6e6f1
netann/channel_update: exposes channel update helpers moved from server 2019-02-14 17:12:03 -08:00
Valentine Wallace
10090a978d contractcourt/chain_arbitrator: notify the ChannelNotifier on newly closed channels. 2019-02-05 18:17:54 -08:00
Valentine Wallace
bdd8ce14c9 switch: notify the ChannelNotifier on newly active/inactive channels. 2019-02-05 18:17:54 -08:00
Valentine Wallace
5405028948 fundingmanager: inform ChannelNotifier about newly open channels. 2019-02-05 18:17:54 -08:00
Valentine Wallace
cb26fd8a17 lnd: introduce the ChannelNotifier.
This commit introduces the channel notifier which is a central source
of active, inactive, and closed channel events.

This notifier was originally intended to be used by the `SubscribeChannels`
streaming RPC call, but can be used by any subsystem that needs to be
notified on a channel becoming active, inactive or closed.

It may also be extended in the future to support other types of notifications.
2019-02-05 18:17:54 -08:00
Olaoluwa Osuntokun
1189f0ea6c
server: use chanInfo instead of chanPolicy to establish channel peer conns
In this commit, we modify the way we attempt to locate the our channel
peer to establish a persistent connection on start up. Before this
commit, we would use the channel policy pointing to the peer to locate
the node as it's embedded in the struct. However, at times it's
currently possible for the channel policy to not be found in the
database if the remote nodes announces before we finalize the process on
our end. This can at times lead to a panic as the pointer isn't checked
before attempting to access it.

To remedy this, we now instead use the channel info which will _always_
be there if the channel is known.
2019-02-03 15:01:37 -08:00
Olaoluwa Osuntokun
bceb048a76
Merge pull request #2385 from cfromknecht/peer-write-buffer
peer: write buffer pool
2019-02-01 17:04:30 -08:00
Olaoluwa Osuntokun
5167b02312
Merge pull request #2500 from Roasbeef/mandatory-dlp
server: require the DLP bit for all incoming/outgoing connections
2019-01-31 20:14:15 -08:00
Conner Fromknecht
24dcd3c44e
peer+server: use server-wide writeBufferPool for peer write buffers 2019-01-31 19:21:36 -08:00
Joost Jager
9e012ecc93
multi: move Input interface and related code
This commit is a step to split the lnwallet package. It puts the Input
interface and implementations in a separate package along with all their
dependencies from lnwallet.
2019-01-31 13:25:33 +01:00
Conner Fromknecht
58f95322c0
server: fix notify-online log message 2019-01-23 20:40:27 -08:00
Wilmer Paulino
41f638c7cf
contractcourt+server: settle invoice for on-chain HTLC sweep
In this commit, we extend the htlcSuccessResolver to settle the invoice,
if any, of the corresponding on-chain HTLC sweep. This ensures that the
invoice state is consistent as when claiming the HTLC "off-chain".
2019-01-22 20:46:14 -08:00
Valentine Wallace
390b974793
server: set max htlc in ChannelUpdate createChannelUpdate
This method is called to convert an EdgePolicy to a ChannelUpdate. We
make sure to carry over the max_htlc value.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:29 +01:00
Valentine Wallace
0fd6004958
multi: partition lnwire.ChanUpdateFlag into ChannelFlags and MessageFlags
In this commit:

* we partition lnwire.ChanUpdateFlag into two (ChanUpdateChanFlags and
ChanUpdateMsgFlags), from a uint16 to a pair of uint8's

* we rename the ChannelUpdate.Flags to ChannelFlags and add an
additional MessageFlags field, which will be used to indicate the
presence of the optional field HtlcMaximumMsat within the ChannelUpdate.

* we partition ChannelEdgePolicy.Flags into message and channel flags.
This change corresponds to the partitioning of the ChannelUpdate's Flags
field into MessageFlags and ChannelFlags.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:26 +01:00
Olaoluwa Osuntokun
c815821679
server: require the DLP bit for all incoming/outgoing connections
In this commit, we modify our default local feature bits to require the
Data Loss Protection (DLP) feature to be active. Once full Static
Channel Backups are implemented, if we connect to a peer that doesn't
follow the DLP protocol, then the SCBs are useless, as we may not be
able to recover funds. As a result, in prep for full SCB deployment,
we'll now ensure that any peer we connect to, knows of the DLP bit. This
could be a bit more relaxed and allow _connections_ to non-DLP peers,
but reject channel requests to/from them. However, this implementation is
much simpler.
2019-01-17 16:53:43 -08:00
Olaoluwa Osuntokun
b700b969e8
Merge pull request #2463 from wpaulino/notify-when-offline
server: implement notifications for disconnected peers
2019-01-15 16:47:29 -08:00
Olaoluwa Osuntokun
509bed614c
Merge pull request #2198 from Roasbeef/sendall-rpc
multi: add ability to sweep all coins in the the wallet to an addr to sendcoins
2019-01-15 14:49:17 -08:00
Wilmer Paulino
cbce6e03fc
server: implement notifications for disconnected peers 2019-01-14 22:44:45 -08:00
Conner Fromknecht
a8b2c093aa
Merge pull request #2416 from cfromknecht/netann-node-signer
netann: move NodeSigner out of main package
2019-01-09 17:01:00 -08:00