Previously, nursery generated and published its own sweep txes. It
stored the sweep tx in nursery_store to prevent a new tx with a new
sweep address from being generated on restart.
In this commit, sweep generation and publication is removed from nursery
and delegated to the sweeper. Also the confirmation notification is
received from the sweeper.
In this commit, we remove the per channel `sigPool` within the
`lnwallet.LightningChannel` struct. With this change, we ensure that as
the number of channels grows, the number of gouroutines idling in the
sigPool stays constant. It's the case that currently on the daemon, most
channels are likely inactive, with only a hand full actually
consistently carrying out channel updates. As a result, this change
should reduce the amount of idle CPU usage, as we have less active
goroutines in select loops.
In order to make this change, the `SigPool` itself has been publicly
exported such that outside callers can make a `SigPool` and pass it into
newly created channels. Since the sig pool now lives outside the
channel, we were also able to do away with the Stop() method on the
channel all together.
Finally, the server is the sub-system that is currently responsible for
managing the `SigPool` within lnd.
In this commit, we update the genNodeAnnouncement method to also write
an updated version of the node announcment to disk. Before this commit,
we would update the in memory version, but then never write the new
version to disk. As a result, when connecting to new peers, we would
never propagate the new version of this announcement to the network.
In this commit, we modify the newly introduced UtxoSweeper.CreateSweepTx
to accept the confirmation target as a param of the method rather than a
struct level variable. We do this as this allows each caller to decide
at sweep time, what the fee rate should be, rather than using a global
value that is meant to work in all scenarios. For example, anytime
we're sweeping an output with a CLTV lock that's has a dependant
transaction we need to sweep/cancel, we may require a higher fee rate
than a regular force close with a CSV output.
In this commit, we also check ErrEdgeNotFound when attempting to send an
active/inactive channel update for a channel to the network. We do this
as it's possible that a channel has confirmed, but it still does not
meet the required number of confirmations to be publicly announced.
In this commit IncubateOutputs are given an extra parameter
broadcastHeight, which is passed from the server and used when called
registerPresschoolConf.
Earlier the utxonursery used the bestHeight as height hint in this case,
which would be wrong in the cases where we got outputs for incubation
which was confirmed below the best height.
over Tor
In this commit, we fix a small bug where we would attempt to start the
Tor controller even if we were not requested to automatically create and
onion service in order to listen for inbound connections over Tor.
In this commit, we restrict the persistent connection logic on startup
to only attempt to establish connections to Tor addresses if Tor
outbound support is enabled. Otherwise, we'll continually attempt to
reach the address even though we never will.
In this commit, we remove signaling for initial routing
dumps, which create unnecessary log spam, bandwidth, and
CPU. Now that gossip syncing is in full force, we will
instead opt to use the more efficient querying/set
reconciliation. Other nodes may still request initial
gossip sync from us, and we will respond.
This commit modifies the connection peer backoff
logic such that it will always backoff for "unstable"
peers. Unstable in this context is determined by
connections whose duration is shorter than 10
minutes. If a disconnect happens with a peer
whose connection lasts longer than 10 minutes,
we will scale back our stored backoff for that peer.
This resolves an issue that would result in a tight
connection loop with remote peers. This stemmed
from the connection duration being very short,
and always driving the backoff to the default
backoff of 1 second. Short connections like
this are now caught by the stable connection
threshold.
This also modifies the computation on the
backoff relaxation to subtract the connection
duration after applying randomized exponential
backoff, which offers better stability when
the connection duration and backoff are roughly
equal.
In this commit, we avoid logging an error when the links associated with
a peer are not found within its termination watcher. We do this to
prevent a benign log message as the links have already been removed from
the switch.
This commit fixes a bug that would cause us to fetch our peer's
ChannelUpdate in some cases, where we really wanted to fetch our own.
The reason this happened was that we passed the peer's pubkey to
fetchLastChanUpdate, making us match on their policy. This would lead to
ChannelUpdates being sent during routing which would have no effect on
the attempted path.
We fix this by always use our own pubkey in fetchLastChanUpdate, and
also uses the common methods within the server to be able to extract the
update even when only one policy is known.
This commit adds a goroutine watchChannelStatus to the server, which
will query the switch for the status of all open channels every
InactiveChanTimeout / 4. If a channel's status has remained unchanged
during the last InactiveChanTimeout it'll send out a ChannelUpdate
setting the disabled bit accordingly.
ProcessLocalAnnouncement will attempt to call UpdateEdge with the new
policy. If we call it manually before handing it to the gossiper, that
call will fail with "Outdated" and the announcement won't propagate.
This commit adds asynchronous starting of peers,
in order to avoid potential DOS vectors. Currently,
we block with the server's mutex while peers exchange
Init messages and perform other setup. Thus, a remote
peer that does not reply with an init message will
cause server to block for 15s per attempt.
We also modify the startup behavior to spawn
peerTerminationWatchers before starting the
peer itself, ensuring that a peer is properly
cleaned up if the initialization fails. Currently,
failing to start a peer does not execute the bulk
of the teardown logic, since it is not spawned
until after a successful Start occurs.
In this commit, we fix a small bug where we would increase epochErrors
by one even if connections were successfully established. Due to this,
we would stay stuck inside of the peer bootstrapper loop without
requerying for new peers.
In this commit, we move the initialization of the server into the
funding manager itself. We do this as it's no longer the case that _any_
RPC needs to access the funding manager. In the past, this was the
only reason that the funding manager was instantiated outside of the
server: to be able to respond to queries _before_ the server was
started.
This change also fixes a bug as atm, the funding manager will try to
register for notifications _before_ the ChainNotifier itself has fully
started.
In this commit, we modify the existing message sending functionality
within the fundingmanager. Due to each mesage send requiring to hold the
server's lock to retrieve the peer, we might run into a case where the
lock is held for a larger than usual amount of time and would therefore
block on sending the message within the fundingmanager. We remedy this
by taking a similar approach to some recent changes within the gossiper.
We now keep track of each peer within the internal fundingmanager
messages and send messages directly to them.
In this commit, we extend the server's functionality to prune link nodes
on startup. Since we currently only decide whether to prune a link node
from the database based on a channel close, it's possible that we have
link nodes lingering from before this functionality was added on.
In this commit, we update all the lncfg methods used to properly pass in
a new resolver. This is required in order to ensure that we don't leak
our DNS queries if Tor mode is active.
In this commit, we move the block height dependency from the links in
the switch to the switch itself. This is possible due to a recent change
on the links no longer depending on the block height to update their
commitment fees.
We'll now only have the switch be alerted of new blocks coming in and
links will retrieve the height from it atomically.
In this commit, we address an existing issue with regards to the inital
peer bootstrapping stage. At times, the bootstrappers can be unreliable
by providing addresses for peers that no longer exist/are currently
offline. This would lead to nodes quickly entering an exponential
backoff method used to maintain a minimum target of peers without first
achieving said target.
We address this by separating the peer bootstrapper into two stages: the
initial peer bootstrapping and maintaining a target set of nodes to
maintain an up-to-date view of the network. The initial peer
bootstrapping stage has been made aggressive in order to provide such
view of the network as quickly as possible. Once done, we continue on
with the existing exponential backoff method responsible for maintaining
a target set of nodes.
traversal
In this commit, we allow our node to automatically advertise its
connection's external IPs on the ports it is currently listening on in
order to accept inbound connections. This is only done when specifying
a NAT traversal technique when starting the daemon.
We also include a handy method that watches for dynamic IP changes in
the background. If a new IP is detected, we'll craft a new node
announcement using the new IP and broadcast it to the network.
In this commit, we finish the fix for the inbound/outbound peer bool in
the server. The prior commit forgot to also flip the inbound/output maps
in Inbound/Outbound peer connected. As a result, the checks were
incorrect and could cause lnd to refuse to accept any more inbound
connections in the case of a concurrent connection attempt.
In this commit, we ensure that if we're already ignoring a connection,
then we also ignore the pending persistent connection request.
Otherwise, we'll move to accept the replaced connection, but then
continue to attempt connection requests.
In this commit, we modify the look up for inbound peers to ensure that
we connect to the "freshest" address until we need to execute the
peerTerminationWatcher. We do this as it's possible for a channel to be
created by the remote peer during our session. If we don't query for the
node's address at the latest point, then we'll miss this new node
announcement for the node.
In this commit, we address the meaning of the inbound parameter to
peerConnected. An inbound connection is defined as a connection
initiated by the peer, rather than ourselves.
We also update the inbound value for the peerConnected calls within
OutboundPeerConnected and InboundPeerConnected to reflect the definition
above.
We remove the internale broadcastMessage method, and instead handle the
mutex handling within BroadcastMessage. This lets us hold the mutex only
when neccessary.
This commit removes the sendToPeer method from the server, and instead
moves the necessary logic into SendToPeer. This let's us make the mutex
acquisition more fine-grained, only holding it while reading from the
peer map. Earlier it was required to be held during the whole call to
sendToPeer, as the method would access the map internally.
In this commit, we go through the codebase looking for TCP address
assumptions and modifying them to include the recently introduced onion
addresses. This enables us to fully support onion addresses within the
daemon.
In this commit, we update the way we reestablish inbound connections if
we lose connectivity to a node we have an open channel with. Rather than
fetching the node's advertised port, we'll fetch one of their advertised
addresses instead. This ensure that if the remote node is running behind
a proxy, we do not see the proxy's address.
In this commit, we allow the daemon to use the recently introduced Tor
Controller implementation. This will automatically create a v2 onion
service at startup in order to listen for inbound connections over Tor.
Co-Authored-By: Eugene <crypt-iq@users.noreply.github.com>
In this commit, we fix a bug where a fallback SRV lookup would leak
information if `lnd` was set to route connections over Tor. We solve
this by using the network-specific functions rather than the standard
ones found in the `net` package.
In this commit, we allow `lnd` to properly parse onion addresses in
order to advertise them to the network when set through the
`--externalip` flag.
Co-Authored-By: Eugene <crypt-iq@users.noreply.github.com>
In this commit, we introduce a new method to the channel router's config
struct: QueryBandwidth. This method allows the channel router to query
for the up-to-date available bandwidth of a particular link. In the case
that this link emanates from/to us, then we can query the switch to see
if the link is active (if not bandwidth is zero), and return the current
best estimate for the available bandwidth of the link. If the link,
isn't one of ours, then we can thread through the total maximal
capacity of the link.
In order to implement this, the missionControl struct will now query the
switch upon creation to obtain a fresh bandwidth snapshot. We take care
to do this in a distinct db transaction in order to now introduced a
circular waiting condition between the mutexes in bolt, and the channel
state machine.
The aim of this change is to reduce the number of unnecessary failures
during HTLC payment routing as we'll now skip any links that are
inactive, or just don't have enough bandwidth for the payment. Nodes
that have several hundred channels (all of which in various states of
activity and available bandwidth) should see a nice gain from this w.r.t
payment latency.
This commit adds a simple scheduling mechanism for
resolving potential deadlocks when dropping a stale
connection (via pubkey inspection).
Ideally, we'd like to wait to activate a new peer until
the previous one has exited entirely. However, the current
logic attempts to disconnect (and wait) until the peer
has been cleaned up fully, which can result in
deadlocks with other portions of the codebase, since
other blocking methods may also need acquire the mutex
before the peer can exit.
When existing connections are replaced, they now
schedule a callback that is executed inside the
peerTerminationWatcher. Since the peer now waits for
the clean exit of the prior peer, this callback is
now executed with a clean slate, adds the peer to
the server's maps, and initiates peer's Start() method.
This skips creating errChans when sending messages to
peer during broadcast. This should be a minor memory
optimization, as well as not requiring channel sends
on those which will never be read.
In this commit, we ensure that any time we send a TempChannelFailure
that's destined for a multi-hop source sender, then we'll always package
the latest channel update along with it.
This commit make the server populate the ChainArbitrator's
ContractBreach method, by a method that will reliably handoff the breach
event ot the breachArbiter. The server will now forward the breach event
to the breachArbiter, and only let the closure return a non-nil error
when the breachArbiter ACKs this event.
In this commit, we fix a minor logging bug introduced in a prior commit.
Before we would directly modify the *net.TCPAddr that was a part of the
brontide connection. This achieved our goal, but would print weird log
messages as we mutated the port in the already established connection.
In this commit, we fix that by ensuring we create a copy iff it's a
net.TCPAddr, then modify that and replace the instance in the
lnwire.NetAddress.
Fixes#991.
This commits changes the behavior of our connection
reestablishment, and resolves some minor issues that
could lead to uncancelled requests or an infinite
connection loop.
- Will not attempt to Remove connection requests with
an ID of 0. This can happen for reconnect attempts
that get scheduled, but have not started at the
time the server cancels the connection requests.
- Adds a per-peer cancellation channel, that is
closed upon a successful inbound or outbound
connection. The goroutine spwaned to handle the
reconnect by the peerTerminationWatch now
selects on this channel, and skips reconnecting
if it is closed before the backoff matures.
- Properly computes the backoff when no entry in
persistentPeersBackoff is found. Previously, a
value of 0 would be returned, cause all subsequent
backoff attempts to use a backoff of 0.
- Cancels a peers retries and remove connections
immediately after receiving an inbound connection,
to mimic the structure of OutboundPeerConnected.
- Cancels all persistent connection requests after
calling DisconnectPeers.
- Allow additional connection attempts to peers, even if
there already exists a pending connection attempt.
In this commit, we fix an existing bug within the codebase: if a peer
connected to us inbound, then we'd attempt to use the assigned port when
re-establishing a connection to them. We fix this issue in this commit
by adding a new method to look up any advertisements for the peer, and
use the specified port that matches our connection attempt. If we can't
find a proper advertisement, then we'll simply use the default peer
port.
In this commit we modify the storage location of the sphinx replay
database to be under the precise network, and not only the graph sub
directory. Before this commit, due to the usage of filepath.Dir(), the
db would lie under /graph/, rather than say, /graph/simnet.
This commit adds an backoff policy to the peer termination
watcher to avoid getting stuck in tight connection loops
with failing peers. The maximum backoff is now set to 128s,
and each backoff is randomized so that two instances using
the same algorithm have some hope of desynchronizing.
This commit adds the `lnnet` package which contains an
implementation of the newly created LightningNet interface which
multiplexes the Dial and DNS-related functions to use net
by default and torsvc if a flag is specified. This modularization
makes for cleaner code.
This commit adds a new interface named NetInterface and two
implementations of it: RegularNet & TorProxyNet. These two structs
are used in config.go in an attempt to clean up the code and
abstract away the dialer and DNS functions.
This commit adds a new module named 'torsvc' which houses all Tor
functionality in an attempt to isolate it and make it reusable in
other projecs. Some additional tweaks were made to config.go and
to the bootstrapper.
This commit adds Tor support. Users can set the --TorSocks flag
to specify which port Tor's SOCKS5 proxy is listening on so that
lnd can connect to it. When this flag is set, ALL traffic gets
routed over Tor including DNS traffic. Special functions for
DNS lookups were added, and since Tor doesn't natively support
SRV requests, the proxySRV function routes connects us to
a DNS server via Tor and SRV requests can be issued directly
to the DNS server.
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
A recent commit modified the mutex in the server to the read/write. In
order to further reduce contention, we’ll grab the read lock when we’re
examining get set of peers to ignore.
This commit aims to reduce the contention on the server's primary
mutex by (1) replacing it with a RWMutex, and (2) not holding an
exclusive lock throughout the duration of the SendToPeer method.
For the latter, we now only have to hold a shared lock until all
messages have been queued, where as before an exclusive lock
was held until all messages had been acknowledged by the target
peer's write handler. Since the initial syncing of announcements
with peer now comprises a couple thousand messages on testnet,
these changes should help keep the server from deadlocking while
completing and long-lived syncing operations.
In this commit, we modify the way that notifications are dispatched
within the chainWatcher. Before we would *always* wait for an ack back
before we started to clean up he database state. This would at times
lead to deadlocks. To remedy this, we now allow callers to decide if
they want notifications to be sync or not. The only current caller that
requires this is the breach arbiter.
In this commit, we add the IsOurAddress field into the config of the
chain arb. With this new function closure, the chain arb is able to
detect co-op on chain closes automatically.
In this commit, we modify the breach arbiter to no longer require
holding a channel object directly in order to receive new notifications
about possible breaches. Instead, we’ll contact the chain arbiter to
request a new channel event subscription.
As a result of the new architecture, we no longer need to receive a
handoff once the new channel comes online, as the chainWatcher will
always be active and watching the channel until it’s been closed.
Peers are treated as transient by default. When a peer is disconnected,
no attempt is made to reconnect. However, if we have a channel open
with a peer that peer will be added as persistent. If a persistent peer
becomes disconnected then we will attempt to reconnect.
This behavior implies that a fresh node - those without any channels -
will fall off the network over time as remote nodes restart or due to
connectivity changes. This change marks bootstrap peers as persistent
and ensures that the node remains connected to those specific peers over
time. This does not keep the node connected in the case that all
bootstrap peers are down.
Fixes#451.
In this commit, we modify the Broadcast to take a *set* of peers to
skip, rather than just a single peer. We make this modification as when
a new channel is discovered, it’s likely the case that we get the
announcement from several peers rather than a single peer. With this
change, we’ll ensure that the caller (who is aware of the set of
senders) is able to properly avoid wasting bandwidth by re-sending the
message to all peers that sent it to us originally.
This commit fixes a deadlock that could occur when
a peer disconnected during a call to sentToPeer. In
This particular case, a message would successfully
be queued, the peer would shutdown, and we would
block waiting for an error to be returned on the
message's error channel, which would deadlock.
This fixes that by also checking for peer shutdown.
This commit reorders logic in the peer termination
watcher such that we short circuit earlier if we
already have pending persistent connection requests.
Before, the debug statement may have indicated that
a reconnection was being attempted, even though it
may have exited before submitting the request to
the connection manager.
In this commit, we fix an existing bug that would cause funding
transaction to be broadcast without any fees attached at all. This is
only an issue if the fee rate reported is extremely so, as can happen
on testnet. In this case, when we went to scale down to sat/weight, we
would return a value of zero due to integer division. If we went via
the EstimateFeePerWeight call directly, then it would've been detected.
However, we accept the fee/byte from the user directly on the command
line this wasn't being done.
To fix this, we'll now manually set the fee to a sane value, if it
returns a value that can't properly be scaled to fee/weight.
In this commit, we fix a bug that would cause the DNS seeds to be
*always* active regardless of which chain+network we were on. Before we
would look up the network in the reverseChainMap. However, if we were
on regtest or testate, then it would still (incorrectly) resolve to a
valid hash.
To remedy this, we now directly use the genesis hash of the current
active chain.
In this commit, we fix an existing issue that could at times cause an
inconsistent view between the set of total coins, and the set of segwit
coins in the wallet of the node. This could be caused by initiating a
funding flow, but then the funding negotiation breaking down somewhere
along the lines. In this case, us or the other peer will disconnect.
When we initiate funding flows, we lock coins exclusively, to ensure
that concurrent funding flows don’t end up double spending the same
coin. Before this commit, we wouldn’t ever unlock those coins. As a
result, our view of available coins would be skewed.
The walletbalance call would show all the coins, but when adding the
—witness_only flag, some coins would be missing, or gone all together.
This is because the former call actually scans the txstore and manually
tallies the amount of available coins, while the latter looks at the
sent of available outputs, which is filtered based on which coins are
locked.
To remedy this, we now ensure that when a peer disconnects, we wipe all
existing reservations which will return any locked outputs to the set
of available outputs for funding flows.
Add option to set trickleDelay for AuthenticatedGossiper in
command line, with default value of 300 milliseconds. Pass this
value to newServer, which uses it when creating a new instance of
AuthenticatedGossiper. Also set this value to 300 milliseconds when
creating nodes in integration tests.
In this commit, we add a new method shouldRequestGraphSync which the
server will use in order to determine if we should request a full
channel graph sync from a newly connected remote peer. Atm, we’ll only
request a full sync iff, we have less than two peers. This is only the
initial basic logic, as we’ll later extend this to be more
comprehensive.
With this change, we’ll no longer be blasted by full channel graph
dumps for _each_ new connection after we deem that we’ve been
sufficiently bootstrapped to the network.
In this commit we add the set of local features advertised as a
parameter to the newPeer function. With this change, the server will be
able to programmatically determine _which_ bits should be set on a
connection basis, rather than re-using the same global set of bits for
each peer.
This commit adds a new field to the switch’s Config, namely the public
key of the backing lightning node. This field will soon be used to
return more detailed errors messages back to the ChannelRouter itself.
This commit removes another case of unnecessary blockage, by modifying
the sendToPeer method to be fully asynchronous. From the PoV of the
callers that utilize this method currently, there’s no reason to block
until the completion of this method. Additionally, as the graph grows
larger without more intelligent the number of messages sent during
initial dump will start to be prohibitive to waiting for full
completion before proceeding.
In this commit, we make the BroadcastMessage method on the server more
asynchronous by abandoning the two wait groups that it used for
synchronization. It has been observed that a circular waiting loop
between the AuthenticatedGossiper and a peer’s readHandler can cause
the system to dead lock.
By removing this unnecessary synchronization, we avoid the deadlock
case and allow the gossiper itself to no longer block in this scenario.
In this commit we modify the main loop within the peerBootstrapper
slightly to check for a sufficient amount of connections, _before_
checking to see if we need to back off the main loop. With this, we
avoid unnecessarily backing off unless an actual error occurs.
This commit implements 2-week zombie channel pruning. This means that
every GraphPruneInterval (currently set to one hour), we’ll scan the
channel graph, marking any channels which haven’t had *both* edges
updated in 2 weeks as a “zombie”. During the second pass, all “zombie”
channel are removed from the channel graph all together.
Adding this functionality means we’ll ensure that we maintain a
“healthy” network view, which will cut down on the number of failed
HTLC routing attempts, and also reflect an active portion of the graph.
This commit fixes an incorrect logging statement within the
peerBootstrapper goroutine. We we’re using a Debug method previously
when we should’ve been using Debugf in order to properly pass the
logging statement through.
This commit adds a listener queue for each peer, that can be used
to queue listeners that will be notified when the targetted peer
eventually comes online.
This commit adds a new primary goroutine to the server struct:
peerBootstrapper. If peer boostrapping isn’t disabled in the config,
this new goroutine will be launched to attempt to establish a set of
initial connections for a new node. The logic is pretty straight
forward: first a set of initial connections is attempted, if after our
first epoch, we don’t have enough connections yet, then we’ll attempt
to query for an additional set. In each iteration, if we haven’t been
successful, then we increase our exponential backoff in order to not
spam any of our bootstrapping sources.
This commit alters the synchronization patterns used in the server
such that the internal state is protected by a single mutex. Overall,
this simplifies the ability to reason about the behavior and
manipulation of the internal state, which has resolved a few of flakes
related to race conditions that were observed before hand.
Invoking DisconnectPeer is now fully synchronous, and waits until
the provided peer's peerTerminationWatcher has exited before
returning. Currently this is done by tracking the watcher using the
peer's WaitGroup, and locking until the peer has shutdown.
The server's API has also been refactored such that all public methods
are safe for concurrent use. Therefore, other subsystems should be
sure to make use of these endpoints to avoid corrupting the internal
state.
This commit adds a new utility method to the server struct itself. This
method will allow callers to query the state of the server in order to
decide if the server has been started or not. This can be useful
elsewhere in the project as we start to decouple the lifetime of
certain sub-systems from others.
This commit updates the main single-funder funding workflow within the
fundingManager (initiated via the rpcserver or by a message from a
connected peer) to fully adhere to the funding protocol outlined in
BOLT-0002.
The major changes are as follows:
* All messages modified to use the new funding messages in BOLT-0002.
* The initiator of a funding workflow no longer decides how many
confirmations must elapse before the channel can be considered open.
* Rather than each side specifying their desired CSV delay, both
sides now specify the CSV delay for the _other_ party.
In previous commits we have intoduced the onion errors. Some of this
errors include lnwire.ChannelUpdate message. In order to change
topology accordingly to the received error, from nodes where failure
have occured, we have to propogate the update to the router subsystem.
In this commit daemon have been changed to set the proper hooks in the
channel link and switch subsystems so that they could send and receive
encrypted onion errors.
This commit overhauls the way that lnd is created with the goal of
ensuring the chain backends are fully synced up before the daemon
itself starts. The rpcServer has been slightly decoupled from the
server itself s.t we can start the rpcServer independently of the
server. This is required as we’ll now wait (unless we’re in simnet
mode) for the chain to fully sync up before we even _start_ any of the
server’s goroutines.
This commit modifies the Peers method on the server such that a caller
is able to query this method without the main serger goroutines
started. This is a small component in a larger change which will let us
start the RPC server independently of the server.
In current commit big shift have been made in direction of unit testable
payments scenarios. Previosly two additional structures have been added
which had been spreaded in the lnd package before, and now we apply
them in the lnd itself:
1. ChannelLink - is an interface which represents the subsystem for
managing the incoming htlc requests, applying the changes to the
channel, and also propagating/forwarding it to htlc switch.
2. Switch - is a central messaging bus for all incoming/outgoing htlc's.
The goal of the switch is forward the incoming/outgoing htlc messages
from one channel to another, and also propagate the settle/fail htlc
messages back to original requester.
With this abtractions the folowing schema becomes nearly complete:
abstraction
^
|
| - - - - - - - - - - - - Lightning - - - - - - - - - - - - -
|
| (Switch) (Switch) (Switch)
| Alice <-- channel link --> Bob <-- channel link --> Carol
|
| - - - - - - - - - - - - - TCP - - - - - - - - - - - - - - -
|
| (Peer) (Peer) (Peer)
| Alice <----- tcp conn --> Bob <---- tcp conn -----> Carol
This commit adds the FeeEstimator interface, which can be used for
future fee calculation implementations. Currently, there is only the
StaticFeeEstimator implementation, which returns the same fee rate for
any transaction.
This commit fixes a bug that was introduced when the connection
handling was re-worked to properly handle the case of concurrent
connections being made. In certain cases after a successful initial
connection, a peer’s stray goroutine would still attempt to establish a
second outbound connection even though a connection had already been
established. This was properly handled by the connecting peer, but not
he receiving peer. This commit adds the additional logic to the
receiving peer to ensure that we properly handle this case.
In this commit waiting proofs array have been replaced with persistant
boltd storage which removes the possibility for the half proof to be
lost during half proof exchange.
This commit modifies the implementation of the new DisconnectPeer RPC
in the following ways:
* all validation has moved from the server to the rpcserver
* rather than iterating over _all_ channels, we now only check the
peer’s channels
* the disconnectPeerMsg now has a public key object
* this allows us to also verify that the user submitted a valid
pub key string
* we now check if a peer was persistent when disconnecting so we can
remove them from the persistent peer map
This commit fixes a prior bug wherein if a user connected to a peer
using the —perm command, then once the peer was disconnected, we
wouldn’t automatically connect to them.
Issue: 139
This commit contains client-side and server-side functionality
for disconnecting peers. rpc-client calls server side method and sends
message with pubKey.
This commit fixes a bug that was introduced when the concurrent
connection handling logic was re-written: if we don’t properly add the
persistent outbound connection to the persistent conn reqs map. The fix
is easy: add the pending conn req to the proper map.
This commit fixes a bug that would possibly result in tens of goroutine
beaching launched in an attempt to persistently connect to a peer. This
bug has been fixed by ensuring that we’ll only launch a new pending
connection attempt if we don’t already have one pending.
The prior methods we employed to handle persistent connections could
result in the following situation: both peers come up, and
_concurrently_ establish connection to each other. With the prior
logic, at this point, both connections would be terminated as each peer
would go to kill the connection of the other peer. In order to resolve
this issue in this commit, we’ve re-written the way we handle
persistent connections.
The eliminate the issue described above, in the case of concurrent peer
connection, we now use a deterministic method to decide _which_
connection should be closed. The following rule governs which
connection should be closed: the connection of the peer with the
“smaller” public key should be closed. With this rule we now avoid the
issue described above.
Additionally, each peer now gains a peerTerminationWatcher which waits
until a peer has been disconnected, and then cleans up all resources
allocated to the peer, notifies relevant sub-systems of its demise, and
finally handles re-connecting to the peer if it's persistent. This
replaces the goroutine that was spawned in the old version of
peer.Disconnect().
This commit re-writes the GetNetworkInfo implenetaiton to use a single
database transaction. We’re now able to do this due to the recent
change in the API for the ChannelGraph struct and it’s related objects.
The recent change allows the passed callback to accept a db
transaction, with this, the callback is now able to issue another
traversal routine _within_ the prior one.
This commit modifies the fundingManager config to use the a SignMesage
function rather than two distinct functions for singing one half the
channel announcement proofs. This change unifies the signing of
messages under a single abstraction: the MessageSigner interface.
This commit eliminates a possible deadlock (or repeated peer connection
failures) that can arise due to the [inbound|outbound]PeerConnected
methods holding the peer mutex too long. We now alleviate this
concurrency issue by calling s.peerConnected in an asynchronous manner.
This is safe as all operations within the method are themselves
goroutine-safe.
This commit implements some minor coding style, commenting and naming
clean up after the recent major discovery service was merged into the
codebase.
Highlights of the naming changes:
* fundingManager.SendToDiscovery -> SendAnnouncement
* discovery.Discovery -> discovery.AuthenticatedGossiper
The rest of the changes consist primary of grammar fixes and proper
column wrapping.
Add usage of the 'discovery' package in the lnd, now discovery service
will be handle all lnwire announcement messages and send them to the
remote party.
This commit modifies the logic around the opening p2p handshake to
enforce a strict timeout around the receipt of the responding init
message. Before this commit, it was possible for the daemon and certain
RPC calls to deadlock as if a peer connected, but didn’t respond with
an init msg, then we’d be sitting there waiting for them to respond.
With this commit, we’ll now time out, kill the connection and then
possible attempt to re-connect if the connection was persistent.
Use addresses and ports from NodeAnnouncement messages for reconnection
attempts. For those nodes that don't explicitly report IP addresses, use
the IP address from previous connections connection request along with
the default peer port number.
Minor change to server.go to add ExternalIPs to
channeldb.LightningNode. Also, added a test that utilizes this
functionality and exercises multiple addresses in NodeAnnouncement.
This commit modifies address handling in the NodeAnnouncement struct,
switching from net.TCPAddr to []net.Addr. This enables more flexible
address handling with multiple types and multiple addresses for each
node. This commit addresses the first part of issue #131 .
If an error occurs during, peer initialization then 'p' is nil. This
may cause a panic while accessing the peer's member
variables.
To avoid such panics, we now omit the call to `p.Disconnect`
and also directly access the `connmgr.ConReq` variable if it's
non-nil.
This commit removes all instances of the fastsha256 library and
replaces it with the sha256 library in the standard library. This
change should see a number of performance improvements as the standard
library has highly optimized assembly instructions with use vectorized
instructions as the platform supports.
This commit renames routing processing method in the funding mangers
config from ProcessRoutingMessage to SendToRouter and also modifies the
signature to only require the message itself and not the server’s
identity public key.
When the funding transaction has been confirmed, the FundingLocked
message is sent by the peers to each other so that the existence of the
newly funded channel can be announced to the network.
This commit also removes the SingleFundingOpenProof message.
Once a channel funding process has advanced to the point of broadcasting
the funding transaction, the state of the channel should be persisted
so that the nodes can disconnect or go down without having to wait for the
funding transaction to be confirmed on the blockchain.
Previously, the finalization of the funding process was handled by a
combination of the funding manager, the peer and the wallet, but if
the remote peer is no longer online or no longer connected, this flow
will no longer work. This commit moves all funding steps following
the transaction broadcast into the funding manager, which is available
as long as the daemon is running.
github.com/lightningnetwork/lnd master ✗
0m ◒
▶ golint
htlcswitch.go:292:4: should replace numUpdates += 1 with numUpdates++
htlcswitch.go:554:6: var onionId should be onionID
htlcswitch.go:629:7: var onionId should be onionID
lnd_test.go:133:1: context.Context should be the first parameter of a
function
lnd_test.go:177:1: context.Context should be the first parameter of a
function
networktest.go:84:2: struct field nodeId should be nodeID
peer.go:1704:16: should omit 2nd value from range; this loop is
equivalent to `for invoice := range ...`
rpcserver.go:57:6: func newRpcServer should be newRPCServer
github.com/lightningnetwork/lnd master ✗
9m ⚑ ◒ ⍉
▶ go vet
features.go:12: github.com/lightningnetwork/lnd/lnwire.Feature
composite literal uses unkeyed fields
fundingmanager.go:380: no formatting directive in Errorf call
exit status 1
Previously, during the channel funding process, peers sent wire
messages using peer.queueMsg. By switching to server.sendToPeer, the
fundingManager is more resilient to network connection issues or system
restarts during the funding process. With server.sendToPeer, if a peer
gets disconnected, the daemon can attempt to reconnect and continue the
process using the peer’s public key ID.
This commit prevent unnecessary connection flapping by ensure we don’t
attempt to auto-connect to a peer that we’re unable to create or start
the goroutines of. With this commit, we won’t attempt to auto-connect
to a peer that has incompatible feature sets to that of ours.
In this commit the support for global and local feature vectors were
added in 'server' and 'peer' structures respectively. Also with commit
additional logic was added and now node waits to receive 'init'
lnwire.Message before sending/responding on any other messages.
This commit modifies the existing syncing logic to launch a new
goroutine on connect to synchronize graph state with the new peer
rather than the prior blocking version. This change should make the
initial sync a little snappier and also possible eliminate a circular
dependency between the ChannelRouter and the server.
This commit moves much of the logic for querying for a potential route,
constructing the HTLC including the Sphinx packet, and sending the
ultimate payment from the rpcServer to the ChannelRouter.
This movement paves the way for muilt-path path finding as well as
adding automatic retry logic to the ChannelRouter. Additionally, by
having the ChannelRouter construct the Sphinx packet, we’ll be able to
also include the proper time-lock and general per-hop-payload
information properly in the future.
This commit fixes a goroutine closure bug introduced by a prior commit.
A prior commit launched a goroutine for each peer to broadcast the
messages in parallel. However, as written this caused the messages to
only be broadcast to a single peer. When launching goroutines in a
for-loop, the “range” variable is actually re-used and re-assigned
within each iteration of the for-loop. As a result, all goroutines
launched will bind onto the _same_ instance of the variable.
We fix this bug in this commit by properly binding the target peer to a
new variable within the closure that launches the goroutine.
Relevant sources:
*
https://github.com/golang/go/wiki/CommonMistakes#using-goroutines-on-loo
p-iterator-variables
* https://golang.org/doc/faq#closures_and_goroutines
This commit slightly optimizes the process of broadcasting a message to
a list of peers, and also sending a set of messages to a target peer.
When broadcasting a message to a set of target peers, we now launch a
goroutine for each send as to not block the ChannelRouter on an
individual send. When sending a set of messages to a target peer, we
now give up the mutex as soon as we’ve access the map, rather than
holding onto it until the sending is complete.
This commit modifies the request handling within the sever’s
queryHandler goroutine to ensure that requests from the ChannelRouter
or other related sub-systems don’t block the main processing loop.
We do this simply by launching a goroutine to handle the dispatch of
the request.
Before this commit there was the possibility of a race occurring
between a call to the “lispers” cli command and the normal operation of
peers being connected and disconnected. With this commit, we now ensure
such a race doesn’t occur by properly acquiring the lock for peersByID
before accessing it.
This commit reverts a prior commit as it broke the integration tests
based on the assumption that all peers use the default port within the
network. The issue which was attempted to be fixed will be remedied
with a patch to the connmgr that allows a caller to cancel a persistent
connection that has failed.
This commit fixes a slight bug in the deamon. Previously we would store
the *net.TCPAddr that we observed when we either connected out to the
peer, or the peer connected to us. When making an outgoing connection
the host+port combination would be correct, but when responding to an
incoming connection, the port assigned after the TCP handshake would be
stored in the database. This would cause many goroutines to repeatedly
fail connections within the connmgr. Atm within the connmgr, it isn’t
possible to cancel requests for failed connection even after we’ve
already established a connection.
This commit fixes that issues by using the default peer port when
attempting to establish outbound connections to our channel peers.
This commit fixes a bug introduced by the past attempt to Make Logging
Great Again. Since we unset the curve parameters when reading/writing
the messages, if we have a lingering reference that’s active elsewhere
in the daemon, then we’ll modify that reference. To fix this, we now
explicitly set the Curve parameters in two areas.
A similar commit has been pushed to lightning-onion.
This commit fixes a slight bug in the interaction between the cli
program and the rpcsever itself. With this commit it’s now again
possible to create a channel with a peer that’s identified by its
peerID, instead of only the pubkey.
This commit modifies the ConnectPeer RPC call and partitions the
behavior of the call into two scenarios: the connection should be
persistent which causes the call to be non-blocking, and the connection
should only attempt to connect once — which causes the call to be
blocking and report any error back to the caller.
As a result, the pendingConnRequest map and the logic around it is no
longer needed.
This commit adds daemon level support for pushing funds as part of the
single funder channel workflow. This new feature allows the user to
open a channel and simultaneously make a channel at the same time which
can improve the UX when setting up a channel for the first time.
Moved transaction states from in-memory maps to persistent BoltDB
buckets. This allows channel force closes to operate reliably if the
daemon is shut down and restarted at any point during the forced
channel closure process.
This commit fully integrates the ChannelRouter of the new routing
package into the main lnd daemon.
A number of changes have been made to properly support the new
authenticated gossiping scheme.
Two new messages have been added to the server which allow outside
services to: send a message to all peers possible excluding one, and
send a series of messages to a single peer. These two new capabilities
are used by the ChannelRouter to gossip new accepted announcements and
also to synchronize graph state with a new peer on initial connect.
The switch no longer needs a pointer to the routing state machine as it
no longer needs to report when channels closed since the channel
closures will be detected by the ChannelRouter during graph pruning
when a new block comes in.
Finally, the funding manager now crafts the proper authenticated
announcement to send to the ChannelRouter once a new channel has bene
fully confirmed. As a place holder we have fake signatures everywhere
since we don’t properly store the funding keys and haven’t yet adapted
the Signer interface (or create a new one) that abstracts out the
process of signing a generic interface.
This commit revamps the way in bound and outbound connections are
handled within lnd. Instead of manually managing listening goroutines
and also outbound connections, all the duty is now assigned to the
connmgr, a new btcsuite package.
The connmgr now handles accepting inbound (brontide) connections and
communicates with the server to hand off new connections via a
callback. Additionally, any outbound connection attempt is now made
persistent by default, with the assumption that (for right now),
connections are only to be made to peers we wish to make connections
to. Finally, on start-up we now attempt to connection to all/any of our
direct channel counter parties in order to promote the availability of
our channels to the daemon itself and any RPC users.
This commit introduces a new sub-system into the daemon whose job it is
to vigilantly watch for any potential channel breaches throughout the
up-time of the daemon. The logic which was moved from the utxoNursery
in a prior commit now resides within the breachArbiter.
Upon start-up the breachArbiter will query the database for all active
channels, launching a goroutine for each channel in order to be able to
take action if a channel breach is detected. The breachArbiter is also
responsible for notifying the htlcSwitch about channel breaches in
order to black-list the breached linked during any multi-hop forwarding
decisions.
Use [33]byte for graph vertex representation.
Delete unneeded stuff:
1. DeepEqual for graph comparison
2. EdgePath
3. 2-thread BFS
4. Table transfer messages and neighborhood radius
5. Beacons
Refactor:
1. Change ID to Vertex
2. Test use table driven approach
3. Add comments
4. Make graph internal representation private
5. Use wire.OutPoint as EdgeId
6. Decouple routing messages from routing implementation
7. Delete Async methods
8. Delete unneeded channels and priority buffer from manager
9. Delete unneeded interfaces in internal graph realisation
10. Renamed ID to Vertex
This commit modifies the existing p2p connection authentication and
encryption scheme to now use the newly designed ‘brontide’
authenticated key agreement scheme for all connections.
Additionally, within the daemon lnwire.NetAddress is now used within
all peers which encapsulates host information, a node’s identity public
key relevant services, and supported bitcoin nets.
This commit adds a few workarounds in order to concurrently support the
REST proxy as well as the regular gRPC interface. Additionally,
concrete support for the following RPC calls has been added:
GetTransactions, SubscriptTransactions, SubscribeInvoices, and
NewWitnessAddress.
This commit properly removes any/all closed channels from the routing
table. In the current implementation individual links (channels)
between nodes are treated sparely from the PoV of the routing table. In
the future, this behavior should be modified such that, the routing
table views all the links between nodes as a single channel. Such a
change will simplify the task of path finding as the links can simply
be viewed as a channel with the sum of their capacities. The link layer
(htlcSwitch) will handle the details of fragmentation on a local basis.
This commit alters the send/receive HTLC pipe line a bit in order to
fully integrate onion routing into the daemon.
The server now stores the global Sphinx router which all active
htlcManagers will used when processing upstream HTLC add messages.
Currently the onion routing private key is static, and identical to the
node’s current identity public key. In the future this key will be
rotated daily the node based on the current block hash.
When sending a payment via the SendPayment RPC, the routing manager is
now queried for the existence of a route before the payment request is
sent to the HTLC switch. If a path is found, then a Sphinx onion packet
encoding the route is created, then populated within the HTLC add
message.
Finally, when processing an upstream HTLC add request, the sphinx
packet is decoded, then processed by the target peer. If the peer is
indicated as the exit node, then the HTLC is queue’d to be settled
within the next state update.
This commit adds a new configuration parameter to the deamon:
‘DebugHTLC’. When true, all outgoing HTLC’s sent via the RPC interface
will be sent paying to a special rHash value which all lnd nodes also
with the flag activated know the preimage to. Therefore all payments
sent to a 1-hop node will immediately be settled by that node.
By default, this flag is false, it it only intended to be used to
exercise local changes to 1-hop behavior manually.
This commit extends the existing invoiceRegistry functionality to wrap
the on-disk invoices available via the channeldb with an in-memory
cache on invoices. Currently the in-memory cache is only reserved for
the storage of special “debug” invoices which all nodes are able to
settle immediately.
This commit introduces the utxoNursery. The duty of the utxoNursery is
to watch over CSV-locked immature outputs until they’ve fully matured.
An output is mature once both its sequence lock indicated by the CSV op
code within its output has become active. Once an output is mature the
nursery sweeps the outputs in batches into the source wallet.
The utxoNursery executes its duties once a commitment transaction has
been broadcast on-chain.
This commit introduces the concept of a manually initiated “force”
closer within the channel state machine. A force closure is a closure
initiated by a local subsystem which broadcasts the current commitment
state directly on-chain rather than attempting to cooperatively
negotiate a closure with the remote party.
A force closure returns a ForceCloseSummary which includes all the
details required for claiming all rightfully owned outputs within the
broadcast commitment transaction.
Additionally two new publicly exported channels are introduced, one
which is closed due a locally initiated force closure, and the other
which is closed once we detect that the remote party has executed a
unilateral closure by broadcasting their version of the commitment
transaction.
LIGHT-138, LIGHT-141. Due to some issues in sending/receiving parts of lnd,
messages with zero length are not sent. So added some mock content to
NeighborAck. Moved sender/receiver from routing message to wrap message
which contains lnwire routing message.
This commit modifies the daemon’s initialization within the `lndMain`
method to create an instance of the current default ChainNotifier
outside of the LightningWallet.
At this point, since there are no other implementations of the
ChainNotifier, the current concrete implementation BtcdNotifier is used
by default. In the future, once other ChainNotifier implementations are
in place, config parsing should be fed into a factory function which
creates the proper ChainNotifier implementation.
Finally, several imports have been updated to reflect the change in
package name.
This commit modifies the internal workflow for opening or closing a
channel in order to create a path in which RPC clients can receive
updates. Updates are now communicated via channels from the goroutines
spawned by the RPC server to process the request, and the sub-system
within the daemon that actually executes the request.
With this change clients can now receive updates that the request is
pending (final message has been sent to the target client), or that the
request has been completed. Confirmation related updates have not yet
been implemented as that will require some changes to the ChainNotifier
interface.
This commit integrates BitFury's current routing functionality into lnd. The
primary ochestration point for the routing sub-system in the routingMgr. The
routingMgr manages all persistent and volatile state related to routing within
the network.
Newly opened channels, either when the initiator or responder are inserted into
the routing table once the channel is fully open. Once new links are inserted
the routingMgr can then perform path selection in order to locate an "optimal"
path to a target destination.
This commit adds a basic test for cooperative channel closure. The
current test ensures correctness of the cooperative closure procedure
initiated by either the channel initiator, or the channel responder.
This commit introduces the invoice registry which is a central
repository of all outstanding invoices related to the daemon.
This registry will be used by the goroutines which manage the htlc’s
for a particular channel, and later by the point-to-point workflow
which negotiates the conditions for payment along with signed recipes.
This commit switches the implementation of the open/close channel RPC’s
from a fully blocking synchronous model to one that’s async by default,
allowing callers to add a sync wrapper.
The new proto specs also allow for “updates” for the pending channels
in the form of new confirmations which progress the pending status of
the channel. At this point, only the final open/close updates have been
implemented. Obtaining confirmation notifications requires a bit of
re-working within the current ChainNotifier interface, thus this has
been deferred to a later time.
This commit adds the necessary plumbing within the server, peer, and
rpcServer to handle opening and cooperatively closing a channel with a
remote peer.
Many new data structures have been added to the peer in order to allow
it to efficiently manage opening+.losing new/existing lightning
channels. Additional documentation has been added to several methods
within the peer struct, with some minor renaming along with way. The
peer has also gained a dedicated goroutine whose job it is to manage
any requests pertaining to opening, or closing any channels with the
remote peer.
The messages have been added to lnrpc define the requests and responses
to channel open+close messages. Additional channel logic has been added
between the rpcServer, peer, and server in order to properly manage the
necessary synchronization.
This commit adds some cursory documentation along wit minor field
modifications to all messages which deal with adding HTLC’s, or
updating remote commitment transactions.
The messages for dual funding of channel is left purposefully
undocumented as all initial negotiations will be single funder by
default.
A revamp of the testing infrastructure of lnwire will be committed in
the near future.
