This commit overhauls the way that lnd is created with the goal of
ensuring the chain backends are fully synced up before the daemon
itself starts. The rpcServer has been slightly decoupled from the
server itself s.t we can start the rpcServer independently of the
server. This is required as we’ll now wait (unless we’re in simnet
mode) for the chain to fully sync up before we even _start_ any of the
server’s goroutines.
Previously the integers would either underflow or the floating point
division would result in odd numbers if a node knew of zero channels
within the network graph. We now fix this situation by checking to see
if any channels exist before computing the network stats.
Fixes#211.
In order to not close the payment stream on payment error the additional
field have been added in payment response. Now error from stream Recv()
function means that something has happend inside the client and we unable
to process any payment farther, and error inside the payment response
means, that something wrong has happend with payment itself.
In current commit big shift have been made in direction of unit testable
payments scenarios. Previosly two additional structures have been added
which had been spreaded in the lnd package before, and now we apply
them in the lnd itself:
1. ChannelLink - is an interface which represents the subsystem for
managing the incoming htlc requests, applying the changes to the
channel, and also propagating/forwarding it to htlc switch.
2. Switch - is a central messaging bus for all incoming/outgoing htlc's.
The goal of the switch is forward the incoming/outgoing htlc messages
from one channel to another, and also propagate the settle/fail htlc
messages back to original requester.
With this abtractions the folowing schema becomes nearly complete:
abstraction
^
|
| - - - - - - - - - - - - Lightning - - - - - - - - - - - - -
|
| (Switch) (Switch) (Switch)
| Alice <-- channel link --> Bob <-- channel link --> Carol
|
| - - - - - - - - - - - - - TCP - - - - - - - - - - - - - - -
|
| (Peer) (Peer) (Peer)
| Alice <----- tcp conn --> Bob <---- tcp conn -----> Carol
This commit adds the FeeEstimator interface, which can be used for
future fee calculation implementations. Currently, there is only the
StaticFeeEstimator implementation, which returns the same fee rate for
any transaction.
This commit fixes a slight logic error that could render the
`pendingchannels` RPC unusable if a node was on the reciting end of a
channel force close with no time-locked balance. In such a case the
channel wouldn’t be sent to the utxoNursery, resulting in an “contract
not found error”.
To fix this behavior, we’ve created a typed error that can be checked
within the RPC, thus we no longer treat this routine case as an error
case.
This commit allows users to sign messages with their node's private key
with the SignMessage interface. The signatures are zbase32 encoded for
human readability/paste-ability. Others users can verify that a message
was signed by another node in their channel database with the
VerifyMessage interface.
This commit adds simple non-blocking stop command to lncli, with an
appropriate proto update and implementation within the rpcserver. When
invoked the interrupt handler routine in signal.go with begin the graceful
shutdown of lnd.
This commit modifies the implementation of the new DisconnectPeer RPC
in the following ways:
* all validation has moved from the server to the rpcserver
* rather than iterating over _all_ channels, we now only check the
peer’s channels
* the disconnectPeerMsg now has a public key object
* this allows us to also verify that the user submitted a valid
pub key string
* we now check if a peer was persistent when disconnecting so we can
remove them from the persistent peer map
Issue: 139
This commit contains client-side and server-side functionality
for disconnecting peers. rpc-client calls server side method and sends
message with pubKey.
This commit re-writes the GetNetworkInfo implenetaiton to use a single
database transaction. We’re now able to do this due to the recent
change in the API for the ChannelGraph struct and it’s related objects.
The recent change allows the passed callback to accept a db
transaction, with this, the callback is now able to issue another
traversal routine _within_ the prior one.
This commit fixes a prior oversight in the implementation of
SendPayment that could result in tens of thousands of goroutines
OOM’ing an lnd daemon. Previously we didn’t limit the number of
outstanding payments that were allowed by a client. Users on machines
with a small amount of RAM were reporting crashes when sending a very
large number of payments in a consistent stream. This commit fixes this
issue by now using a semaphore to limit the number of outstanding
payments (and therefore) goroutines allowed in the SendPayment method.
This commit slightly modifies the handling of the Connect RPC to allow
users to omit the port when specifying the target node to connect to.
If the port isn’t specified, then the default p2p port will be used in
place.
This commit modifies address handling in the NodeAnnouncement struct,
switching from net.TCPAddr to []net.Addr. This enables more flexible
address handling with multiple types and multiple addresses for each
node. This commit addresses the first part of issue #131 .
This commit removes all instances of the fastsha256 library and
replaces it with the sha256 library in the standard library. This
change should see a number of performance improvements as the standard
library has highly optimized assembly instructions with use vectorized
instructions as the platform supports.
This commit removes a number of sleeps from the set of current
integration tests by replacing them with a synchronous (w/ a timeout)
block until one or many channels are detected as being open within the
network.
As a result, the tests are now more robust, many flakes have been
eliminated, and finally this shaves a few second off of the integration
testing runs.
This commit modifies the two newly added network announcement hook stop
be at the lightningNode level rather than on the level of the entire
test framework. With this, callers are now able to better utilize the
newly added RPC’s since they can target particular peers and wait for
network messages to be processed rather then depending on a single node
(Alice) for information about the announcements propagated within the
network.
This commit implements the new server-side streaming RPC call within
the current default implementation of the RPC server. With this, the
new functionality can now be used within the integration tests to
achieve a greater degree of synchronization in the tests. As a result,
we should be able to eliminate many of the sleeps lingering within the
tests.
This commit adds a new field to the ListChannels RPC command which
indicates if the peer is currently online or not. This is useful as
UI’s will be able to use this information to communicate the
availability of each channel to an end user.
This commit updates all the RPC’s that deal with querying for data
stored within the graph to the latest version of the public API for the
graph itself.
With the funding manager’s state gaining full durability, it only will
return channels that are currently “pending” as a response to the
PendingChannels RPC call. As a result, the prior edge case where a user
would issue a PendingChannels RPC call mid funding flow is no longer
possible and channels are read from disk rather then from the
fundingMgr’s limbo state.
With the concept of open/pending channels on disk, we now will no
longer show channels under ListChannels that are still pending. This
fixes a slight existing bug wherein channels that weren’t yet open
would be listed in this RPC call since we write to disk immediately
after broadcasting the funding transaction.
In order to facilitate persistence during the funding process, added
the isPending flag to channels so that when the daemon restarts, we can
properly re-initialize the chain notifier and update the state of
channels that were going through the funding process.
github.com/lightningnetwork/lnd master ✗
0m ◒
▶ golint
htlcswitch.go:292:4: should replace numUpdates += 1 with numUpdates++
htlcswitch.go:554:6: var onionId should be onionID
htlcswitch.go:629:7: var onionId should be onionID
lnd_test.go:133:1: context.Context should be the first parameter of a
function
lnd_test.go:177:1: context.Context should be the first parameter of a
function
networktest.go:84:2: struct field nodeId should be nodeID
peer.go:1704:16: should omit 2nd value from range; this loop is
equivalent to `for invoice := range ...`
rpcserver.go:57:6: func newRpcServer should be newRPCServer
github.com/lightningnetwork/lnd master ✗
9m ⚑ ◒ ⍉
▶ go vet
features.go:12: github.com/lightningnetwork/lnd/lnwire.Feature
composite literal uses unkeyed fields
fundingmanager.go:380: no formatting directive in Errorf call
exit status 1
Previously, during the channel funding process, peers sent wire
messages using peer.queueMsg. By switching to server.sendToPeer, the
fundingManager is more resilient to network connection issues or system
restarts during the funding process. With server.sendToPeer, if a peer
gets disconnected, the daemon can attempt to reconnect and continue the
process using the peer’s public key ID.
This commit addresses an edge case which has been discovered by testers
of lnd of testnet. When/if channels get out of sync the unilateral
channel closure of a remote node may go undetected, which causes the
local node to lose their ability to purge the now closed channel from
their database state.
With this commit, if we try to force close a channel but detect it as
double-spent due to a prior commitment transaction being broadcast,
then we simply forget the channel as it has already been closed.
This commit removes the BlockChainIO interface as a dependency to the
LightningChannel struct as the interface is no longer used within the
operation of the LightningChannel.
This commit fixes a potential panic that could arise when one was
attempting to open a channel with the target peer identified by it’s
peerID and an error occurred. In this case, the nodepubKey pointer
would be nil, resulting in a panic when attempting to propagate the
error.
This commit fixes this bug by using a nil byte slice for the node’s
serialized pubkey in the case that the node was identified according to
ti’s peer ID.
This commit moves much of the logic for querying for a potential route,
constructing the HTLC including the Sphinx packet, and sending the
ultimate payment from the rpcServer to the ChannelRouter.
This movement paves the way for muilt-path path finding as well as
adding automatic retry logic to the ChannelRouter. Additionally, by
having the ChannelRouter construct the Sphinx packet, we’ll be able to
also include the proper time-lock and general per-hop-payload
information properly in the future.
This commit slightly optimizes the process of broadcasting a message to
a list of peers, and also sending a set of messages to a target peer.
When broadcasting a message to a set of target peers, we now launch a
goroutine for each send as to not block the ChannelRouter on an
individual send. When sending a set of messages to a target peer, we
now give up the mutex as soon as we’ve access the map, rather than
holding onto it until the sending is complete.
This commit adds a new restriction around funding channels at the
daemon level: lnd nodes will not allow either the initiation or the
acceptance of a channel before the node is fully synced to the best
known chain.
This fixes a class of bug that arises when a new node joins the network
and either attempts to open a channel or has a channel extended to them
before the node is fully synced to the network.
This commit implements the newly added RPC to decode payment requests
passed over the command line or directly via gRPC.
With this tool, users can now examine payment requests they see in the
wild for diagnostic or debugging purposes.
This commit adds a soft-limit for the minimum allowed channel size.
Without this limit a user may inadvertently create an invalid or
unspendable output due to the hard coded fees in a few areas of the
codebase.
This is a temporary measure, and will be removed once we add dynamic
fees into the mix.
This commit augments the server’s response to successful SendPayment
commands by also returning the full path taken in order to fulfill the
payment. With this information, the user now obtains more context about
their payment whcih can be useful when debugging, or just exploring the
capabilities of the daemon.
This commit fixes a slight bug in the interaction between the cli
program and the rpcsever itself. With this commit it’s now again
possible to create a channel with a peer that’s identified by its
peerID, instead of only the pubkey.
This commit modifies the ConnectPeer RPC call and partitions the
behavior of the call into two scenarios: the connection should be
persistent which causes the call to be non-blocking, and the connection
should only attempt to connect once — which causes the call to be
blocking and report any error back to the caller.
As a result, the pendingConnRequest map and the logic around it is no
longer needed.
This commit adds daemon level support for pushing funds as part of the
single funder channel workflow. This new feature allows the user to
open a channel and simultaneously make a channel at the same time which
can improve the UX when setting up a channel for the first time.
This commit makes a large number of minor changes concerning API usage
within the deamon to match the latest version on the upstream btcsuite
libraries.
The major changes are the switch from wire.ShaHash to chainhash.Hash,
and that wire.NewMsgTx() now takes a paramter indicating the version of
the transaction to be created.
This commit adds a much needed feature to the daemon, namely the
ability to force close a channel while the source daemon doesn’t have
an active connection to the counter party. Previously this wasn’t
possible as ALL channel closures were routed through the htlcSwitch
which is only able to trigger a channel closure if the peer is online.
To remedy this, if the closure type is “force” then, we now handle the
channel closure and related RPC streaming updates from the call handler
site of the RPC itself. As a result, there are now only two htlcSwitch
channel closure types: breach, and regular. The logic that’s now in the
rpcSever should likely be refactored into a distinct sub-system, but
getting the initial functionality in is important.
Finally, the channel breach integration test has been modified to skip
connection the peers before attempting the forceful channel closure of
a revoked state as the remote peer no longer needs to be online.
This commit modifies the generated response to an “AddInvoice” RPC by
including an encoded payment request in the response. This change gives
callers a new atomic piece of information that they can present to the
payee, to allow completion of the payment in a seamless manner.
This commit slightly refactors the logic for the new outgoing payment
related RPC’s to more closely match the style of the rest of the
codebase. Additionally the tests have been updated to reflect the
changes to the protos of the new RPC’s.
This commit expands the data returned by the current GetCurrentHeight
to also return the current best block hash, expanding the method into
GetBestBlock. Additionally, the current best BlockHash is also now
displayed within the GetInfo RPC call.
Go-fmt files. Refactored code according to the guidelines.
Enhanced payment test: add error checking
and individual context for each API call.
Add Timestamp field to payment struct.
This commit introduces a new sub-system into the daemon whose job it is
to vigilantly watch for any potential channel breaches throughout the
up-time of the daemon. The logic which was moved from the utxoNursery
in a prior commit now resides within the breachArbiter.
Upon start-up the breachArbiter will query the database for all active
channels, launching a goroutine for each channel in order to be able to
take action if a channel breach is detected. The breachArbiter is also
responsible for notifying the htlcSwitch about channel breaches in
order to black-list the breached linked during any multi-hop forwarding
decisions.
Use [33]byte for graph vertex representation.
Delete unneeded stuff:
1. DeepEqual for graph comparison
2. EdgePath
3. 2-thread BFS
4. Table transfer messages and neighborhood radius
5. Beacons
Refactor:
1. Change ID to Vertex
2. Test use table driven approach
3. Add comments
4. Make graph internal representation private
5. Use wire.OutPoint as EdgeId
6. Decouple routing messages from routing implementation
7. Delete Async methods
8. Delete unneeded channels and priority buffer from manager
9. Delete unneeded interfaces in internal graph realisation
10. Renamed ID to Vertex
This commit modifies the ChannelBalance RPC to fetch the balance from
disk since channels are now able to efficiently retrieved from disk due
to recent index that have been added. Previously this RPC would only
return accurate information if we had an active connection to the
counter party for each channel.
This RPC implements the two new RPC methods added in a prior commit.
This involved a slight refactoring to make use of duplicated code
amongst the sync and async variants of the methods.
This commit modifies both the Sphinx packet generation and processing
for recent updates to the API.
With the version 1 Sphinx specification, the payment hash is now
included in the MACs in order to thwart any potential replay attacks.
As a result, any attempts to replay previous HTLC packets MUST re-use
the same payment hash, meaning that the first-hop node can simply
settle the HTLC immediately, thwarting the attacker.
Additionally, within the Sphinx packet, each hop now gets a per-hop
payload which contains the necessary details (CTLV value, fee, etc) for
the node to successfully forward the payment. This per-hop payload is
protected by a packet-wide MAC.
This commit modifies the existing p2p connection authentication and
encryption scheme to now use the newly designed ‘brontide’
authenticated key agreement scheme for all connections.
Additionally, within the daemon lnwire.NetAddress is now used within
all peers which encapsulates host information, a node’s identity public
key relevant services, and supported bitcoin nets.
This commit takes advantage of the newly added
channeldb.FetchAllChannels method to return the state of all active
channels for the ListChannels RPC command. With this change the state
of all channels can now be queried regardless of if any/all the peers
are currently online.
In a future modification a bit will be added to the channel information
which indicates if the LinkNode the channel was created with is
currently online or not.
This commit modifies the server-side handling of the streaming
SendPayment RPC to launch a new goroutine which is dedicated to reading
new requests from the client from the bi-directional stream. This
modification decouples error handling from stream reading allowing
errors to be returned to the client as the arise rather than after the
next payment has been sent.
This commit adds a few workarounds in order to concurrently support the
REST proxy as well as the regular gRPC interface. Additionally,
concrete support for the following RPC calls has been added:
GetTransactions, SubscriptTransactions, SubscribeInvoices, and
NewWitnessAddress.
This commit modifies the on-disk storage of invoices to stop the
optional fields (memo+receipt) on-disk as variable length byte arrays.
This change saves space as the optional fields now only take up as much
space as is strictly needed, rather than always being padded out to max
size (1KB).
This commit modifies the sendpayment command slightly to use the pub
key of the *first* hop within the route as the destination within the
htlcPacket sent to the htlcSwitch.
Previously, since only single hop was implemented, the final
destination would be set within the htlcPkt, causing the route to fail
as the switch doesn’t have a direct link to the dest in the multi-hop
case.
This commit alters the send/receive HTLC pipe line a bit in order to
fully integrate onion routing into the daemon.
The server now stores the global Sphinx router which all active
htlcManagers will used when processing upstream HTLC add messages.
Currently the onion routing private key is static, and identical to the
node’s current identity public key. In the future this key will be
rotated daily the node based on the current block hash.
When sending a payment via the SendPayment RPC, the routing manager is
now queried for the existence of a route before the payment request is
sent to the HTLC switch. If a path is found, then a Sphinx onion packet
encoding the route is created, then populated within the HTLC add
message.
Finally, when processing an upstream HTLC add request, the sphinx
packet is decoded, then processed by the target peer. If the peer is
indicated as the exit node, then the HTLC is queue’d to be settled
within the next state update.
This commit adds a new configuration parameter to the deamon:
‘DebugHTLC’. When true, all outgoing HTLC’s sent via the RPC interface
will be sent paying to a special rHash value which all lnd nodes also
with the flag activated know the preimage to. Therefore all payments
sent to a 1-hop node will immediately be settled by that node.
By default, this flag is false, it it only intended to be used to
exercise local changes to 1-hop behavior manually.
This commit adds a new RPC command: `channelbalance` which returns the
sum of all available channel capacity across all open channels. The
total balance is currently returned in units of `satoshis`. Additionally
the `networkHarness` has been modified slightly to allow specifying the
additional "extra" command line parameters when creating the initial
seed nodes. Minor refactoring within the integration tests has been
undertaken in order to increase code re-use across tests.
Closes#29.
This commit includes some slight refactoring to properly execute force
closures which are initiated by RPC clients.
The CloseLink method within the htlcSwitch has been extended to take an
additional parameter which indicates if the link should be closed
forcefully. If so, then the channelManager which dispatches the request
executes a force closure using the target channel state machine. Once
the closing transaction has been broadcast, the summary is sent to the
utxoNursery so the outputs can be swept once they’re mature.
LIGHT-131, LIGHT-140, LIGHT-138
`lncli showroutingtable` may output routing table as image.
Use graphviz for graph rendering.
Add explicit version dependency for tools. Add error checking.
LIGHT-133, LIGHT-138 Make output of `lncli showrouting table` in
two different formats: table and json.
Instead of sending serialized routing table send list of channels.
This commit adds a new feature to the network harness enabling callers
to receive async notifications once a particular transaction is seen on
the network. Such a feature is useful when due to the asynchronous
behavior of node communications.
With this new feature, tests can now wait for a particular transaction
to be seen within the network before proceeding.
This commit modifies the internal workflow for opening or closing a
channel in order to create a path in which RPC clients can receive
updates. Updates are now communicated via channels from the goroutines
spawned by the RPC server to process the request, and the sub-system
within the daemon that actually executes the request.
With this change clients can now receive updates that the request is
pending (final message has been sent to the target client), or that the
request has been completed. Confirmation related updates have not yet
been implemented as that will require some changes to the ChainNotifier
interface.
This commit integrates BitFury's current routing functionality into lnd. The
primary ochestration point for the routing sub-system in the routingMgr. The
routingMgr manages all persistent and volatile state related to routing within
the network.
Newly opened channels, either when the initiator or responder are inserted into
the routing table once the channel is fully open. Once new links are inserted
the routingMgr can then perform path selection in order to locate an "optimal"
path to a target destination.
With this commit, support for changing the target network (testnet,
simnet, etc) has been finalized. Previously a command line option was
present to swap networks, but the RPC port wouldn’t automatically be
updated to reflect the network.
This commit implements the “pendingchannels” RPC within the rpcserver.
This RPC allows callers to receive details concerning the current
pending channels associated with the daemon. Pending channels are those
waiting for additional confirmations before they can be considered
opened/closed.
At the time of this commit, only open channels are shown. A future
commit will also add the confirmation updates, along with information
for close channels.
This commit switches the implementation of the open/close channel RPC’s
from a fully blocking synchronous model to one that’s async by default,
allowing callers to add a sync wrapper.
The new proto specs also allow for “updates” for the pending channels
in the form of new confirmations which progress the pending status of
the channel. At this point, only the final open/close updates have been
implemented. Obtaining confirmation notifications requires a bit of
re-working within the current ChainNotifier interface, thus this has
been deferred to a later time.
This commit implements the “send coins” RPC request which was
introduced at both the lnrpc and command line level in a prior commit.
A small refactoring has taken place w.r.t to sendmany+sendcoins in
order to eliminate some code duplication.
This commit fixes a bug introduced within a prior commit. The prior
commit failed to drollery reverse the txid string taken in as user
input, therefore in order to properly close a channel, the user needed
to manually reverse the txid themselves.
With this change, `wire.NewShaHashFromStr` is used which properly
reverses the string within the constructor. This allows the string
reported not be directly used to the close an active channel.
This commit also corrects a few logging messages.
This commit adds the necessary plumbing within the server, peer, and
rpcServer to handle opening and cooperatively closing a channel with a
remote peer.
Many new data structures have been added to the peer in order to allow
it to efficiently manage opening+.losing new/existing lightning
channels. Additional documentation has been added to several methods
within the peer struct, with some minor renaming along with way. The
peer has also gained a dedicated goroutine whose job it is to manage
any requests pertaining to opening, or closing any channels with the
remote peer.
The messages have been added to lnrpc define the requests and responses
to channel open+close messages. Additional channel logic has been added
between the rpcServer, peer, and server in order to properly manage the
necessary synchronization.
The new available address types are: regular p2pkh, p2wkh, and finally
p2wkh nested within p2sh. The latter address type will initially be
used to allow users to “fund” the lnd wallet.
