This commit adds each channel's short chan id to the `feereport` rpc.
Without this, it can be tedious to lookup more info about a particular
channel since most rpcs only accept short chan ids and not channel
points. For instance, now one can take a channel id from `feereport` and
look it up directly via `getchaninfo` to examine the policy in more
detail.
The synchronous call to get all channel backups also include
channels that are pending at the moment of the call. A previous
commit added pending channels to the file based backup as well. So
this is the last backup method that needs to be adjusted to also
contain unconfirmed channels.
This change instructs the REST proxy server to overwrite its default
JSON marshaler settings. That allows us to set EmitDefaults to true
which will result in all JSON fields returned in REST responses to
be fully populated, even if their values are falsey.
This commit adds PendingOpenChannel to SubscribeChannelEvents stream in
the gRPC API.
This is useful for keeping track of channel openings that Autopilot does.
It can also be used for the non-initator side of a channel opening to keep
track of channel openings.
Returning from the sendPayments method closes the underlying stream, so
we would risk payments still being in flight when returning.
We add the running goroutines to a waitgroup, such that we can wait for
them all to exit before exiting and closing the stream.
In case an error is encountered in the process, we will return directly,
which will close the `reqQuit` channel and prompt the gorpoutines to
shut down.
This commit fixes a problem that could arise when handling a continuous
stream of payIntents. We would risk the client sending a set of payment
intents and closing the stream, but we wouldn't be sure the sendLoop had
read all messages on the `payChan` before exiting with the nil error
from the `errChan`.
Instead, we close the `payChan` to indicate that no more payments are
going to be received.
This commit fixes a potential issue in the bidirectional sendPayment
case, where multiple goroutines could be sending on an errChan with
buffer 1. Instead we select on default as well, as it is enough to
handle the first error that was received.
Intead of checking for shutdown in the receive loop, we let the sendLoop
handle it, as it can return the error directly. This works since the
returning sendLoop will trigger a close of the `reqQuit` channel, which
will ensure the receive loop exits.
The default was increased for the main sendpayment RPC in commit
d3fa9767a9729756bab9b4a1121344b265410b1a. This commit sets the
same default for QueryRoutes, routerrpc.SendPayment and
router.EstimateRouteFee.
In this commit, we implement the currently defined transition methods
for the new `FundingStateStep` method. At this point, we're now able to
serve the "responder" of the externally initiated channel funding flow
by being able to register and cancel a funding flow according to its
expected pending channel ID.
In this commit, we update the `OpenChannel` method to observe the new
`funding_shim` field in the main open channel request. If this is
specified, and is a channel point shim, then we'll create a custom
`chanfunding.Assembler` for the wallet to use in place of the regular
funding workflow.
With this commit, the "initiator" of an external funding flow can now
delegate the remainder of the channel funding workflow to lnd.
In this commit, we start to expose some of the new external funding
functionality over the RPC interface.
First, we add a new `funding_shim` field to the regular `OpenChannel`
method. This can be used by a caller to express that certain parameters
of the funding flow have already been negotiated outside the protocol,
and should be used instead. For example, a shim can be provided to use a
particular key for the commitment key (ideally cold) rather than use one
this is generated by the wallet as normal, or signal that signing will
be carried out in an interactive manner (PSBT based).
Next, we add a brand new method: `FundingStateStep`. FundingStateStep is
an advanced funding related call that allows the caller to either
execute some preparatory steps for a funding workflow, or manually
progress a funding workflow. The primary way a funding flow is
identified is via its pending channel ID. As an example, this method can
be used to specify that we're expecting a funding flow for a particular
pending channel ID, for which we need to use specific parameters.
Alternatively, this can be used to interactively drive PSBT signing for
funding for partially complete funding transactions.
The new transition methods (funding state machine modifiers) in this
commit allow a party to register a funding intent that should be used
for a specified incoming pending channel ID. The "responder" to the
external channel flow should use this to prep lnd to be able to handle
the channel flow properly.
Earlier we used emtpy grpc server options if custom listeners were set.
This was done to disable TLS. Now, we reuse the same server options as
for the regular listeners, in a move towards enabling TLS also here.
In this commit, the channelEventStore in the channel
fitness subsystem is changed to identify channels
by their outpoint rather than short channel id. This
change is made made becuase outpoints are the preferred
way to expose references over rpc, and easier to perform
queries within lnd.
Upgrade logging of channel uptime/lifespan query errors
to returning the error, with the exception of the case
where the channel is known to the store. This error is
logged, due to the potential for race conditions between
the channel event store being notified of a new open
channel and the rpc channel subscription creating a rpc
channel struct (which queries for uptime).
In the process, we also move the feature serialization into the
invoicesrpc package, so that it can be shared between the invoicesrpc
and main rpcserver.
This commit prepares for more manipulation of custom records. A list of
tlv.Record types is more difficult to use than the more basic
map[uint64][]byte.
Furthermore fields and variables are renamed to make them more
consistent.
This commit is adapted from @Bluetegu's original
pull request #1462.
This commit reads an optional address to pay funds out to
from a user iniitiated close channel address. If the channel
already has a shutdown script set, the request will fail if
an address is provided. Otherwise, the cooperative close will
pay out to the address provided.
In this commit, we add `msats` to the return value of `DecodePayReq` to
ensure we always show full value information as we're moving to do
generally for all RPC calls that deal with off-chain amounts.
In this commit, we update the `AbandonChannel` method to also remove the
state from the countract court as well as the channel graph. Abandoning
a channel is now a three step process: remove from the open channel
state, remove from the graph, remove from the contract court. Between
any step it's possible that the users restarts the process all over
again. As a result, each of the steps below are intended to be
idempotent.
We also update the integration test to assert that no channel is found
in the graph any longer. Before this commit, this test would fail as the
channel was still found in the graph, which can cause other issues for
an operational daemon.
Fixes#3716.
The channel arbitrator no longer passes the direct commitment output to
the nursery for incubation. Instead the resolver itself will await the
csv lock if any.
The reason to change this now is to prevent having to deal with the
(legacy) nursery code for a planned anchor outputs related change to the
commit sweep resolver (also csv lock to_remote).
It is no problem if there are any lingering incubating outputs at the
time of upgrade. This just means that the output will be offered twice
to the sweeper and this doesn't hurt.
In this commit, we create a new chainfee package, that houses all fee
related functionality used within the codebase. The creation of this new
package furthers our long-term goal of extracting functionality from the
bloated `lnwallet` package into new distinct packages. Additionally,
this new packages resolves a class of import cycle that could arise if a
new package that was imported by something in `lnwallet` wanted to use
the existing fee related functions in the prior `lnwallet` package.
This commit adds the total observed lifetime of a channel and the
totaluptime of its remote peer to the lnrpc channel struct. These
fields are marked as experimential because they are subject to
change.
With the introduction of the max CLTV limit parameter, nodes are able to
reject HTLCs that exceed it. This should also be applied to path
finding, otherwise HTLCs crafted by the same node that exceed it never
left the switch. This wasn't a big deal since the previous max CLTV
limit was ~5000 blocks. Once it was lowered to 1008, the issue became
more apparent. Therefore, all of our path finding attempts now have a
restriction of said limit in in order to properly carry out HTLCs to the
network.
Fixes https://github.com/lightningnetwork/lnd/issues/3357. When
start_time isn't specified, its default value is 0. This meant when
users explicitly specified a start_time of 0, we would incorrectly set
start_time to 24 hours in the past. Now, n0 means n0.
The policy update logic that resided part in the gossiper and
part in the rpc server is extracted into its own object.
This prepares for additional validation logic to be added for policy
updates that would otherwise make the gossiper heavier.
It is also a small first step towards separation of our own channel data
from the rest of the graph.
In this commit, we fix a bug where if a user updates a forwarding policy to be
zero, the update will be applied to the policy correctly on-disk, but not
in-memory.
We solve this issue by having the gossiper return the list of on-disk updated
policies and passing these policies to the switch, so the switch can assume
that zero-valued fields are intentional and not just uninitialized.
In this commit, we add a new field to the Hop proto to allow callers to
be able to specify TLV records for the SendToRoute call, and also to be
able to display TLV records that were used during regular path finding.
We also update SendPayment to support dest TLV records.
Debug invoices are rarely used nowadays, but keep asking for maintenance
every time refactoring in primarily the invoice registry occurs. We have
passed the cost/benefit tipping point, so therefore the debug invoice
concept is removed in this commit.
Previously the debughtlc flag also controlled whether hodl masks were
active. It is safe to remove that additional condition because the hodl
masks are still guarded by the dev build tag.
Not all errors that occur when serving client requests in the gRPC
server are logged. As a result, at times, we can be lacking critic
information that can be used to debug issues that pop up. With this PR,
we create a basic streaming+unary interceptor that will log all errors
that occur when servicing calls.
The current format looks something like this in the logs when an error
occurs:
```
[ERR] RPCS: [/lnrpc.Lightning/SendCoins]: decoded address is of unknown format
```
lnrpc/rpc.proto: updates RoutingPolicy last_update number
rpcserver: adding LastUpdate field to marshalDbEdge
This commit adds a LastUpdate field to each RoutingPolicy, it will show
the time each RoutingPolicy was updated.
lnrpc: regenerating proto files
In this commit two fields were added to the Channl RPC result in both
open and pending states.
The fields: local_chan_reserve, remote_chan_reserve represents the
reservation the nodes are rquired to keep in both sides of the channel.
This is usefull when calculating the "real" inbound and outbound
liquidity in an accurate way.
This commit fixes a but where restarting LND at the same process causes
It to fail.
The problem resides in the fact that an array of permissions is
initialized as a package variable and when creating the
RPCServer all subserver permissions are appended while checking for
duplicates.
On subsequent restart this array is left over from the previous run and
being populated again with the same permissions causing a duplicate
error.
The solution is simple, just to extract out the initial permissions to
a separate function and call it from the context is is needed.
Earlier versions of ListPayments only included completed payments. We
return to this behavior by ignore all other payments if the nonSucceeded
boolean is not set in the request.
Start the Prometheus exporter in rpcserver.go if monitoring is enabled through the
build tag. Also allow users to specify what address they want the Prometheus
exporter to be listening on.
This commit moves the default timeout out of router and thereby fixes a
bug that caused SendToRoute to not return the actual error, but a
timeout result instead. SendToRoute only tries a single route, so a
timeout should never happen.
This commit makes the router use the ControlTower to drive the payment
life cycle state machine, to keep track of active payments across
restarts. This lets the router resume payments on startup, such that
their final results can be handled and stored when ready.
Now that the success resolver preimage field is always populated by the
incoming contest resolver, preimage lookups earlier in the
process (channel and channel arbitrator) can mostly be removed.
This commit restructures the creation of various tls related object. It
also fixes a bug where wildcard IP addresses where only instantiated for
the main RPC server and not the WalletUnlocker service.
In this commit, we modify the `closeObserver` to fast path the DLP
dispatch case if we detect that the channel has been restored. We do
this as otherwise, we may inadvertently enter one of the other cases
erroneously, causing us to now properly look up their dlp commitment
point.
lnd_test: adding address validation for send coins
The commit adds a test that checks that when a user calls sendcoins, the
receiving address is validated according to the current network. If the
address is not compatible with the current network, it will return an
error to the user.
rpcserver: adding a check for compatible network in SendCoins
This commit adds a check in SendCoins that checks whether the receiving
address is compatible with the current network.
Fixes#2677.
This is a better alternative than retrieving it from the graph as it's
possible that the channel is pruned from it doing to not having any
updates within the past two weeks.
This commit moves the query routes backend logic from the main
rpc server into the sub server. It is another step towards splitting up
the main rpc server code.
In addition to this, a unit test is added to verify rpc parameter
parsing.
Currently public keys are represented either as a 33-byte array (Vertex) or as a
btcec.PublicKey struct. The latter isn't useable as index into maps and
cannot be used easily in compares. Therefore the 33-byte array
representation is used predominantly throughout the code base.
This commit converts the argument types of source and target nodes for
path finding to Vertex. Path finding executes no crypto operations and
using Vertex simplifies the code.
Additionally, it prepares for the path finding source parameter to be
exposed over rpc in a follow up commit without requiring conversion back
and forth between Vertex and btcec.PublicKey.
This commit allows the execution of QueryRoutes to be controlled using
lists of black-listed edges and nodes. Any path returned will not pass
through the edges and/or nodes on the list.
In this commit, we rename the new `MaxHtlc` field to `MaxHtlcMsat`. We
do this in order to adhere to the new practice to suffix each amount
related field with their proper unit.
A recent change in this area of the codebase refactored some code to
make way for the deprecation of the streaming multi-route `SendToRoute`
method. Along the way, it removed the parsing of the `PaymentHashString`
field for the sync `SendToRouteSync` which is meant to be used via the
REST interface. In this commit, we fix this by populating _both_
`PaymentHash` and `PaymentHashString`. This is OK, as
`extractPaymentIntent` will check both values to extract the hex-encoded
payment hash if it's specified.
`createRPCChannel` is used by the `listchannels` RPC call and will be
used by `subscribechannels` as well.
`createRPCClosedChannel` is used by the `closedchannels` RPC call and
will also be used by `subscribechannels`.
This is used by the `listchannels` RPC call and will be used by
`subscribechannels` as well. Its purpose is to mitigate code duplication
between the two RPC calls.
This commit is a step to split the lnwallet package. It puts the Input
interface and implementations in a separate package along with all their
dependencies from lnwallet.
In this commit, we modify areas where we need to force close a channel
to use the new FetchChannel method instead of manually scanning. This
dramatically reduces the CPU usage when doing things like closing a
large number of channels within lnd.
In this commit:
* we partition lnwire.ChanUpdateFlag into two (ChanUpdateChanFlags and
ChanUpdateMsgFlags), from a uint16 to a pair of uint8's
* we rename the ChannelUpdate.Flags to ChannelFlags and add an
additional MessageFlags field, which will be used to indicate the
presence of the optional field HtlcMaximumMsat within the ChannelUpdate.
* we partition ChannelEdgePolicy.Flags into message and channel flags.
This change corresponds to the partitioning of the ChannelUpdate's Flags
field into MessageFlags and ChannelFlags.
Co-authored-by: Johan T. Halseth <johanth@gmail.com>
In this commit, we implement the new feature which allows sendcoins to
sweep all the coins in the wallet. We use the new sweep.CraftSweepAllTx
method, and also use WithCoinSelectLock to ensure that we don't trigger
any double-spend errors triggered by coin selection race conditions.
In this commit, we add a new method WithCoinSelectLock. This method will
allow us to fix bugs in the project atm that can arise if a channel
funding is attempted (either manually or by autopilot) while a users is
attempting to send an on-chain transaction. If this happens
concurrently, then both contexts will grab the set of UTXOs and attempt
to lock them one by one. However, since they didn't obtain an exclusive
snapshot of the UTXO set of the wallet, they may both attempt to lock
the same input.
We also ensure that calls to SendMany cannot run into this issue by
using the WithCoinSelectLock synchronization when attempting to instruct
the internal wallet to send payments.
In this commit, we extract the existing determineFeePerKw method on the
RPC server into a new file in the sweep package. Along the way, we
consolidate code by introducing a new FeePreference struct, which allows
the caller to express their fee preference either in blocks to
confirmation, or a direct fee rate. This move takes a small step to
father decoupling calls in the main RPC server.
This commit is a preparation for the addition of new invoice
states. A database migration is not needed because we keep
the same field length and values.
In this commit, we remove the per channel `sigPool` within the
`lnwallet.LightningChannel` struct. With this change, we ensure that as
the number of channels grows, the number of gouroutines idling in the
sigPool stays constant. It's the case that currently on the daemon, most
channels are likely inactive, with only a hand full actually
consistently carrying out channel updates. As a result, this change
should reduce the amount of idle CPU usage, as we have less active
goroutines in select loops.
In order to make this change, the `SigPool` itself has been publicly
exported such that outside callers can make a `SigPool` and pass it into
newly created channels. Since the sig pool now lives outside the
channel, we were also able to do away with the Stop() method on the
channel all together.
Finally, the server is the sub-system that is currently responsible for
managing the `SigPool` within lnd.
Returns a brief json summary of each utxo found by calling
ListUnspentWitness in the wallet. The two arguments are the
minimum and maximum number of conrfirmations (0=include
unconfirmed)
In this commit, we extend the admin macaroon with signer capabilities in
order to allow it to be used with the new signer sub-server. As a
result, users will need to clear out their old macaroons in order to
have the new improved admin macaroon generated. In the future, we may
want to restructure the way the admin macaroon functions: rather than
white listing each of these entities and operations, we can instead add
a catch all capability. This capability will instead allow access to any
call, as each new call would be modified to permit this capabilities and
no others.
In this commit, we modify the existing rpcServer to fully manage the
macaroons, gRPC server, and also seek out and create all sub-servers.
With this change, the RPC server gains more responsibility, as it
becomes the "root" server in the hierarchy of gRPC sub-servers.
In addition to creating each sub-server, it will also merge the set of
macaroon permissions for each sub-server, with the permissions of the
rest of the RPC infra. As a result, each sub-server is able to
independently specify what it needs w.r.t macaroon permissions and have
that taken care of by the RPC server. In order to achieve this, we need
to unify the creation of the RPC interceptors, and also fully manage the
gRPC server ourselves.
Some examples with various build configs:
```
⛰i make build
Building debug lnd and lncli.
go build -v -tags="dev" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd
go build -v -tags="dev" -o lncli-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd/cmd/lncli
⛰i ./lnd-debug --debuglevel=debug --signrpc.signermacaroonpath=~/sign.macaroon
unknown flag `signrpc.signermacaroonpath'
unknown flag `signrpc.signermacaroonpath'
⛰i make build tags=signerrpc
Building debug lnd and lncli.
go build -v -tags="dev signerrpc" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd
go build -v -tags="dev signerrpc" -o lncli-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd/cmd/lncli
⛰i ./lnd-debug --debuglevel=debug --signrpc.signermacaroonpath=~/sign.macaroon
2018-10-22 17:31:01.132 [INF] LTND: Version: 0.5.0-beta commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty, build=development, logging=default
2018-10-22 17:31:01.133 [INF] LTND: Active chain: Bitcoin (network=simnet)
2018-10-22 17:31:01.140 [INF] CHDB: Checking for schema update: latest_version=6, db_version=6
2018-10-22 17:31:01.236 [INF] LTND: Primary chain is set to: bitcoin
2018-10-22 17:31:02.391 [INF] LNWL: Opened wallet
2018-10-22 17:31:03.315 [INF] LNWL: The wallet has been unlocked without a time limit
2018-10-22 17:31:03.315 [INF] LTND: LightningWallet opened
2018-10-22 17:31:03.319 [INF] LNWL: Catching up block hashes to height 3060, this will take a while...
2018-10-22 17:31:03.320 [INF] HSWC: Restoring in-memory circuit state from disk
2018-10-22 17:31:03.320 [INF] LNWL: Done catching up block hashes
2018-10-22 17:31:03.320 [INF] HSWC: Payment circuits loaded: num_pending=0, num_open=0
2018-10-22 17:31:03.322 [DBG] LTND: Populating dependencies for sub RPC server: Signrpc
```
As for the config, an example is:
```
[signrpc]
signrpc.signermacaroonpath=~/signer.macaroon
```
