This commits builds on top of PR #3694 to further clarify invoice
state by defining pending invoices as the ones which are not
settled or canceled. Automatic cancellation of expired invoices
makes this possbile. While this change only directly affects
ChannelDB, users of the listinvoices RPC will receive actual
pending invoices when pending_only flag is set.
In the process, we also move the feature serialization into the
invoicesrpc package, so that it can be shared between the invoicesrpc
and main rpcserver.
This commit prepares for more manipulation of custom records. A list of
tlv.Record types is more difficult to use than the more basic
map[uint64][]byte.
Furthermore fields and variables are renamed to make them more
consistent.
To allow signing of messages with any key in the key chain
we add two new methods to the signer RPC. These behave differently
to the methods with the same name in the main RPC as described
in the documentation comment.
In this commit, we add `msats` to the return value of `DecodePayReq` to
ensure we always show full value information as we're moving to do
generally for all RPC calls that deal with off-chain amounts.
This commit modifies Lighting.AddInvoice and InvoicesRPC.AddHoldInvoice
to include the node's supported feature bits on the invoice. For now
this only includes the optional TLV Onion Payload bit.
In this commitment, we make the `SendToRoute` RPC call consistent with
all the other payment RPCs which will properly adhere to the current max
payment sat limit. This is a prep commit for the future wumbo soft cap
that will eventually land in lnd.
This commit restructures an invoice's ContractTerms to better encompass
the restrictions placed on settling. For instance, the final ctlv delta
and invoice expiry are moved from the main invoice body (where
additional metadata is stored). Additionally, it moves the State field
outside of the terms since it is rather metadata about the invoice
instead of any terms offered to the sender in the payment request.
This commit modifies the FetchPayment method to return MPPayment structs
converted from the legacy on-disk format. This allows us to attach the
HTLCs to the events given to clients subscribing to the outcome of an
HTLC.
This commit also bubbles up to the routerrpc/router_server, by
populating HTLCAttempts in the response and extracting the legacy route
field from the HTLCAttempts.
With this PR we deprecate fields that have been specifically added to
to work around a bug in the gRPC/REST gateway that didn't allow bytes
fields to be encoded in REST requests.
That bug has now been fixed so the fields are no longer required.
To make it more clear how bytes fields have to be used in REST,
comments have been added to all those fields.
This commit parses mpp_total_amt_msat and mpp_payment_addr from the
SendToRoute rpc and populates an MPP record on the internal hop
reprsentation. When the router goes to encode the onion packet, these
fields will be serialized for the destination. We also populate the mpp
fields when marshalling routes in rpc responses.
This commit add mpp_total_amt_msat and mpp_payment_addr to the Hop
message. Doing so enables users submitting mpp payments via rpc to
set these parameters for the destination. In addition, it will allow us
to display these fields in rpc responses.
In this commit, we create a new chainfee package, that houses all fee
related functionality used within the codebase. The creation of this new
package furthers our long-term goal of extracting functionality from the
bloated `lnwallet` package into new distinct packages. Additionally,
this new packages resolves a class of import cycle that could arise if a
new package that was imported by something in `lnwallet` wanted to use
the existing fee related functions in the prior `lnwallet` package.
Probabilities are no longer returned for querymc calls. To still provide
some insight into the mission control internals, this commit adds a new
rpc that calculates a success probability estimate for a specific node
pair and amount.
With a separate proto message, it becomes possible to also return the
pair data for a single pair. This prepares for the new mc probability
querying rpc.
This commit adds the total observed lifetime of a channel and the
totaluptime of its remote peer to the lnrpc channel struct. These
fields are marked as experimential because they are subject to
change.
This commit changes mission control to partially base the estimated
probability for untried connections on historical results obtained in
previous payment attempts. This incentivizes routing nodes to keep all
of their channels in good shape.
Probability estimates are amount dependent. Previously we assumed an
amount, but that starts to make less sense when we make probability more
dependent on amounts in the future.
This sets the `jstype` option to `JS_STRING` for all `chan_id` fields
in the proto rpc definition. `chan_id` is a 64 bit integer, which is
not natively supported by javascript's floating-point `number` with
only 52 bit precision. Nevertheless, by default protobuf will use the
`number` type for 64 bit integer fields in javascript, which can cause
loss of precision problems with `chan_id`. Explicitly setting the type
for javascript as a string will prevent these issues, and should not
interfere with its use as an identifier.
With the introduction of the max CLTV limit parameter, nodes are able to
reject HTLCs that exceed it. This should also be applied to path
finding, otherwise HTLCs crafted by the same node that exceed it never
left the switch. This wasn't a big deal since the previous max CLTV
limit was ~5000 blocks. Once it was lowered to 1008, the issue became
more apparent. Therefore, all of our path finding attempts now have a
restriction of said limit in in order to properly carry out HTLCs to the
network.
In order to prevent future unforeseen issues, we are temporarily
disabling the ability to send custom tlv records to the receiver of a
payment. Currently the receiver does not process or expose these
additional fields via rpc or internally, so they are being disabled
until the end-to-end flow is finished and fully validated.
Extends the invalid payment details failure with the new accept height
field. This allows sender to distinguish between a genuine invalid
details situation and a delay caused by intermediate nodes.
This commit adds a set of htlcs to the Invoice struct and
serializes/deserializes this set to/from disk. It is a preparation for
accurate invoice accounting across restarts of lnd.
A migration is added for the invoice htlcs.
In addition to these changes, separate final cltv delta and expiry
invoice fields are created and populated. Previously it was required
to decode this from the stored payment request. The reason to create
a combined commit is to prevent multiple migrations.
This commit modifies paymentLifecycle so that it not only feeds
failures into mission control, but successes as well.
This allows for more accurate probability estimates. Previously,
the success probability for a successful pair and a pair with
no history was equal. There was no force that pushed towards
previously successful routes.
In this commit, we add a new field to the Hop proto to allow callers to
be able to specify TLV records for the SendToRoute call, and also to be
able to display TLV records that were used during regular path finding.
We also update SendPayment to support dest TLV records.
In this commit, we address an issue that would cause us to scan from the
genesis block for the spend of an output that we wish to use to raise
the fee of a transaction through CPFP. This was due to setting a 0
height hint when constructing the input required by the sweeper and was
discovered due to the recently added validation checks at the chain
notifier level. We'll now use the current height as the height hint
instead as the sweeper will end up creating a new transaction that
spends the input.
Previously mission control tracked failures on a per node, per channel basis.
This commit changes this to tracking on the level of directed node pairs. The goal
of moving to this coarser-grained level is to reduce the number of required
payment attempts without compromising payment reliability.
Align naming better with the lightning spec. Not the full name of the
failure (FailIncorrectOrUnknownPaymentDetails) is used, because this
would cause too many long lines in the code.
lnrpc/rpc.proto: updates RoutingPolicy last_update number
rpcserver: adding LastUpdate field to marshalDbEdge
This commit adds a LastUpdate field to each RoutingPolicy, it will show
the time each RoutingPolicy was updated.
lnrpc: regenerating proto files
In this commit two fields were added to the Channl RPC result in both
open and pending states.
The fields: local_chan_reserve, remote_chan_reserve represents the
reservation the nodes are rquired to keep in both sides of the channel.
This is usefull when calculating the "real" inbound and outbound
liquidity in an accurate way.
In this commit, we modify the parsing of user provided outpoints to
ensure that we catch a nil (empty) output early. Otherwise, passing a
set of incorrect arguments would cause `lnd` to crash.
This RPC exposes the recently added BumpFee functionality to the
UtxoSweeper in order to allow users of the RPC to manually bump fees of
low fee inputs/transactions.
This commit exposes the three main parameters that influence mission
control and path finding to the user as command line or config file
flags. It allows for fine-tuning for optimal results.
Previously every payment had its own local mission control state which
was in effect only for that payment. In this commit most of the local
state is removed and payments all tap into the global mission control
probability estimator.
Furthermore the decay time of pruned edges and nodes is extended, so
that observations about the network can better benefit future payment
processes.
Last, the probability function is transformed from a binary output to a
gradual curve, allowing for a better trade off between candidate routes.
This PR replaces the previously used edge and node ignore lists in path
finding by a probability based system. It modifies path finding so that
it not only compares routes on fee and time lock, but also takes route
success probability into account.
Allowing routes to be compared based on success probability is achieved
by introducing a 'virtual' cost of a payment attempt and using that to
translate probability into another cost factor.
This commit upgrades the protobuf version. Compared to the previous
v1.2.0 it generates smaller diffs in generated code. This change was
introduced in:
fffb0f7828
This fixes a bug where the RHashStr field on the PaymentHash request for
SubscribeSingleInvoice was being ignored. This replaces the PaymentHash
message for the SubscribeSingleInvoice request with a new message called
SubscribeSingleInvoiceRequest that contains only the RHash field.
This commit modifies the invoice registry to handle invoices for which
the preimage is not known yet (hodl invoices). In that case, the
resolution channel passed in from links and resolvers is stored until we
either learn the preimage or want to cancel the htlc.
This commit moves the query routes backend logic from the main
rpc server into the sub server. It is another step towards splitting up
the main rpc server code.
In addition to this, a unit test is added to verify rpc parameter
parsing.
Now that QueryRoutes gained the ability to route from any source node
and takes in edge and node black lists, all pieces are in place to have
users implemented their own k-shortests path algorithm. Or any other algorithm
they might wish to use and currently can't.
This commit marks the num_routes field as deprecated as a preparation
for removing k-shortest for lnd.
This commit allows the execution of QueryRoutes to be controlled using
lists of black-listed edges and nodes. Any path returned will not pass
through the edges and/or nodes on the list.
In this commit, we add two new address types to `NewAddressRequest`.
These address types will allow the caller to retrieve the last _unused_
address, rather than always rotating to the next address in the
keychain.
In this commit, we rename the new `MaxHtlc` field to `MaxHtlcMsat`. We
do this in order to adhere to the new practice to suffix each amount
related field with their proper unit.
In this commit, we implement the full RouterServer as specified by the
newly added sub-service as defined in router.proto. This new sub-server
has its own macaroon state (but overlapping permissions which can be
combined with the current admin.macaroon), and gives users a simplified
interface for a gRPC service that is able to simply send payment. Much
of the error reporting atm, is a place holder, and a follow up commit
will put up the infrastructure for a proper set of errors.
This commit is a step to split the lnwallet package. It puts the Input
interface and implementations in a separate package along with all their
dependencies from lnwallet.
Update is necessary to get protoc to generate files in the right
directory when using modules outside the gopath. The old version of this
dependency did not support paths=source_relative.
This commit also fixes a build error in lnd_test.go because of a
slight difference in generated code.
This commit adds a new service lnrpc/autopilot, that is to be used to
communicate with the running autopilot. Currently a RPC for getting the
status of the agent is included, as well as enabling/disabling at
runtime.
Returns a brief json summary of each utxo found by calling
ListUnspentWitness in the wallet. The two arguments are the
minimum and maximum number of conrfirmations (0=include
unconfirmed)
In this commit, we implement the newly defiend WalletKitServer gRPC
service. We use the same template w.r.t build tags as the existing
signrpc service.
In this commit, we add a new sub-RPC server to the existing set of gRPC
servers. This new sub-RPC server is the WalletKit. It's a utility
toolkit that contains method which allow clients to perform common
interactions with a wallet such as getting a new address, or sending a
transaction. It also includes some supplementary actions such as fee
estimation.
One thing to note in the RPC file is that we _import_ the existing
signer.proto file in order to get at some existing proto definitions
which are useful in our use case.
In this commit, we add the ComputeInputScript which will allow callers
to obtain witnesses for all outputs under control of the wallet. This
allows external scripting of things like coin join, etc.
In this commit, we add the glue infrastructure to make the sub RPC
server system work properly. Our high level goal is the following: using
only the lnrpc package (with no visibility into the sub RPC servers),
the RPC server is able to find, create, run, and manage the entire set
of present and future sub RPC servers. In order to achieve this, we use
the reflect package and build tags heavily to permit a loosely coupled
configuration parsing system for the sub RPC servers.
We start with a new `subRpcServerConfigs` struct which is _always_
present. This struct has its own group, and will house a series of
sub-configs, one for each sub RPC server. Each sub-config is actually
gated behind a build flag, and can be used to allow users on the command
line or in the config to specify arguments related to the sub-server. If
the config isn't present, then we don't attempt to parse it at all, if
it is, then that means the RPC server has been registered, and we should
parse the contents of its config.
The `subRpcServerConfigs` struct has two main methods:
`PopulateDependancies` and `FetchConfig`. The `PopulateDependancies` is
used to dynamically locate and set the config fields for each new
sub-server. As the config may not actually have any fields (if the build
flag is off), we use the reflect pacakge to determine if things are
compiled in or not, then if so, we dynamically set each of the config
parameters. The `PopulateDependancies` method implements the
`lnrpc.SubServerConfigDispatcher` interface. Our goal is to allow sub
servers to look up their actual config in this main config struct. We
achieve this by using reflect to look up the target field _as if it were
a key in a map_. If the field is found, then we check if it has any
actual attributes (it won't if the build flag is off), if it is, then we
return it as we expect it to be populated already.
In this commit, we create a lnrpc.SubServerDriver for signrpc. Note that
this file will only have its init() method executed if the proper build
flag is on. As a result, only if the build flag is set, will the RPC
server be registered, and visible at the packge lnrpc level for the root
server to manipulate.
In this commit, we add a full implementation of the new SignerServer sub
RPC service within the main root RPC service. This service is able to
fully manage its macaroons, and service any connected clients. Atm, this
service only has a single method: SignOutputRaw which mimics the
existing lnwallet.Signer interface within lnd itself. As the API's are
so similar, it will be possible for a client to directly use the
lnwallet.Signer interface, and have a proxy that sends the request over
RPC, and translates the proto layer on both sides. To the client, it
doesn't know that it's using a remote, or local RPC.
In this commit, we add the scafolding for the future sub-server RPC
system. The idea is that each sub server will implement this particular
interface. From there on, a "root" RPC server is able to query this
registry, and dynamically create each sub-sever instance without
knowing the details of each sub-server.
In the init() method of the pacakge of a sub-server, the sub-server is
to call: RegisterSubServer to claim its namespace. Afterwards, the root
RPC server can use the RegisteredSubServers() method to obtain a slice
of ALL regsitered sub-servers. Once this list is obtained, it can use
the New() method of the SubServerDriver struct to create a new
sub-server instance.
Each sub-server needs to be able to locate it's primary config using the
SubServerConfigDispatcher interface. This can be a map of maps, or a
regular config structr. The main requirement is that the sub-server be
able to find a config under the same name that it registered with. This
string of abstractions will allow the main RPC server to find, create,
and run each sub-server without knowing the details of its configuration
or its role.
In this commit, we add a new proto generation script to match the one in
the main lnrpc package. This script differs, as we don't need to
generate the REST proxy stuff (for now).
In this commit, we introduce a new sub-package within the greater RPC
package. This new sub-package will house a new set of sub-RPC servers
to expose experimental features behind build flags for upstream
consumers. In this commit, we add the first config for the service,
which will simply expose the lnwallet.Signer interface over RPC.
In the default file, we have what the config will be if the build tag
(signerrpc) is off. In this case, the config parser won't detect any
times, and if specified will error out. In the active file, we have the
true config that the server will use. With this new set up, we'll
exploit these build flags heavily in order to create a generalized
framework for adding additional sub RPC servers.
