This commit moves the query routes backend logic from the main
rpc server into the sub server. It is another step towards splitting up
the main rpc server code.
In addition to this, a unit test is added to verify rpc parameter
parsing.
Currently public keys are represented either as a 33-byte array (Vertex) or as a
btcec.PublicKey struct. The latter isn't useable as index into maps and
cannot be used easily in compares. Therefore the 33-byte array
representation is used predominantly throughout the code base.
This commit converts the argument types of source and target nodes for
path finding to Vertex. Path finding executes no crypto operations and
using Vertex simplifies the code.
Additionally, it prepares for the path finding source parameter to be
exposed over rpc in a follow up commit without requiring conversion back
and forth between Vertex and btcec.PublicKey.
This commit allows the execution of QueryRoutes to be controlled using
lists of black-listed edges and nodes. Any path returned will not pass
through the edges and/or nodes on the list.
In this commit, we rename the new `MaxHtlc` field to `MaxHtlcMsat`. We
do this in order to adhere to the new practice to suffix each amount
related field with their proper unit.
A recent change in this area of the codebase refactored some code to
make way for the deprecation of the streaming multi-route `SendToRoute`
method. Along the way, it removed the parsing of the `PaymentHashString`
field for the sync `SendToRouteSync` which is meant to be used via the
REST interface. In this commit, we fix this by populating _both_
`PaymentHash` and `PaymentHashString`. This is OK, as
`extractPaymentIntent` will check both values to extract the hex-encoded
payment hash if it's specified.
`createRPCChannel` is used by the `listchannels` RPC call and will be
used by `subscribechannels` as well.
`createRPCClosedChannel` is used by the `closedchannels` RPC call and
will also be used by `subscribechannels`.
This is used by the `listchannels` RPC call and will be used by
`subscribechannels` as well. Its purpose is to mitigate code duplication
between the two RPC calls.
This commit is a step to split the lnwallet package. It puts the Input
interface and implementations in a separate package along with all their
dependencies from lnwallet.
In this commit, we modify areas where we need to force close a channel
to use the new FetchChannel method instead of manually scanning. This
dramatically reduces the CPU usage when doing things like closing a
large number of channels within lnd.
In this commit:
* we partition lnwire.ChanUpdateFlag into two (ChanUpdateChanFlags and
ChanUpdateMsgFlags), from a uint16 to a pair of uint8's
* we rename the ChannelUpdate.Flags to ChannelFlags and add an
additional MessageFlags field, which will be used to indicate the
presence of the optional field HtlcMaximumMsat within the ChannelUpdate.
* we partition ChannelEdgePolicy.Flags into message and channel flags.
This change corresponds to the partitioning of the ChannelUpdate's Flags
field into MessageFlags and ChannelFlags.
Co-authored-by: Johan T. Halseth <johanth@gmail.com>
In this commit, we implement the new feature which allows sendcoins to
sweep all the coins in the wallet. We use the new sweep.CraftSweepAllTx
method, and also use WithCoinSelectLock to ensure that we don't trigger
any double-spend errors triggered by coin selection race conditions.
In this commit, we add a new method WithCoinSelectLock. This method will
allow us to fix bugs in the project atm that can arise if a channel
funding is attempted (either manually or by autopilot) while a users is
attempting to send an on-chain transaction. If this happens
concurrently, then both contexts will grab the set of UTXOs and attempt
to lock them one by one. However, since they didn't obtain an exclusive
snapshot of the UTXO set of the wallet, they may both attempt to lock
the same input.
We also ensure that calls to SendMany cannot run into this issue by
using the WithCoinSelectLock synchronization when attempting to instruct
the internal wallet to send payments.
In this commit, we extract the existing determineFeePerKw method on the
RPC server into a new file in the sweep package. Along the way, we
consolidate code by introducing a new FeePreference struct, which allows
the caller to express their fee preference either in blocks to
confirmation, or a direct fee rate. This move takes a small step to
father decoupling calls in the main RPC server.
This commit is a preparation for the addition of new invoice
states. A database migration is not needed because we keep
the same field length and values.
In this commit, we remove the per channel `sigPool` within the
`lnwallet.LightningChannel` struct. With this change, we ensure that as
the number of channels grows, the number of gouroutines idling in the
sigPool stays constant. It's the case that currently on the daemon, most
channels are likely inactive, with only a hand full actually
consistently carrying out channel updates. As a result, this change
should reduce the amount of idle CPU usage, as we have less active
goroutines in select loops.
In order to make this change, the `SigPool` itself has been publicly
exported such that outside callers can make a `SigPool` and pass it into
newly created channels. Since the sig pool now lives outside the
channel, we were also able to do away with the Stop() method on the
channel all together.
Finally, the server is the sub-system that is currently responsible for
managing the `SigPool` within lnd.
Returns a brief json summary of each utxo found by calling
ListUnspentWitness in the wallet. The two arguments are the
minimum and maximum number of conrfirmations (0=include
unconfirmed)
In this commit, we extend the admin macaroon with signer capabilities in
order to allow it to be used with the new signer sub-server. As a
result, users will need to clear out their old macaroons in order to
have the new improved admin macaroon generated. In the future, we may
want to restructure the way the admin macaroon functions: rather than
white listing each of these entities and operations, we can instead add
a catch all capability. This capability will instead allow access to any
call, as each new call would be modified to permit this capabilities and
no others.
In this commit, we modify the existing rpcServer to fully manage the
macaroons, gRPC server, and also seek out and create all sub-servers.
With this change, the RPC server gains more responsibility, as it
becomes the "root" server in the hierarchy of gRPC sub-servers.
In addition to creating each sub-server, it will also merge the set of
macaroon permissions for each sub-server, with the permissions of the
rest of the RPC infra. As a result, each sub-server is able to
independently specify what it needs w.r.t macaroon permissions and have
that taken care of by the RPC server. In order to achieve this, we need
to unify the creation of the RPC interceptors, and also fully manage the
gRPC server ourselves.
Some examples with various build configs:
```
⛰i make build
Building debug lnd and lncli.
go build -v -tags="dev" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd
go build -v -tags="dev" -o lncli-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd/cmd/lncli
⛰i ./lnd-debug --debuglevel=debug --signrpc.signermacaroonpath=~/sign.macaroon
unknown flag `signrpc.signermacaroonpath'
unknown flag `signrpc.signermacaroonpath'
⛰i make build tags=signerrpc
Building debug lnd and lncli.
go build -v -tags="dev signerrpc" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd
go build -v -tags="dev signerrpc" -o lncli-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd/cmd/lncli
⛰i ./lnd-debug --debuglevel=debug --signrpc.signermacaroonpath=~/sign.macaroon
2018-10-22 17:31:01.132 [INF] LTND: Version: 0.5.0-beta commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty, build=development, logging=default
2018-10-22 17:31:01.133 [INF] LTND: Active chain: Bitcoin (network=simnet)
2018-10-22 17:31:01.140 [INF] CHDB: Checking for schema update: latest_version=6, db_version=6
2018-10-22 17:31:01.236 [INF] LTND: Primary chain is set to: bitcoin
2018-10-22 17:31:02.391 [INF] LNWL: Opened wallet
2018-10-22 17:31:03.315 [INF] LNWL: The wallet has been unlocked without a time limit
2018-10-22 17:31:03.315 [INF] LTND: LightningWallet opened
2018-10-22 17:31:03.319 [INF] LNWL: Catching up block hashes to height 3060, this will take a while...
2018-10-22 17:31:03.320 [INF] HSWC: Restoring in-memory circuit state from disk
2018-10-22 17:31:03.320 [INF] LNWL: Done catching up block hashes
2018-10-22 17:31:03.320 [INF] HSWC: Payment circuits loaded: num_pending=0, num_open=0
2018-10-22 17:31:03.322 [DBG] LTND: Populating dependencies for sub RPC server: Signrpc
```
As for the config, an example is:
```
[signrpc]
signrpc.signermacaroonpath=~/signer.macaroon
```
By passing a pubkey into SendToRoute, it becomes unnecessary for lnd to
query the channel graph to retrieve the hop pubkey. This allows routes
over private channels that are not present in the graph.
In this commit, we ensure that we don't include routing hints for
unadvertised nodes at the time of invoice creation. Otherwise, this
would lead us to leak these unadvertised nodes to anyone who can get
their hands on the invoice being created. To prevent this, we'll now
look at the network graph and ensure that the node in unadvertised if
all of their edges are unadvertised and only extend to us.
In this commit, we address a slight regression in the defaults provided
by the OpenChannel RPC with regards to spending unconfirmed outputs to
fund funding transactions. We now add a new boolean parameter to
OpenChannelRequest: SpendUnconfirmed. If set, we'll use this to
indicate to the server that the funding transaction of the channel to be
created can spend the wallet's unconfirmed outputs to fund it. This
addition is needed because otherwise, if the caller doesn't specify a
MinConfs parameter to the request, then a default of 0 will be used,
which is not ideal.
In this commit the dependency of unmarshallRoute on edge policies being
available is removed. Edge policies may be unknown and reported as nil.
SendToRoute does not need the policies, but it does need pubkeys of the
route hops. In this commit, unmarshallRoute is modified so that it
takes the pubkeys from edgeInfo instead of channelEdgePolicy.
In addition to this, the route structure is simplified. No more connection
to the database at that point. Fees are determined based on incoming and
outgoing amounts.
In this commit, we extend the AbandonChannel RPC method to store as much
information as possible within the CloseSummary. Most importantly, we
store the current and next (if available) revocation of the remote party
along with the channel config for the local party. With this
information, we have enough data to be able to generate a custom signer
for a new commitment state, or to prove that the channel was indeed
ours if needed.
In this commit, we remove the unnecessary NewWitnessAddress RPC. We do
this as, at the moment, we only expose the wallet to generate witness
addresses. Now that the RPC has been removed, its REST endpoint has been
assigned to the NewAddress RPC, which should be the de facto way of
having the wallet generate any type of supported address.
Fixes#1986.
Using AbandonChannel, a channel can be abandoned. This means
removing all state without any on-chain or off-chain action.
A close summary is the only thing that is stored in the db after
abandoning.
A specific close type Abandoned is added. Abandoned channels
can be retrieved via the ClosedChannels RPC.
In a previous commit, we modified our fee rates to be sat/kw internally
as it was possible that the estimated fee rate was not enough to
properly broadcast the transaction. To remedy this, we decided to add a
fee floor, but this would only be enforced when querying the fee
estimator. In this commit, we attempt to do the same thing, but for fee
rates input manually by the user.
The commit ensures that for every channel, there will always
be two entries in the edges bucket. If the policy from one or
both ends of the channel is unknown, it is marked as such.
This allows efficient lookup of incoming edges. This is
required for backwards payment path finding.
In this commit, we explicitly convert sat/vbyte fee rates input by the
user to sat/kw. We do this as users are typically more accustomed to
sat/vbyte fee rates, rather than sat/kw.
In this commit, we fix a slight regression that happened when the
sendtoroute RPC call was implemented. As a result of the refactoring,
the debughtlc mode silently broke. In this commit, we fix this by
checking against the zero hash, rather than the length of the payment
hash, as since we use a fixed sized byte slice, it will always be zero.
- Extend SendRequest and QueryRoutesRequest protos
- newRoute function takes fee limit and cuts off routes that exceed it
- queryRoutes, payInvoice and sendPayment commands take the feeLimit inputs and pass them down to newRoute
- When no feeLimit is included, don't enforce any feeLimits at all (by setting feeLimit to maxValue)
In this commit, we modify the logic for the two primary payment related
RPCs to reduce duplication, indentation, and consolidate logic. As a
result, we'll now accept rpcPaymentIntents, turn those into regular
paymentIntents (convert from proto) before finally dispatching via
unified function.
In this commit, we now allow connections to onion addresses due to
recently adding support to properly parse them.
Co-Authored-By: Eugene <crypt-iq@users.noreply.github.com>
In this commit, we fix an existing panic bug related to the recently
added routing hints feature. If it's the case that the remote node
didn't send us their edge, then when we go to compare the public keys to
see if they match, we may attempt to deref an nil pointer.
In order to fix this, we'll instead check the edgeInfo, which is
guaranteed to also exist if the channel was found in the database. As a
defensive step, before we go to actually aces the struct, we'll check
that's it's non-nil and proceed if it is nil.
In this commit, we modify the SignMessage+VerifyMessage slightly in
order to bind the message we're signing to our specific context. We do
this by pre-pending the ascii text "Lightning Signed Message:" to the
msg before we run it through dsha256. Adding this prefix binds the
message to our context in order to avoid accidentally signing a sighash
or the like.
In this commit, we avoid hitting a possible panic in the case that we
were unable to retrieve the link of a channel from the HTLC switch while
determining which channels should be used as routing hints for an
invoice.
This reverts commit 0f90c950daaec7843e305f32d10e36411fb0b47f.
We actually still need to notify the BRAR of a force close, as
otherwise, it doesn't have a signal to exit due to a local force close
event.
This commit fixes a bug where all the HTLC rhash slices in a
ListChannelsResponse would be tied to the loop variable, making them all
take the hash of the last HTLC in the list. This commit fixes it by
making a copy of the slice.
In this commit, we update the testUpdateChannelPolicy to exercise the
recent set of changes within the switch. If one applies this test to a
fresh branch (without those new changes) it should fail. This is due to
the fact that before, Bob would attempt to apply the constraints of the
incoming link (which we updated) instead of the outgoing link. With the
recent set of changes, the test now properly passes.
In this commit, we follow up to the prior commit by ensuring we won't
accept a co-op close request for a chennel with active HTLCs. When
creating a chanCloser for the first time, we'll check the set of HTLC's
and reject a request (by sending a wire error) if the target channel
still as active HTLC's.
In this commit, we fix a bug that at times could cause any dangling
HTLC's to be sent to miner's fees if a user attempted to close out a
channel cooperatively that still had pending HTLC's. We'll first prevent
this at the RPC level by rejecting any attempts to trigger a co-op
channel closure while a channel still have dangling HTLC's.
In this commit, we modify the existing invoice RPC macaroon permissions
to target a more specific entity: “invoices”. As a result of this
commit, once node operators update, they’ll need to regenerate their
readonly.macaroon as it now needs this additional entity encoded within
it.
In the prior commit, we added a new set of permissions and also a new
entity: “invoices”. We’ll add this set of entities to the read and
write permissions accordingly as well to ensure that the existing
macaroons have access to all the items that the invoice.macaroon does.
In this commit, we add a new invoicePermissions slice. This contains
all the permission that a holder of an invoice.macaroon is able to
access, and no others. We also include read and write access to
addresses as this may be useful from the PoV of a merchant or exchange.
In this commit, we raise the min channel size to 20k satoshis. This
will be evaluated before we check for dusty commitments. The goal of
this is to ensure ample room for fees at current, and future fee
levels.
In this commit, we fix a bug in the query routes RPC that could at
times lead to a panic. This would happen if the number of returned
routes was less than the number of expected routes. To remedy this,
we’ll return the minimum of the number of requested routes, and the
number of routes actually returned.
This commit adds wallet_best_block_timestamp to the gRPC interface.
This is done in order to allow clients to calculate progress while
lnd syncs to the blockchain. wallet_best_block_timestamp is exposed
via the GetInfo() rpc call. Additionally, IsSynced() returns the
WalletBestBlockTimestamp as the second value in the tuple
that is returned, providing additional detail when querying about the
status of the sync. The BtcWallet interface has also been updated
accordingly.
This commit was created to support the issue to
[Add progress bar for chain sync] (lightninglabs/lightning-app#10) in
lightning-app
This commit adds a new interface named NetInterface and two
implementations of it: RegularNet & TorProxyNet. These two structs
are used in config.go in an attempt to clean up the code and
abstract away the dialer and DNS functions.
This commit adds Tor support. Users can set the --TorSocks flag
to specify which port Tor's SOCKS5 proxy is listening on so that
lnd can connect to it. When this flag is set, ALL traffic gets
routed over Tor including DNS traffic. Special functions for
DNS lookups were added, and since Tor doesn't natively support
SRV requests, the proxySRV function routes connects us to
a DNS server via Tor and SRV requests can be issued directly
to the DNS server.
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
This commit reworks the macaroon authentication framework to use the
v2 macaroon format and bakery API. It also replaces the code in each
RPC method which calls the macaroon verifier with interceptors which
call the macaroon verifier instead. In addition, the operation
permissions are reworked to fit the new format of "allow" commands
(specifically, entity/operation permissions instead of method
permissions).
Since a ChannelPoint's funding txid can now be get/set as raw bytes or
a string, we first need to check what type it's currently set to before
accessing it.
Fixes#481.
Prior to this commit, payments stored in the channel DB only kept a
record of the payment hash. This is a problem as the preimage is what
serves as proof of payment and a user should be able to look up this
value in the future (not just immediately after payment).
Instead of storing both the payment hash and the preimage, we store the
preimage only since the hash can be derrived from this using a SHA256.
In the RPC listpayments command, we now give the preimage in addition to
the payment hash.
We no longer need to hand off new channels that come online as the
chainWatcher will be persistent, and always have an active signal for
the entire lifetime of the channel.
This commit changes the name of the UpdateFee method to
UpdateChannelPolicy, to mimic the recent proto change.
It also reads and validates the passed TimeLockDelta,
and sends it to the gossiper for announcing it to the
network, and to the switch for updating the forwarding
policy of the links.
In this commit, we remove the blocks_till_open from
PendingChannelsResponse as in its current state, the values that are
assigned to this field don’t accurately reflect the naming. This has
caused a good bit of confusion amongst users lately. As a result, we’re
temporarily removing this field until we have proper incremental
notifications within the chain notifier.
In this commit we rename the lnrpc.PendingChannelRequest and
lnrpc.PendingChannelResponse to
lnrpc.PendingChannelsRequest/lnrpc.PendingChannelsResponse. We do this
as we strive to ensure that the naming scheme across the RPC interface
is consistent.
On testate as times the fee estimation can swing widely. As we
currently don’t yet use vsize everywhere internally, we’re forced to
manually scale to weight for the moment. If the returned fee rate is
too low, then it can cause our estimate to go to zero. This also has
the effect of meaning that the chanCloser doesn’t currently advance if
the initial starting fee is zero.
In this commit we fix a cosmetic bug within our RPC output for list
channels. We have a policy of always showing SAT instead of mSAT
externally. This led to user confusion, as if Alice or Bob ended up
with a fractional amount of satoshis, then the sum of trimmed amount
would be silently sent to miner’s fees. An example being: Alice ending
up with `8998999 mSAT` (`8998.999 SAT`). Bob similarly ends up with
`1001001 mSAT` (`1001.001 SAT`). `8998.999 + 1001.001 = 10000.0 SAT`.
However, we can't express that fractional amount (totaling `1 SAT`
across both commitment transactions) so it goes to miner fees.
To remedy this on the RPC interface level, we’ll now detect if we have
a dangling satoshi, and properly list it as going towards the miner fee
on the commitment transaction.
Fixes#468.
In this commit, we further constrain the candidacy for an “active”
channel. In addition to being present within the link, it *must* also
have the RemoteNextRevocation set. Otherwise, this indicates that we
haven’t yet processed a FundingLocked message for this channel.
In this commit, the relevant RPC’s have been modified to properly
parse, and pass through the new fee control related parameters for each
of the RPC’s.
For a calls to ListChannels we now only set the
ActiveChannel.Active=true if the link is found by the
htlcswitch. This is done to be able to make it possible
to tell if a newly opened channel has been added to
the htlcswitch, such that we can synchronize on this
during tests before we attempt to close the channel.
This helps catch issues in the RPC tests faster by putting a timeout
on all SendPayment requests. Otherwise, if a payment stalled, the test
would run until the test suite timeout, 10 minutes. Also simplifies
some code with a helper function and using SendPaymentSync where
possible.
In this commit we modify the primary InvoiceRegistry interface within
the package to instead return a direct value for LookupInvoice rather
than a pointer. This fixes an existing race condition wherein a caller
could modify or read the value of the returned invoice.
This commit changes the rpcserver to rely on the new zpay32
package, and support the new payment request options available
in the BOLT-11 invoice format.
This commit modifies the relevant RPC’s to enforce the max payment size
as defined in BOLT-0002. With this commit the largest payment possible
is now 2^32 mSAT, or 4,294,967 satoshis.
