This commit aims to address a flaw in our anchor reserve enforcement
logic in which an inbound "legacy" channel (i.e. a channel with a
commitment type that precedes anchors) would be rejected by the
recipient if they have at least one opened channel using the anchors
commitment type and do not have enough on-chain funds to meet the
anchors reserve.
We do this to trigger a bug that will be resolved in a follow-up commit.
This bug prevents inbound legacy channels from being rejected by the
recipient if they have at least one anchor channel already opened
without an on-chain balance.
This log can be "spammy" while nodes throughout the network have yet to
upgrade to v0.13.0-beta, which includes several enhancements to prevent
the broadcast of zombie edges/updates.
GetNodeInfo retrieves the policies for every edge the node belongs to.
When these policies are retrieved from the database, they're returned
in the following order: the first policy is the outgoing policy from the
node, and the second is the incoming policy to the node. This ordering
is not consistent with the ordering we have within our other RPCs like
GetChanInfo and DescribeGraph, where policies are sorted based on the
smaller public key of the nodes within an edge.
We fix this by maintaining the same order as our other RPCs.
This ensures the waiting receiving channel always receives an error to
prevent a deadlock when processing a network update that fails due to
the validation barrier.
On commit d5aedbcbd9:
1000 @ 0x43a285 0x44a38f 0xc42e86 0xc80fda 0xc8682d 0xc976c9 0x46fce1
github.com/lightningnetwork/lnd/routing.(*ChannelRouter).AddNode+0x245 github.com/lightningnetwork/lnd/routing/router.go:2218
github.com/lightningnetwork/lnd/discovery.(*AuthenticatedGossiper).addNode+0x3b9 github.com/lightningnetwork/lnd/discovery/gossiper.go:1510
github.com/lightningnetwork/lnd/discovery.(*AuthenticatedGossiper).processNetworkAnnouncement+0x574c github.com/lightningnetwork/lnd/discovery/gossiper.go:1554
github.com/lightningnetwork/lnd/discovery.(*AuthenticatedGossiper).networkHandler.func1+0x24github.com/lightningnetwork/lnd/discovery/gossiper.go:1043
Since we don't have to worry about network latency within our
integration tests, we can shorten the broadcast timeout for neutrino
integration tests from 5s to 1s.
We'd never decrement the number of pending backups upon a watchtower
accepting one, making it confusing for users to determine whether their
backups have actually been accepted. Along the way, we also rename
NumTasksReceived to NumTasksPending to better reflect its purpose.
Previously, we would always allow dependent jobs to be processed,
regardless of the result of its parent job's validation. This isn't
correct, as a parent job contains actions necessary to successfully
process a dependent job. A prime example of this can be found within the
AuthenticatedGossiper, where an incoming channel announcement and update
are both processed, but if the channel announcement job fails to
complete, then the gossiper is unable to properly validate the update.
This commit aims to address this by preventing the dependent jobs to
run.
The recently added gossip throttling was shown to be too aggressive,
especially with our auto channel enable/disable signaling. We switch to
a token bucket based system instead as it's based on time, rather than a
block height which isn't constantly updated at a given rate.
We do this instead of using the source of the AnnounceSignatures
message, as we filter out the source when broadcasting any
announcements, leading to the remote node not receiving our channel
update. Note that this is done more for the sake of correctness and to
address a flake within the integration tests, as channel updates are
sent directly and reliably to channel counterparts.
We do this instead of using the source of the AnnounceSignatures
message, as we filter out the source when broadcasting any
announcements, leading to the remote node not receiving our channel
update. Note that this is done more for the sake of correctness and to
address a flake within the integration tests, as channel updates are
sent directly and reliably to channel counterparts.
As similarly done with premature channel announcements, we'll no longer
allow premature channel updates to be rebroadcast once mature. This is
no longer necessary as channel announcements that we're not aware of are
usually broadcast to us with their accompanying channel updates.
This change was largely motivated by an increase in high disk usage as a
result of channel update spam. With an in memory graph, this would've
gone mostly undetected except for the increased bandwidth usage, which
this doesn't aim to solve yet. To minimize the effects to disks, we
begin to rate limit channel updates in two ways. Keep alive updates,
those which only increase their timestamps to signal liveliness, are now
limited to one per lnd's rebroadcast interval (current default of 24H).
Non keep alive updates are now limited to one per block per direction.
This addresses a potential panic when a tower has one of its candidate
sessions chosen, but its only reachable address was removed by a
user-initiated RPC before the fact.
A while back, changes were made to the wallet such that it waits for the
backend to be synced before beginning to store the latest 10,000 blocks
of the chain. This inherently broke sync progress implementations based
on the best_header_timestamp result from the GetInfo RPC for
neutrino-based nodes as the wallet is no longer tracking all blocks in
the chain. To work around this, we now make sure to return the backend's
best header timestamp instead of the wallet's, allowing said sync
progress implementations to work again.
This is useful when we wish to have a channel frozen for a specific
amount of blocks after its confirmation. This could also be done with an
absolute thaw height, but it does not suit cases where a strict block
delta needs to be enforced, as it's not possible to know for certain
when a channel will be included in the chain. To work around this, we
add a relative interpretation of the field, where if its value is below
500,000, then it's interpreted as a relative height. This approach
allows us to prevent further database modifications to account for a
relative thaw height.
This was initially done as there were a few assertions throughout the
codebase requiring a channel's policy to be known. Now that these have
been addressed, we no longer need to store restored channels in the
graph, as their policies where incomplete anyway.
The current implementation of LeaseOutput already checked whether the
output had already been leased by the persisted implementation, but not
the in-memory one used by lnd internally. Without this check, we could
potentially end up with a double spend error if lnd acquired the UTXO
internally before the LeaseOutput call.
This addresses a panic when a notification is canceled after its been
detected as included in a block and before its confirmation notification
is dispatched.
This is meant to handle a quirk in which key descriptors obtained
through walletrpc.DeriveKey don't result in the derived key being
persisted to the wallet's database, unlike with DeriveNextKey. Due to
this and some fallback logic in the wallet with regards to empty key
locators, if a request only specified the compressed public key, the
signature returned would be over a different key, namely the one derived
from (family=0, index=0).
These race conditions originate from the mock database storing and
returning pointers, rather than returning a copy.
Observed on Travis:
WARNING: DATA RACE
Read at 0x00c0003222b8 by goroutine 149:
github.com/lightningnetwork/lnd/watchtower/wtclient.(*sessionQueue).drainBackups()
/home/runner/work/lnd/lnd/watchtower/wtclient/session_queue.go:288 +0xed
github.com/lightningnetwork/lnd/watchtower/wtclient.(*sessionQueue).sessionManager()
/home/runner/work/lnd/lnd/watchtower/wtclient/session_queue.go:281 +0x450
Previous write at 0x00c0003222b8 by goroutine 93:
github.com/lightningnetwork/lnd/watchtower/wtclient.getClientSessions()
/home/runner/work/lnd/lnd/watchtower/wtclient/client.go:365 +0x24f
github.com/lightningnetwork/lnd/watchtower/wtclient.(*TowerClient).handleNewTower()
/home/runner/work/lnd/lnd/watchtower/wtclient/client.go:1063 +0x23e
github.com/lightningnetwork/lnd/watchtower/wtclient.(*TowerClient).backupDispatcher()
/home/runner/work/lnd/lnd/watchtower/wtclient/client.go:784 +0x10b9
The logger string used to identify the wtclient and wtclientrpc loggers
was the same, leading to being unable to modify the log level of the
wtclient logger as it would be overwritten with the wtclientrpc's one.
To simplify things, we decide to use the existing RPC logger for
wtclientrpc.
There seems to be a misinterpretation of a variable between the
btcwallet and gozmq libraries. When establish a ZMQ connection, it
expects a timeout, which is used to set read deadlines and determine how
long we should wait before attempting a reconnection. Within btcwallet
and lnd however, this is is interpreted as a polling duration,
explaining the current value of 100ms. Under load, especially on
less-capable hardware, this leads to high resource usage as we get into
a constant reconnection loop. To remedy this, we use a timeout of 5s
instead, which is a much more reasonable value for read timeouts, and is
also what's used for LN peers.
A peer's remote address isn't known to us if we accepted the connection
over Tor, instead we know the address the onion service used to dial to
lnd. If said peer also doesn't have any advertised addresses, then we
don't have enough information to attempt a reconnect, so we avoid doing
so. Allowing the reconnection to happen isn't necessarily an issue, but
not allowing it prevents the configured SOCKS proxy from dialing to
private addresses.
This addresses a bug in which a funding reservation wasn't cleaned up
properly if the remote peer didn't support upfront shutdown.
Alternatively, we could just cancel the reservation on error, but
instead we move the check above so that we don't attempt coin selection
in the first place.
If the server hasn't fully started yet, it's possible that the channel
event store hasn't either, so it won't be able to consume any requests
until then. To prevent blocking, we'll just omit the uptime related
fields for now.
This is mainly motivated by a now fixed bug in the wallet in which
change addresses could at times be created outside of the default key
scopes. Recovery only used to be performed on the default key scopes, so
ideally this test case would've caught the bug earlier.
We'll only do so when we have at least one confirmed advertised channel
with the remote peer. This ensures that our node announcement can
propagate throughout the network even for nodes that were offline when
we initially broadcast it.