This commit restructures the initialization procedure
for chain watchers such that they can proceed in parallel.
This is primarily to help nodes running with the neutrino
backend, which otherwise forces a serial rescan for each
active channel to check for spentness.
Doing so allows the rescans to take advantage of batch
scheduling in registering for the spend notifications,
ensuring that only one or two passes are made, as opposed
to one for each channel.
Lastly, this commit ensures that the chain arb is properly
shutdown if any of it's chain watchers or channel arbs
fails to start, so as to cancel their goroutines before
exiting.
In this commit, we no longer assume that the bucket hierarchy has been
created properly when applying the latest DB migration. On older nodes
that never obtained a channel graph, or updated _before_ the query sync
stuff was added, then they're missing buckets that the migration expects
them to have.
We fix this by simply creating the buckets as we go, if needed.
Previously, gossiper was the only object that validated channel
updates. Because updates can also be received as part of a
failed payment session in the routing package, validation logic
needs to be available there too. Gossiper already depends on
routing and having routing call the validation logic inside
gossiper would be a circular dependency. Therefore the validation
was moved to routing.
This commit replaces the simplistic rate limiting
technique added in 557cb6e2, to use the
golang.org/x/time's rate limiter. This has the
benefit of performing traffic shaping to meet a
target maximum rate, and yet tolerate bursts. Bursts
are expected during initial sync, though should become
more rare afterwards. Performing traffic shaping with
this mechanism should improve the ability of the gossip
syncer to detect sustained bursts from the remote peer,
and penalize them appropriately.
This commit also modifies the default parameters to
accept bursts of 10 queries, with a target rate of 1
reply every 5 seconds.
In this commit, we fix a bug in the bucket creation code in
createChannelDB. This bug can cause migrations on older nodes to fail,
as we expect the bucket to already have been created. With this commit,
we ensure that all the buckets under the main node and edges bucket are
properly created. Otherwise, a set of the newer migrations will fail to
apply for nodes updating from 0.4.
In this commit, we move the declaration of the key for an unused bucket.
In the past, this bucket was used to store the revocation forwarding
package log. However, this has been moved under the key
`fwdPackagesKey`.
In this commit, we restrict the persistent connection logic on startup
to only attempt to establish connections to Tor addresses if Tor
outbound support is enabled. Otherwise, we'll continually attempt to
reach the address even though we never will.
In this commit, we fix a flake in the link node garbage collection test
by ensuring the channels have been fully closed on both sides before we
attempt to restart and ensure that they don't actually establish
connections. Without this check, it's possible that either side hasn't
yet processed all the blocks, so they'll still reconnect to each other on
start up.
In this commit, we fix an existing but that would cause the daemon to at
times crash. Before this commit, we access the wrong edge, which would
possibly actually be nil, leading to a panic. In this commit we fix this
by ensuring we access the proper edge which is known to be non-nil at
this point in the control flow.
This commit ensures that the mock attachment
directives use unique keys, ensuring that they
aren't skipped due to already having pending
connection requests. The tests fail when
they're all the same since they collide
in the pendingConns map.
This commit modifies the autopilot agent to track
all pending connection requests, and forgo further
attempts if a connection is already present.
Previously, the agent would try and establish
hundreds of requests to a node, especially if the
connections were timing out and not returning.
This resulted in an OOM OMM when cranking up
maxchannels to 200, since there would be close
to 10k pending connections before the program was
terminated. The issue was compounded by periodic
batch timeouts, causing autopilot to try and
process thousands of triggers for failing
connections to the same peer.
With these fixes, autopilot will skip nodes that we
are trying to connect to during heuristic selection.
The CPU and memory utilization have been significantly
reduced as a result.
