In this commit, we introduce a migration to fix some of the recent
issues found w.r.t. the edge update index. The migration attempts to fix
two things:
1) Edge policies include an extra byte at the end due to reading an
extra byte for the node's public key from the serialized node info.
2) Properly prune all stale entries within the edge update index.
As a result of this migration, nodes will have a slightly smaller in
size channeldb. We will also no longer send stale edges to our peers in
response to their gossip queries, which should also fix the fetching
channel announcement for closed channels issue.
The commit ensures that for every channel, there will always
be two entries in the edges bucket. If the policy from one or
both ends of the channel is unknown, it is marked as such.
This allows efficient lookup of incoming edges. This is
required for backwards payment path finding.
In this commit, we extend the server's functionality to prune link nodes
on startup. Since we currently only decide whether to prune a link node
from the database based on a channel close, it's possible that we have
link nodes lingering from before this functionality was added on.
In this commit, we migrate the database away from a partially migrated
state. In a prior commit, we migrated the database in order to update
the Invoice struct with three new fields: add index, settle index, paid
amt. However, it was overlooked that the OutgoingPayment struct also
embedded an Invoice within it. As a result, nodes that upgraded to the
first migration found themselves unable to start up, or call
listpayments, as the internal invoice within the OutgoignPayment hadn't
yet been updated. This would result in an OOM typically as we went to
allocate a slice with a integer that should have been small, but may
have ended up actually being a set of random bytes, so a very large
number.
In this commit, we finish the DB migration by also migrating the
internal invoice within each OutgoingPayment.
Fixes#1538.
Fixes#1546.
In this commit, we add a new database migration required to update old
database to the version of the database that tracks the update index for
the nodes and edge policies. The migration is straight forward, we
simply need to populate the new indexes for the all the nodes, and then
all the edges.
This commit adds a new method FetchWaitingCloseChannels to the database,
used for fetching OpenChannels that have a ChanStatus != Default. These
are channels that are borked, or have had a commitment broadcasted, and
is now waiting for it to confirm.
The fetchChannels method is rewritten to return channels exclusively
based on wheter they are pending or waitingClose.
This commit adds a FetchClosedChannel method to the
channeldb, which allows querying based on a known
channel point. This will be used in the nursery to
load channel close summaries, which can be used to
provide more accurate height hints when recovering
from failures.
In this commit we add a new MarkAsOpen method to the OpenChannel
struct. This method replaces the existing MarkChannelAsOpen method
which targeted the database struct itself.
This commit modifies the OpenChannel struct to include the full short
channel ID rather than simply the opening height. This new field will
be needed by an upcoming change to uniformly switch to using short
channel ID’s when forwarding HTLC’s due to the change in per-hop
payloads.
This commit expands the field within the OpenChannel struct in order to
start tracking the height that the funding transaction was initially
broadcast. Other sub-systems within lnd can now use this data to give a
more accurate height hint to the ChainNotifier, or to use during the
funding workflow to decide if a channel should be forgotten after it
fails to confirm for N blocks.
This commit modifies the OpenChannel structure on-disk to also track
that opening height of a channel. This change is being made in order to
make and more light client friendly. A follow up commit will modify
several areas of the codebase to use this new functionality.
This commit adds a new method to the database which allows callers to
mark a channel as fully closed once certain conditions have been
reached. If a channel was cooperatively closed, then it can be marked
as fully closed as soon as the channel has been confirmed. If the
channel was marked as force closed, then it should be marked as closed
as soon as _all_ the funds in limbo have been swept.
This commit builds upon the prior commit by adding a new method to
allows callers to query the state of all closed (fully) and pending
closed channel within the database.
In order to facilitate persistence during the funding process, added
the isPending flag to channels so that when the daemon restarts, we can
properly re-initialize the chain notifier and update the state of
channels that were going through the funding process.
This commit fixes a bug which would previously lead to corruption of
the channel state when a node had one or more channels open and one of
them was closed either forcibly or cooperatively. The source of the bug
itself as a typo: rather than using the construed `deliveryKey`
variable to fetch/put/delete the delivery scripts, `deliveryScriptsKey`
(the key prefix itself) as used. This bug would cause the database to
be unable to read _any_ channel from the database after one was
deleted, as each channel would actually be reading/writing-to the
_exact same_ delivery script.
The fix for the bug itself is simple: eliminate the typo.
Previously, the edge index bucket which maps a channelPoint ->
channelID wasn’t properly created one start up during the initial
creation of the database. This caused some extraneous failure as
queries would unnecessarily fail with bucket non-existence errors.
To fix this we now properly create the bucket on start up if the
database doesn’t exist, and also properly delete the bucket within the
Wipe() function.
This commit adds support for channel graph pruning, which is the method
used to keep the channel graph in sync with the current UTXO state. As
the channel graph is essentially simply a subset of the UTXO set, by
evaluating the channel graph with the set of outfits spent within a
block, then we’re able to prune channels that’ve been closed by
spending their funding outpoint. A new method `PruneGraph` has been
provided which implements the described functionality.
Upon start up any upper routing layers should sync forward in the chain
pruning the channel graph with each newly found block. In order to
facilitate such channel graph reconciliation a new method `PruneTip`
has been added which allows callers to query current pruning state of
the channel graph.
This commit introduces a new capability to the database: storage of an
on-disk directed channel graph. The on-disk representation of the graph
within boltdb is essentially a modified adjacency list which separates
the storage of the edge’s existence and the storage of the edge
information itself.
The new objects provided within he ChannelGraph carry an API which
facilitates easy graph traversal via their ForEach* methods. As a
result, path finding algorithms will be able to be expressed in a
natural way using the range methods as a for-range language extension
within Go.
Additionally caching will likely be added either at this layer or the
layer above (the RoutingManager) in order keep queries and outgoing
payments speedy. In a future commit a new set of RPC’s to query the
state of a particular edge or node will also be added.
