In this commit, we migrate all the code in `channeldb` to only reference
the new `kvdb` package rather than `bbolt` directly.
In many instances, we need to add two version to fetch a bucket as both
read and write when needed. As an example, we add a new
`fetchChanBucketRw` function. This function is identical to
`fetchChanBucket`, but it will be used to fetch the main channel bucket
for all _write_ transactions. We need a new method as you can pass a
write transaction where a read is accepted, but not the other way around
due to the stronger typing of the new `kvdb` package.
In this commit, we create a new package `kvdb`, which is meant to serve
as the basis for any future database abstractions within `lnd`. Rather
than directly use the `walletdb` package (which we base off of), we
instead use a series of type-aliases to re-type the fundamental
types/interfaces of the `walletdb` package. This lets us type
`kvdb.RwTx` instead of `walletdb.ReadWriteTransaction` everywhere.
Additionally, our usage of type-aliases is also intended to create an
easy pathway in the future wherein we can gradually re-defined or
re-implement these types to wean off of the `walletdb` package.
This commit migrates the payments in the database to a new structure
that allows for multiple htlcs per payments. The migration introduces a
new sub-bucket that contains a list of htlcs and moves the old single
htlc into that.
Add an optional channel status CloseChannel which will be stored on the
hitsorical channel which is persisted at channel close. This status is
used to set the close initiator for channels that do not complete the
funding flow or we abandon. In follow up commits, this status will be
used to record force and breach closes. The value is written to the
historical channel bucket for diplay over rpc.
This changes replaces the pending an waiting booleans in fetchChannels
with optional filters which can be more flexibly used. This change
allows filtering of channels without having to reason about the matrix
of possible boolean combinations. A test is added to ensure that the
combinations of these filters act as expected.
This commit removes the migrations from channeldb and references those
in the migrations_01_to_11 package. This creates a one-way dependency on
the migrations. Future changes to channeldb won't be able to break
migrations anymore.
This commit adds a set of htlcs to the Invoice struct and
serializes/deserializes this set to/from disk. It is a preparation for
accurate invoice accounting across restarts of lnd.
A migration is added for the invoice htlcs.
In addition to these changes, separate final cltv delta and expiry
invoice fields are created and populated. Previously it was required
to decode this from the stored payment request. The reason to create
a combined commit is to prevent multiple migrations.
This commit specifies two bbolt options when opening the underlying
channel and watchtower databases so that there is reduced heap
pressure in case the bbolt database has a lot of free pages in the
B+ tree.
migrateOutgoingPayments moves the OutgoingPayments into a new bucket format
where they all reside in a top-level bucket indexed by the payment hash. In
this sub-bucket we store information relevant to this payment, such as the
payment status.
To avoid that the router resend payments that have the status InFlight (we
cannot resume these payments for pre-migration payments) we delete those
statuses, so only Completed payments remain in the new bucket structure.
During the restore process, it may be possible that we have already
heard about our prior edge from a node on the network (or our channel
peers). As a result, we shouldn't exit if this happens, and instead
should continue with the rest of the restoration process.
In this commit, we modify the `AddrsForNode` method to not fail if no
graph node is found. We do this as when the backup is being
restored/created, it's possible that we don't yet have a
NodeAnnouncement for that node.
In this commit, we move the location where we restore the channel status
to within the `RestoreChannelShells` method itself. Before this commit,
we attempted to use `ApplyChanStatus` which creates a DB transaction and
relies on a fully populated channel state, which in the restoration
case, we don't yet have.
In this commit, we add a zombie edge index to the database. This allows
us to quickly determine across restarts whether we're attempting to
process an edge we've previously deemed as zombie.
In this commit, we introduce a migration for the message store
sub-bucket that will migrate all keys within it to a new key format.
This new key format is composed of the peer's public key, followed by
the short channel ID, followed by the message type. This migration is
needed in order to provide backwards-compatibility with messages that
were previously stored before the introduction of the new key format.
In this commit, we add a new type (ChannelShell) along with a new
method, RestoreChannelShells which allows a caller to insert a series of
channel shells into the database. These channel shells will allow a
restored node to initiate the DLP protocol and recover their set of
existing channels.
When we insert a channel shell, we re-create the original link node, and
also add the outgoing edge to the channel graph. This way we can be sure
that upon start up, we attempt to connect to the remote peers, and that
the normal graph query commands will operate as expected.
In this commit, we add a prefix naming scheme to the ChannelStatus enum
variables. We do this as it enables outside callers to more easily
identify each individual enum variable as a part of the greater
enum-like type.
In this commit, we add a new AddrsForNode method. This method will allow
a wrapper sturct to implement the new chanbackup.LiveChannelSource
method which is required to implement the full SCB feature set.
In this commit, we address an issue with the FetchWaitingCloseChannels
method where it would not properly return channels that are unconfirmed
and also have an unconfirmed closing transaction because they were
filtered out. We fix this by fetching channels that remain unconfirmed
that are also waiting for a confirmed closing transaction.
This will allow the recently added test TestFetchWaitingCloseChannels to
pass.
In this commit, we fix a bug in the bucket creation code in
createChannelDB. This bug can cause migrations on older nodes to fail,
as we expect the bucket to already have been created. With this commit,
we ensure that all the buckets under the main node and edges bucket are
properly created. Otherwise, a set of the newer migrations will fail to
apply for nodes updating from 0.4.
In this commit, we introduce a migration to fix some of the recent
issues found w.r.t. the edge update index. The migration attempts to fix
two things:
1) Edge policies include an extra byte at the end due to reading an
extra byte for the node's public key from the serialized node info.
2) Properly prune all stale entries within the edge update index.
As a result of this migration, nodes will have a slightly smaller in
size channeldb. We will also no longer send stale edges to our peers in
response to their gossip queries, which should also fix the fetching
channel announcement for closed channels issue.
The commit ensures that for every channel, there will always
be two entries in the edges bucket. If the policy from one or
both ends of the channel is unknown, it is marked as such.
This allows efficient lookup of incoming edges. This is
required for backwards payment path finding.
