In this commit we comptely overhaul the existing storage of the
OpenChannel struct to use the new common serialization defined within
the codec.go file. Additionally, we’ve modified the structure of the
channel database on disk. Rather then use the existing prefix based
segmentation, everything is now bucket based. This has resulted in much
simpler and easier to follow code. The bucket progression is:
openChannelBucket -> nodeBucket -> chainBucket -> channelBucket. We add
a chainBucket as it’s possible that in the future we may have several
channels on distinct chains with a given node.
With the above changes, we’re able to delete much of the existing code
within the file, drastically reducing its size.
By adding these two fields, it is now possible to fully reconstruct the
channel’s update log from the set of HTLC’s stored on disk, as we now
properly note both the log index and HTLC index. Prior to this commit
we would simply start the new log index based on the amount of HTLC’s
that were present in the prior state.
In this commit, we restructure the OpenChannel struct to used two
distinct ChannelCommitments: one for the remote party, and one for the
local party. With this, we now partition the local state and the remote
state. Previously, we weren’t properly tracking the state of the remote
party. As a result, there were certain classes of failures we were
unable to properly recover from. With this separation, we can now
handle them, and the OpenChannel struct is slimmer and more
understandable.
In this commit we’ve added a new struct to the package:
ChannelCommitment. This sturct houses all the common data the comprises
a particular commitment state. This will soon replace the open fielded
commitmetn fields within the OpenChannel struct.
In this commit htlc channeldb representation have been augmented
with onion blob field, and (de)serialisaion functions have been changed
to make the onion blob persistant.
Use binary.Read/Write in functions to serialize and deserialize
channel close summary and HTLC boolean data, as well as in
methods to put and fetch channel funding info. Remove lnd
implementations of readBool and writeBool as they are no
longer needed. Also fix a few minor typos.
Persists the state of a channel opening process after funding
transaction is confirmed. This tracks the messages sent to
the peer such that the process can be continued in case of a
restart. Also introduces that the receiver side forgets about
channel if funding transaction is not confirmed in 48hrs.
This commit removes the RevocationDelay field from the HTLC struct as
with the latest commitment transaction scheme, it is no longer needed.
This is due to the fact the the delay is now observed when an on-chain
HTLC claim is attempted, rather than from Shane the HTLC itself has
been broadcast.
This commit adds a new method to the OpenChannel struct:
InsertNextRevocation. This functionality is required in order to
properly implement the new funding workflow and commitment transaction
state machine.
In the prior state machine, we utilized the “initial revocation window”
which would provide both sides with the necessary revocation keys that
they needed to create new commitment states. The version of the
commitment state machine within the specification has been simplified a
bit, and instead essentially incorporates a revocation window of 1. The
window is initially filled at the _opening_ of the commitment
transaction, then never extended, only “slid down” from there on.
This commit modifies the HTLC struct within the channeldb struct in
order to accommodate the additional storage that’s now required as a
result of the change to 2-of-2 HTLC covenant transactions within the
commitment transaction.
This commit is the next phase in a rather major overhaul of the
OpenChannel struct. With this commit we remove the old ours vs theirs
semantics with channel related state variables, and instead encapsulate
all local vs remote state into a ChannelConfig structure for each party
within the channel.
This commit introduces two new structs to the channeled package:
ChannelConfig and ChannelConstraints.
The ChannelConfig struct will eliminate many of the fields on the
OpenChannel struct (ours vs theirs), in favor of having a single config
for both sides (local and remote).
The ChannelConstraints struct will house the channel flow control and
HTLC policies for both sides. The fields of this struct will be used to
control the flow of HTLC’s within the channel. In the occasion that any
of these constraints are violated, either the connection, or the
channel entirely is to be failed.
This commit adds the possibility for the initiator of a
channel to send the update_fee message, as specified
in BOLT#2. After the message is sent and both parties
have committed to the updated fee, all new commitment
messages in the channel will use the specified fee.
This commit modifies the OpenChannel struct to include the full short
channel ID rather than simply the opening height. This new field will
be needed by an upcoming change to uniformly switch to using short
channel ID’s when forwarding HTLC’s due to the change in per-hop
payloads.
This commit expands the field within the OpenChannel struct in order to
start tracking the height that the funding transaction was initially
broadcast. Other sub-systems within lnd can now use this data to give a
more accurate height hint to the ChainNotifier, or to use during the
funding workflow to decide if a channel should be forgotten after it
fails to confirm for N blocks.
This commit modifies the name of a field in the OpenChannel struct to
better reflect its actual usage within this protocol. The FeePerKw
represents the amount of satoshi to be paid as fees per kilo-weight.
This field is set at the opening of a transaction and will be able to
be updated properly via the usage of the update_fee method.
In order to cleanly handle shutdowns and restarts during state machine operation, the fee for the current
commitment transaction must be persisted. This allows the fee to be
reapplied when the current state is reloaded.
This commit adds the total time locked balance of a closed channel at
closure time to the CloseChannelSummary struct. With this information,
we now provide sub-systems within the daemon further knowledge which
can aide them in properly handling querying for the state of pending
close transactions, or if they should watch a channel for closure or
not.
This commit modifies the OpenChannel structure on-disk to also track
that opening height of a channel. This change is being made in order to
make and more light client friendly. A follow up commit will modify
several areas of the codebase to use this new functionality.
This commit expands the close summaries within the database to include
additional information such as: how the channel was closed, if the
channel if fully closed, and other balance information. This new
information will be used in an upcoming expansion of the
pendingChannels RPC to display more comprehensive data w.r.t where the
node’s funds are and how soon they’ll be fully available.
The data stored in the closeChannel bucket has been expanded to include
the above. Additionally the CloseChannel method on the OpenChannel
struct now takes in the additional information that details how and who
it was closed by.
This commit borrows heavily from the initial work in #179 by @halseth.
This commit adds a new method to the channel state: RevocationLogTail.
This new method will return the information concerning the latest
revoked state of the remote party’s commitment chain.
This new data can be used to properly initialize the states of the
in-memory commitment chains on node start up.
When a pending channel is persisted and then reloaded upon system startup
it's necessary to also persist the number of confirmations that will be required
before the pending channel can be opened.
In order to facilitate persistence during the funding process, added
the isPending flag to channels so that when the daemon restarts, we can
properly re-initialize the chain notifier and update the state of
channels that were going through the funding process.
In this commit the initial implementation of revocation hash
generation 'elkrem' was replaced with 'shachain' Rusty Russel
implementation which currently enshrined in the spec. This alghoritm has
the same asymptotic characteristics but has more complex scheme
to determine wish hash we can drop and what needs to be stored
in order to be able to achive full compression.
Fix SetStateNumHint and GetStateNumHint to properly
set and get the stateNumHints using the lower 24 bits
of the locktime of the commitment transaction as the
lower 24 bits of the obfuscated state number and the
lower 24 bits of the sequence field as the higher 24
bits.
This commit fixes a bug that was introduced when we moved to using
64-bit integers for storing the revocation log state. When we made this
change, we forgot to increase the size of the buffer which stores the
key for the particular channel state from 40 to 44 bytes to account for
the 4 additional bytes in the new 64-bit integer.
This bug has been fixed by properly sizing the key buffer. We’ve also
added an additional test to ensure that we retrieve the proper state
after multiple state updates.
This commit modifies the running update count within all ChannelDelta’s
to track the number of updates using a uint64 rather than a uint32.
This change reflects the fact that the obsfucated commitment hints are
to be encoded using a 48-bit integer, rather than a 32-bit integer.
This commit fixes a bug which would previously lead to corruption of
the channel state when a node had one or more channels open and one of
them was closed either forcibly or cooperatively. The source of the bug
itself as a typo: rather than using the construed `deliveryKey`
variable to fetch/put/delete the delivery scripts, `deliveryScriptsKey`
(the key prefix itself) as used. This bug would cause the database to
be unable to read _any_ channel from the database after one was
deleted, as each channel would actually be reading/writing-to the
_exact same_ delivery script.
The fix for the bug itself is simple: eliminate the typo.
This commit addresses some lingering TODO’s which ensure that related
state to a channel is properly deleted by the CloseChannel method.
Previously the values for the respective dust-limits of either side,
the on-disk HTLC’s, and any entries the revocation log for the channel
weren’t being properly deleted.
Additionally, we now modify the checks within the unit tests to ensure
that we can still read the channel from disk w/o running into an error
(thought the slice will be blank), and also the the revocation log is
properly garbage collected.
This commit makes a large number of minor changes concerning API usage
within the deamon to match the latest version on the upstream btcsuite
libraries.
The major changes are the switch from wire.ShaHash to chainhash.Hash,
and that wire.NewMsgTx() now takes a paramter indicating the version of
the transaction to be created.
This commit adds to new functions to the ChannelGraph struct which
allow the callers to query for the existence or non-existence of a
vertex (node) or edge (channel) within the graph. In addition to
returning whether the edge exists, the functions will also return the
last time the state has been modified for the edge or vertex. This will
allow callers to ensure that only the most up to date state is
committed to disk.
This commit adds an additional check within CloseChannel to ensure that
sub-systems attempting to delete the channel one after the other (in
the event of any sort of closure) doesn’t result in an extraneous
error.
To fix this, we now check if the channel exists before attempting a
deletion. If the channel doesn’t exist, then we simply exit early with
a nil error.
This commit adds a new method to the `OpenChannel` struct:
CommitmentHeight(). This method allows multiple callers holding the
same instance of an OpenChannel struct tied to the same on-disk channel
to consistently query the current commitment height for a channel. Such
a modification will prove useful later as sections of the code-base are
separated in order to allow more vigilant watching of channel breaches.
This commit modifies the composition of the boltdb pointer within the
DB struct to use embedding.
The rationale for this change is that the daemon may soon store some
semi-transient items within the database which requires us to expose
the boltdb’s transaction API. The logic for serialization of this data
will likely lie outside of the channeldb package as the items that may
be stored in the future will be specific to the current sub-systems
within the daemon and not generic channel related data.
In this commit the upgrade mechanism for database was added which makes he current schema rigid and upgradeable. Additional bucket 'metaBucket' was added which stores
key that house meta-data related to the current/version state of the database. 'createChannelDB' was modified to create this new bucket+key during initializing. Also
backup logic was added which makes a complete copy of the current database during migration process and restore the previous version of database if migration failed.
This commit adds persistence for a channel’s
TotalSatoshis[Sent|Received] fields. Also, the functions that perform
this persistence were renamed from [put|delete|fetch]ChanTotalFlow to
[put|delete|fetch]ChanAmountsTransferred.
This commit modifies the ChannelType enum to no longer use iota as
changes in the definition would cause the values to shift, breaking the
long-term stability required for persistence. Instead, we now select
values manually to indicate the particular channel type.
This commit modifies the existing OpenChannel struct slightly to store
a bool which indicates if we were the one who initially initiated the
channel or not. This information is relevant as in the current draft of
the specification, much of the fee related negotiation is contingent on
who initiated the channel.
This commit removes all prior fee tracking attributes along with the
persistence code from OpenChannel. The rationale is that fees actually
don’t exist at the channel level, and instead should be tracked at
higher level of abstraction as fees come from the inbound/outbound
satoshi spread.
This commit modifies the OpenChannel struct along with all related
persistent to additional store a single byte which indicates the exact
type of the channel. This may be useful in the future as higher level
behavior may change depending on the precise type of the channel.
This commit adds a new method ‘FullSyncWithAddr’ which is identical to
the existing ‘FullSync’ method other than it also creates an
association from the channel to a LinkNode object within the database.
This new method is required in order to create persistent links between
channels and link nodes which will later allow the development of
heuristics which decided when it “makes sense” to close a channel due
to inactivity. Additionally, this new association will allow for a
sub-system within the daemon to attempt to establish persistent
connections out to all LinkNodes in order to strive for channel
availability.
This commit slightly modifies the existing structure of the channeldb
scheme to replace the former concept of a “nodeID” with simply the
compressed public key of the remote node. This change paves the way for
adding useful indexes mapping a node to all it’s active channels and
the other way around.
Additionally, the current channeldb code was written before it was
agreed by many of those implementing Lightning that a node’s ID will
simply be its compressed public key.
This commit consists of a mass variable renaming to call the pkScript being executed for segwit outputs the `witnessScript` instead of `redeemScript`. The latter naming convention is generally considered to be reserved for the context of BIP 16 execution. With segwit to be deployed soon, we should be using the correct terminology uniformly through the codebase.
In addition some minor typos throughout the codebase has been fixed.
This commit introduces the concept of a manually initiated “force”
closer within the channel state machine. A force closure is a closure
initiated by a local subsystem which broadcasts the current commitment
state directly on-chain rather than attempting to cooperatively
negotiate a closure with the remote party.
A force closure returns a ForceCloseSummary which includes all the
details required for claiming all rightfully owned outputs within the
broadcast commitment transaction.
Additionally two new publicly exported channels are introduced, one
which is closed due a locally initiated force closure, and the other
which is closed once we detect that the remote party has executed a
unilateral closure by broadcasting their version of the commitment
transaction.
This commit splits the previously added RecordChannelDelta method into
two distinct methods: UpdateCommitment and AppendToRevocationLog. The
former method is to be used once the local party revokes their current
commitment, and the latter method to be used once the remote party
revokes their current commitment.
With the addition of the UpdateCommitment method, the active HTLC’s
from the local node’s point of view are now persisted to disk.
Snapshots returned by the channel now also includes the current set of
active HTLC’s. In order to maintain thread safety the channels mutex is
now grabbed within methods which modify/read state but don’t do so
solely via a boltDB transaction.
The tests have been updated to account for the storage of HTLC’s needed
in order to assert proper behavior.
This commit removes the revocation hash/keys from the channel deltas.
In the case of an uncooperative closure, we can efficiently re-generate
the proper elkrem pre-image so this storage was completely unnecessary
This commit implements a state update log which is intended the record
the relevant information for each state transition on disk. For each
state transition a delta should be written recording the new state. A
new method is also provided which is able to retrieve a previous
channel state based on a state update #.
At the moment no measures has been taken to optimize the space
utilization of each update on disk. There are several low-hanging
fruits which can be addressed at a later point. Ultimately the update
log itself should be implemented with an append-only flat file at the
storage level. In any case, the high level abstraction should be able
to maintained independent of differences in the on-disk format itself.
This commit removes the storage+encryption of private keys within
channeldb. We no longer need to encrypt these secrets before storing as
the base wallet is now expected to retain full control of these secrets
rather than the database.
As a result, we now only store public keys within the database.
This commit removes the EncryptorDecryptor interface, and all related
usage within channeldb. This interface is no longer needed as wallet
specific secrets such as private keys are no longer stored within the
database.
This commit fixes a bug caused by overriding the prefix key for storing
commitment keys with the first few bytes of a channel’s channel point.
Once a channel was deleted, then all future channels would result in a
panic due to a nil pointer deference since the prefix key was mutated,
causing all future stores/gets to fail.
We now also store their current revocation hash which is given to us
along with the revocation key once an initial HTLC is added to a
commitment transaction.
This commit updates the stored on-disk channel state to store a current
revocation key rather than a revocation hash. This change coincides
with the new commitment transaction format which uses revocation keys
rather than hashes.
Additionally, this commit updates the decoding/encoding of local+remote
elkrem trees to the latest changes in the elkrem API.
This commit introduces the concept of “closing” an already active
channel. Closing a channel causes all the channel state to be purged
from the database, and also triggers the creation of a small “summary”
kept concerning details of the previously open channel.
This commit also updates the previous test case(s), and includes the
close channel bucket in the database deletion in the .Wipe() method.
This commit overhauls the current schema for storing active channels in
order to support tracking+updating multiple open channels for a
particular peer.
Channels are now uniquely identified by an output (txid:index) rather
than an arbitrary hash value. As a result, the funding transaction is
no longer stored, as only the txin is required to lookup the original
transaction, and to sign for new commitment states.
A new bucket, nested within the bucket for a node’s Lightning ID has
been created. This new bucket acts as an index to the active channels
for a particular peer by storing all the active channel points as keys
within the bucket. This bucket can then be scanned in a linear fashion,
or queried randomly in order to retrieve channel information.
The split between top-level, and channel-level keys remains the same.
The primary modification comes in using the channel ID (the funding
outpoint) as the key suffix for all top-level and channel-level keys.
The state of OpenChannel on disk has now been partitioned into several
buckets+keys within the db. At the top level, a set of prefixed keys
stored common data updated frequently (with every channel update).
These fields are stored at the top level in order to facilities prefix
scans, and to avoid read/write amplification due to
serialization/deserialization with each read/write.
Within the active channel bucket, a nested bucket keyed on the node’s
ID stores the remainder of the channel.
Additionally OpenChannel now uses elkrem rather than shachain, delivery
scripts instead of addresses, stores the total net fees, and splits the
csv delay into the remote vs local node’s.
Several TODO’s have been left lingering, to be visited in the near
future.
* Initial draft of brain dump of chandler. Nothing yet set in stone.
* Will most likely move the storage of all structs to a more “column”
oriented approach. Such that, small updates like incrementing the total
satoshi sent don’t result in the entire struct being serialized and
written.
* Some skeleton structs for other possible data we might want to store
are also included.
* Seem valuable to record as much data as possible for record keeping,
visualization, debugging, etc. Will need to set up a time+space+dirty
cache to ensure performance isn’t impacted too much.
