The pending state definitin in ChannelCloseSummary was slightly changed
in such a way that channels that has had their commitment broadcasted
now is no longer considered "pending close". They now instead stay in
the open chan bucket with the ChanStatus "CommitmentBroadcasted" until
their commitment is confirmed. This commit updates the IsPending godoc
to reflect this.
In this commit, we modify the existing updateChanBucket function to no
longer auto-create buckets if they don't exist. We do this in order to
fix a class of bug that could arise wherein after a channel has actually
be closed (and the parent buckets removed) a method that mutates the
channel state is called, which then re-creates the relevant set of
buckets. As a result, subsequent calls to any RPCs which need to read
all the channels will fail as most of the fields won't actually be
populated.
After this commit, the fullSync method is the only one that's able to
create the full bucket hierarchy.
In this commit, we extend the CloseChannelSummary by also storing: the
current unrevoked revocation for the remote party, the next pending
unused revocation, and also the local channel config. We move to store
these as the provide an extra level of defense against bugs as we'll
always store information required to derive keys for any current and
prior states.
This commit adds a new method FetchWaitingCloseChannels to the database,
used for fetching OpenChannels that have a ChanStatus != Default. These
are channels that are borked, or have had a commitment broadcasted, and
is now waiting for it to confirm.
The fetchChannels method is rewritten to return channels exclusively
based on wheter they are pending or waitingClose.
This commit changes the bool `IsBorked` in OpenChannel to a `ChanStatus`
struct, of type ChannelStatus. This is used to indicated that a channel
that is technically still open, is either borked, or has had a
commitment broadcasted, but is not confirmed on-chain yet.
The ChannelStatus type has the value 1 for the status Borked, meaning it
is backwards compatible with the old database format.
Modifies TestFetchPendingChannels to verify that calls to
MarkAsOpen also modify the in-memory state. Previously we
only tested the persistent state loaded immediately after.
Modifies the MarkAsOpen operation to also update the
ShortChanID and IsPending fields in-memory. Before,
only the on-disk representation was updated, which
may have lead to stale data channel states being
passed in-memory.
In this commit, we remove references to raw keys from the main
ChannelConfig struct and instead replace it with usage of
keychain.KeyDescriptor. We do this, as the ChannelConfig as it stands
is a near complete static description of a channel. In the future, it
will be possible to export these static descriptions as backups. We
prefer the KeyDescriptor of a plain PublicKey, as the KeyLocator
portion of the struct allows a stateless signer to re-derive the keys
as needed when signing.
In this commit, we add a new storage namespace to channeldb: the
ForwardingLog. This log will be used by higher level sub-systems to log
each successfully completed HTLC. Each payment circuit will be
summarized as a “ForwardingEvent”. A series of events can then be
queried via a time slice query. In a time slice query, the caller
specifies a time range, a number of events to skip, and the max number
of events to return. Each query will return the index of the final
item. As we have a max number of events we’ll return in a response,
callers may need to use this last offset index to seek further by
skipping that number of entries. Combining these fields, callers are
able to query the time series, skipping an arbitrary amount of events,
and capping the max number of returned events.
In this commit, we made a series of modification to the way we handle
reading edges and vertexes from disk, in order to reduce the amount of
garbage generated:
1. Properly use PubKeyBytes are required rather than PubKey()
2. Return direct structs rather than pointers, and leave it to the
runtime to perform escape analysis.
3. In-line the former readSig() method when reading sigs from disk.
This commit changes the definition of the
constraints in the ChannelConstraints struct
to specify that these are all constraints the
*owner* of the set of constraints must *never
violate*.
This is done to make it easier to check that
a particular node is not violating any
constraint for a gien update, as before it
could violate constraints found both in its
local and the remote contraints.
In this commit, we make an API change that’s meant to reduce the amount
of garbage we generate when doing pathfinding or syncing nodes with our
latest graph state. Before this commit, we would always have to fully
decode the public key and signatures when reading a edge or vertex
struct. For the edges, we may need several EC operations to fully
decode all the pubkeys. This has been seen to generate a ton of
garbage, as well as slow down path finding a good bit.
To remedy this, we’ll now only ever read the *raw* bytes from disk. In
the event that we actually need to verify a signature (or w/e), only
*then* will we fully decode everything.
This commit adds Tor support. Users can set the --TorSocks flag
to specify which port Tor's SOCKS5 proxy is listening on so that
lnd can connect to it. When this flag is set, ALL traffic gets
routed over Tor including DNS traffic. Special functions for
DNS lookups were added, and since Tor doesn't natively support
SRV requests, the proxySRV function routes connects us to
a DNS server via Tor and SRV requests can be issued directly
to the DNS server.
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
Before this commit, we’d unnecessarily use a write transaction within
the FetchChannelEdgesByOutpoint. This is wasteful as the function only
actually reads items from the database, and doesn’t attempt any
mutations at all.
Fixes#481.
Prior to this commit, payments stored in the channel DB only kept a
record of the payment hash. This is a problem as the preimage is what
serves as proof of payment and a user should be able to look up this
value in the future (not just immediately after payment).
Instead of storing both the payment hash and the preimage, we store the
preimage only since the hash can be derrived from this using a SHA256.
In the RPC listpayments command, we now give the preimage in addition to
the payment hash.
This commit adds a FetchClosedChannel method to the
channeldb, which allows querying based on a known
channel point. This will be used in the nursery to
load channel close summaries, which can be used to
provide more accurate height hints when recovering
from failures.
In this commit, we add the WitnessCache sub-storage system of the
greater database. The WitnessCache is a persistent cache of all
witnesses we’ve encountered on the network. We’ll use this cache to
share any on-chain discoveries between active channels. Eventually
we’ll also use this to enforce the variant that a preimage is only to
be used ONCE on the network.
This commit adds the ChannelFlags field, of type
lnwire.FundingFlags, to the OpenChannel struct,
including serialization for database storage.
This is done to preserve the flags that were
sent during channel opening, currently used
to determine whether a channel should be made
public or not after opening.
In this commit, we fix an existing bug that arose due to incorrectly
crafting the key we use to store channel commitments. Before this
commit, we tried to copy to a slice that hadn’t been allocated yet. As
a result, the key would only have the 0x00 or 0x01 as its value. We fix
this by properly crafting the key using the built-in append function.
In this commit, we fix an existing bug wherein if we closed two
channels, then we were unable to read the channel state afterwards as
we deleted the enclosing bucket.
In this commit, we fix an existing bug wherein we failed to update the
channels state once we accepted a new commitment. As a result, after a
state transition, if the channel state was read from disk, values like
TotalMSatSent wouldn’t be properly updated.
In this commit we’ve extended the TestChannelStateTransition method to
exercise the new state transition related messages. This includes
ensuring that when we add a new dangling commitment, and then the
remote party revokes it, then the on-disk state is update accordingly.
In this commit, we update the CloseChannel method to respect the new
on-disk bucket based structure. Additionally, we now ensure that we
delete the new chainBucket.
In this commit, in addition to the renaming we’ve modified the behavior
of AdvanceCommitChainTail as follows: this method now will simply
atomically advance the commitment tail, set the new commitment to the
prior dangling commitment, and update the on-disk revocation log.
The macho expects the new revocation state to already be stored within
the channel. This method is to be called once the remote party revokes
their current commitment state.
In this commit, we add a new method: RemoteCommitChainTip. This method
allows callers to poll the database state to check if we have an
un-acked commitment for the remote party. If so, then it should be
retransmitted once a communication channel has been re-established with
the channel peer. This method will return ErrNoPendingCommit if we
don’t currently have a dangling commitment.
In this commit, we add a new method AppendRemoteCommitChain. This
method is meant to be used once we extend a new state to the remote
party, but before we actually transmit the CommitSig message. With this
method, we store a fully valid CommitDiff on disk which can be used in
the case that we need to retransmit the state to the party as they
didn’t fully receive it.
In this commit we finalized the structure of the CommitDiff struct by
adding a set of LogUpdates, and also a valid CommitSig message.
The LogUpdate struct houses a messages that were transmitted and
locked-in with the new commitment state. We include the LogIndex along
with the wire messages itself in order to be able to properly
reconstruct the update log upon restart.
The CommitSig message included should be transmitted after the set of
LogUpdates, and fully covers the new commitment state and any new (or
already present) HTLC’s that were included as part of the state.
In this commit, we modify the UpdateCommitment method to accept a full
ChannelCommitment rather than a new transaction, the sig, and a
ChannelDelta. This new structure of this method also takes advantage of
the new bucket structure of the storage schema. Additionally, this
method will now atomically swap in the new passed commitment to point
to the LocalCommitment value within the struct.
In this commit we add a new MarkAsOpen method to the OpenChannel
struct. This method replaces the existing MarkChannelAsOpen method
which targeted the database struct itself.
In this commit we comptely overhaul the existing storage of the
OpenChannel struct to use the new common serialization defined within
the codec.go file. Additionally, we’ve modified the structure of the
channel database on disk. Rather then use the existing prefix based
segmentation, everything is now bucket based. This has resulted in much
simpler and easier to follow code. The bucket progression is:
openChannelBucket -> nodeBucket -> chainBucket -> channelBucket. We add
a chainBucket as it’s possible that in the future we may have several
channels on distinct chains with a given node.
With the above changes, we’re able to delete much of the existing code
within the file, drastically reducing its size.
By adding these two fields, it is now possible to fully reconstruct the
channel’s update log from the set of HTLC’s stored on disk, as we now
properly note both the log index and HTLC index. Prior to this commit
we would simply start the new log index based on the amount of HTLC’s
that were present in the prior state.
In this commit, we restructure the OpenChannel struct to used two
distinct ChannelCommitments: one for the remote party, and one for the
local party. With this, we now partition the local state and the remote
state. Previously, we weren’t properly tracking the state of the remote
party. As a result, there were certain classes of failures we were
unable to properly recover from. With this separation, we can now
handle them, and the OpenChannel struct is slimmer and more
understandable.
In this commit we’ve added a new struct to the package:
ChannelCommitment. This sturct houses all the common data the comprises
a particular commitment state. This will soon replace the open fielded
commitmetn fields within the OpenChannel struct.
In this commit, we add a new files to the channeldb package: codec.go.
This file is similar to the way that serialization is done within the
lnwire package. The goal of this file is to reduce all the duplication
within the common serialization methods across the package.
In this commit htlc channeldb representation have been augmented
with onion blob field, and (de)serialisaion functions have been changed
to make the onion blob persistant.
This commit adds the method DisconnectBlockAtHeight to the channel
graph database, making it possible to "rewind" the database in case
a block is disconnected from the main chain. To accomplish this,
a prune log is introduced, making it possible to keep track of the
point in time where the database was pruned. This is necessary for
the case where lnd might wake up on a stale branch, and must "walk
backwards" on this branch after it finds a common block fro the
graph database and the new main chain.
This commit modifies the recently modified logic for self-channel
retransmission to exclude pruning *our* channels which haven’t been
updated since the broadcastInterval. Instead, we only re-broadcast
channels of ours that haven’t been updated in 24 hours.
Use binary.Read/Write in functions to serialize and deserialize
channel close summary and HTLC boolean data, as well as in
methods to put and fetch channel funding info. Remove lnd
implementations of readBool and writeBool as they are no
longer needed. Also fix a few minor typos.
This modifies the tests that deal serializing the Invoice type to limit
the creation date to seconds since Go1.9 added the concept of a
monotonic component to times which does not round trip through
MarshalBinary and UnmarshalBinary and therefore causes the tests to fail.
In particular, it modifies the creation dates in the randInvoice,
makeFakePayment, makeRandomFakePayment, and TestInvoiceWorkflow
functions.
This results in allowing TestOutgoingPaymentSerialization,
TestOutgoingPaymentWorkflow, and TestInvoiceWorkflow to pass.
This commit modifies the node.ForEachChannel to *also* return the
incoming edge as well as the outgoing edge. We make this modification
as when we’re doing path finding, we need to return the incoming edge
as well, since we need to use the to properly compute the time lock and
fees for transit on that edge.
Persists the state of a channel opening process after funding
transaction is confirmed. This tracks the messages sent to
the peer such that the process can be continued in case of a
restart. Also introduces that the receiver side forgets about
channel if funding transaction is not confirmed in 48hrs.
Adds a HaveNodeAnnouncement field to the LightningNode
struct, which is used to indicate if we have gotten
all the necessary information to fill the remaining
fields in the struct. If we haven't gotten a node
announcement for this specific node, then we only
know the pubkey, and can only fill that field in
the struct. Still, we should be able to add it to the
channel graph and use it for routes, as long as we
know about channels to this node.
This commit removes the RevocationDelay field from the HTLC struct as
with the latest commitment transaction scheme, it is no longer needed.
This is due to the fact the the delay is now observed when an on-chain
HTLC claim is attempted, rather than from Shane the HTLC itself has
been broadcast.
This commit adds a new method to the OpenChannel struct:
InsertNextRevocation. This functionality is required in order to
properly implement the new funding workflow and commitment transaction
state machine.
In the prior state machine, we utilized the “initial revocation window”
which would provide both sides with the necessary revocation keys that
they needed to create new commitment states. The version of the
commitment state machine within the specification has been simplified a
bit, and instead essentially incorporates a revocation window of 1. The
window is initially filled at the _opening_ of the commitment
transaction, then never extended, only “slid down” from there on.
This commit modifies the HTLC struct within the channeldb struct in
order to accommodate the additional storage that’s now required as a
result of the change to 2-of-2 HTLC covenant transactions within the
commitment transaction.
This commit is the next phase in a rather major overhaul of the
OpenChannel struct. With this commit we remove the old ours vs theirs
semantics with channel related state variables, and instead encapsulate
all local vs remote state into a ChannelConfig structure for each party
within the channel.
This commit introduces two new structs to the channeled package:
ChannelConfig and ChannelConstraints.
The ChannelConfig struct will eliminate many of the fields on the
OpenChannel struct (ours vs theirs), in favor of having a single config
for both sides (local and remote).
The ChannelConstraints struct will house the channel flow control and
HTLC policies for both sides. The fields of this struct will be used to
control the flow of HTLC’s within the channel. In the occasion that any
of these constraints are violated, either the connection, or the
channel entirely is to be failed.
This commit removes a database migration that was added to fix a bug
introduced within the first alpha release. This migration is no longer
needed as the underlying database scheme has changed since then.
This commit adds the possibility for the initiator of a
channel to send the update_fee message, as specified
in BOLT#2. After the message is sent and both parties
have committed to the updated fee, all new commitment
messages in the channel will use the specified fee.
The btclog package has been changed to defining its own logging
interface (rather than seelog's) and provides a default implementation
for callers to use.
There are two primary advantages to the new logger implementation.
First, all log messages are created before the call returns. Compared
to seelog, this prevents data races when mutable variables are logged.
Second, the new logger does not implement any kind of artifical rate
limiting (what seelog refers to as "adaptive logging"). Log messages
are outputted as soon as possible and the application will appear to
perform much better when watching standard output.
Because log rotation is not a feature of the btclog logging
implementation, it is handled by the main package by importing a file
rotation package that provides an io.Reader interface for creating
output to a rotating file output. The rotator has been configured
with the same defaults that btcd previously used in the seelog config
(10MB file limits with maximum of 3 rolls) but now compresses newly
created roll files. Due to the high compressibility of log text, the
compressed files typically reduce to around 15-30% of the original
10MB file.
This commit modifies the OpenChannel struct to include the full short
channel ID rather than simply the opening height. This new field will
be needed by an upcoming change to uniformly switch to using short
channel ID’s when forwarding HTLC’s due to the change in per-hop
payloads.
This commit expands the field within the OpenChannel struct in order to
start tracking the height that the funding transaction was initially
broadcast. Other sub-systems within lnd can now use this data to give a
more accurate height hint to the ChainNotifier, or to use during the
funding workflow to decide if a channel should be forgotten after it
fails to confirm for N blocks.
This commit modifies the name of a field in the OpenChannel struct to
better reflect its actual usage within this protocol. The FeePerKw
represents the amount of satoshi to be paid as fees per kilo-weight.
This field is set at the opening of a transaction and will be able to
be updated properly via the usage of the update_fee method.
In order to cleanly handle shutdowns and restarts during state machine operation, the fee for the current
commitment transaction must be persisted. This allows the fee to be
reapplied when the current state is reloaded.
This commit adds the total time locked balance of a closed channel at
closure time to the CloseChannelSummary struct. With this information,
we now provide sub-systems within the daemon further knowledge which
can aide them in properly handling querying for the state of pending
close transactions, or if they should watch a channel for closure or
not.
This commit modifies the OpenChannel structure on-disk to also track
that opening height of a channel. This change is being made in order to
make and more light client friendly. A follow up commit will modify
several areas of the codebase to use this new functionality.
This commit introduces a new method to the ChannelGraph struct:
ChannelView. This struct returns all the outpoints that represent the
set of active channels within the network. The set of items returned by
this new method will possibly shrink with each call to `PruneGraph`,
and possibly be expanded by each call to `AddChannelEdge`.
The graph pruning tests have been updated to ensure the description
above holds true.
This commit adds a new method to the database which allows callers to
mark a channel as fully closed once certain conditions have been
reached. If a channel was cooperatively closed, then it can be marked
as fully closed as soon as the channel has been confirmed. If the
channel was marked as force closed, then it should be marked as closed
as soon as _all_ the funds in limbo have been swept.
This commit builds upon the prior commit by adding a new method to
allows callers to query the state of all closed (fully) and pending
closed channel within the database.
This commit expands the close summaries within the database to include
additional information such as: how the channel was closed, if the
channel if fully closed, and other balance information. This new
information will be used in an upcoming expansion of the
pendingChannels RPC to display more comprehensive data w.r.t where the
node’s funds are and how soon they’ll be fully available.
The data stored in the closeChannel bucket has been expanded to include
the above. Additionally the CloseChannel method on the OpenChannel
struct now takes in the additional information that details how and who
it was closed by.
This commit borrows heavily from the initial work in #179 by @halseth.
This commit modifies the ForEachNode on the ChannelGraph and
ForEachChannel on the LightningNode struct to accept a database
transaction as its first argument. With this change, it’ll now be
possible to implement graph traversals that typically required a nested
loop with all the vertex loaded into memory using the callback API
instead:
c.ForEachNode(nil, func(tx, node) {
node.ForEachChannel(tx, func(…) {
})
})
This commit implements an easy optimization by using bolt db’s Batch
method when writing payment details to disk. The AddPaymnent method can
be concurrently called by thousands of grouting due to the way the
payment dispatch pipeline is architected. With this commit, we shave of
a significant amount of running time when users are sending thousands
of payments a second as what would’ve been thousands of writes can now
be coalesced into one or two writes!
Originally we adding the edge without proof in order to able to use it
for payments path constrcution. This method will allow us to populate
the announcement proof after the exchange of the half proofs and
constrcutrion of full proof finished.
In case if the channel shouldn't be announced to the rest of the network
the proof, which is needed to announce the channel, will not be
populated, fot that reason the ability to store the empty proof has
been added.
Minor change to server.go to add ExternalIPs to
channeldb.LightningNode. Also, added a test that utilizes this
functionality and exercises multiple addresses in NodeAnnouncement.
This commit modifies address handling in the NodeAnnouncement struct,
switching from net.TCPAddr to []net.Addr. This enables more flexible
address handling with multiple types and multiple addresses for each
node. This commit addresses the first part of issue #131 .
This commit adds a new method to the channel state: RevocationLogTail.
This new method will return the information concerning the latest
revoked state of the remote party’s commitment chain.
This new data can be used to properly initialize the states of the
in-memory commitment chains on node start up.
This commit removes all instances of the fastsha256 library and
replaces it with the sha256 library in the standard library. This
change should see a number of performance improvements as the standard
library has highly optimized assembly instructions with use vectorized
instructions as the platform supports.
