Commit Graph

652 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
9191fbd317
channeldb: add FetchAllChannels method to return all active channels
This commit introduces a new method to channeldb: ‘FetchAllChannels’.
This method can be used to obtain the state of all active (currently
open) channels within the database. This method can be used for compute
basic channel-based metrics or exposed as an RPC in order to allow
clients to display/query channel data.
2016-10-26 14:53:26 -07:00
Olaoluwa Osuntokun
3354685292
channeldb: add FullSyncWithAddr method to OpenChannel
This commit adds a new method ‘FullSyncWithAddr’ which is identical to
the existing ‘FullSync’ method other than it also creates an
association from the channel to a LinkNode object within the database.

This new method is required in order to create persistent links between
channels and link nodes which will later allow the development of
heuristics which decided when it “makes sense” to close a channel due
to inactivity. Additionally, this new association will allow for a
sub-system within the daemon to attempt to establish persistent
connections out to all LinkNodes in order to strive for channel
availability.
2016-10-26 14:47:10 -07:00
Olaoluwa Osuntokun
f5d1785ab2
channeldb: factor out LinkNode serialization+db write into new func 2016-10-26 14:36:38 -07:00
Olaoluwa Osuntokun
b853be7b9a
channeldb: ignore duplicate addresses in LinkNode.AddAdress 2016-10-26 14:32:49 -07:00
Olaoluwa Osuntokun
cb328e65c4
channeldb: switch to using a full public key to identity channel->node
This commit slightly modifies the existing structure of the channeldb
scheme to replace the former concept of a “nodeID” with simply the
compressed public key of the remote node. This change paves the way for
adding useful indexes mapping a node to all it’s active channels and
the other way around.

Additionally, the current channeldb code was written before it was
agreed by many of those implementing Lightning that a node’s ID will
simply be its compressed public key.
2016-10-25 16:11:30 -07:00
Olaoluwa Osuntokun
d109bd9298
channeldb: add bucket for storing p2p counterparty meta-data
This commit adds a new bucket to the database which is dedicated to
storing data pertaining to p2p related reachability for direct channel
counter parties. The data stored in this new bucket can be used within
heuristics when deciding to unilaterally close a channel due to
inactivity. Additionally, all known reachable IP addresses for a
particular LinkNode are to be stored and updated within the database in
order to facilitate the establishment of persistent connections to
direct channel counter parties.
2016-10-25 16:06:32 -07:00
andrew.shvv
e515710a7d multi: use witnessScript everywhere instead of redeemScript
This commit consists of a mass variable renaming to call the pkScript being executed for segwit outputs the `witnessScript` instead of `redeemScript`. The latter naming convention is generally considered to be reserved for the context of BIP 16 execution. With segwit to be deployed soon, we should be using the correct terminology uniformly through the codebase. 

In addition some minor typos throughout the codebase has been fixed.
2016-10-15 16:02:09 -07:00
Olaoluwa Osuntokun
3247a160c5
channeldb: remove obsolete idKey methods 2016-10-13 17:31:47 -07:00
Olaoluwa Osuntokun
0b841ec686
channeldb: store optional invoice fields as variable length byte arrays
This commit modifies the on-disk storage of invoices to stop the
optional fields (memo+receipt) on-disk as variable length byte arrays.
This change saves space as the optional fields now only take up as much
space as is strictly needed, rather than always being padded out to max
size (1KB).
2016-09-23 15:15:26 -07:00
Olaoluwa Osuntokun
62b37f45f5
channeldb: include in the invoice bucket when wiping and initializing the db 2016-09-23 15:06:06 -07:00
Olaoluwa Osuntokun
d7c4074cd6
channeldb: add ability to retrieve all invoices from the database
This commit adds a new invoice related method: FetchAllInvoices. This
method allows callers to query the state of all invoices currently
stored within the database. The method takes a toggle bit which
determines if only pending (unsettled) invoices should be returned, or
if they al should be.
2016-09-21 19:48:15 -07:00
Olaoluwa Osuntokun
fa70990452
channeldb: store the invoice counter within the invoice index bucket
This commit moves the location of the invoice counter key which is an
auto-incrementing primary key for all invoices. Rather than storing the counter
in the same top-level invoice bucket, the counter is now stored within the
invoiceIndex bucket. With this change, the top-level bucket can now cleanly be
scanned in a sequential manner to retrieve all invoices.
2016-09-21 19:48:10 -07:00
Olaoluwa Osuntokun
b264ba198f
channeldb: add persistent invoice workflow
This commit adds the necessary database functionality required for a
high-level payment invoice workflow. Invoices can be added dealing the
requirements for fulfillment, looked by payment hash, and the finally
also settled by payment hash. For record keeping and the possibility of
reconciling future disputes, invoices are currently never deleted from
disk. Instead when an invoice is settled a bit is toggled indicating as
much.

The current invoiceManger within the daemon will be modified to use
this persistent invoice store, only storing certain “debug” invoices in
memory as dictated by a command line flag.
2016-09-16 17:24:55 -07:00
Olaoluwa Osuntokun
d0353b2864
lnwallet: add ability to trigger a force closure within channel state machine
This commit introduces the concept of a manually initiated “force”
closer within the channel state machine. A force closure is a closure
initiated by a  local subsystem which broadcasts the current commitment
state directly on-chain rather than attempting to cooperatively
negotiate a closure with the remote party.

A force closure returns a ForceCloseSummary which includes all the
details required for claiming all rightfully owned outputs within the
broadcast commitment transaction.

Additionally two new publicly exported channels are introduced, one
which is closed due a locally initiated force closure, and the other
which is closed once we detect that the remote party has executed a
unilateral closure by broadcasting their version of the commitment
transaction.
2016-09-12 19:07:35 -07:00
Olaoluwa Osuntokun
e60778a867
channeldb: split RecordChannelDelta into two distinct methods
This commit splits the previously added RecordChannelDelta method into
two distinct methods: UpdateCommitment and AppendToRevocationLog. The
former method is to be used once the local party revokes their current
commitment, and the latter method to be used once the remote party
revokes their current commitment.

With the addition of the UpdateCommitment method, the active HTLC’s
from the local node’s point of view are now persisted to disk.
Snapshots returned by the channel now also includes the current set of
active HTLC’s. In order to maintain thread safety the channels mutex is
now grabbed within methods which modify/read state but don’t do so
solely via a boltDB transaction.

The tests have been updated to account for the storage of HTLC’s needed
in order to assert proper behavior.
2016-09-12 19:07:00 -07:00
Olaoluwa Osuntokun
2235785ed8
channeldb: add some missing nil checks 2016-09-12 19:06:57 -07:00
Olaoluwa Osuntokun
e83a6a0f6c
channeldb: remove revocation keys from channel deltas
This commit removes the revocation hash/keys from the channel deltas.
In the case of an uncooperative closure, we can efficiently re-generate
the proper elkrem pre-image so this storage was completely unnecessary
2016-09-12 19:06:52 -07:00
Olaoluwa Osuntokun
c1b98da530
channeldb: also pass in current tx and sig into RecordChannelDelta 2016-09-12 19:06:49 -07:00
Olaoluwa Osuntokun
6684f6aedf
channeldb: implement commitment state update log
This commit implements a state update log which is intended the record
the relevant information for each state transition on disk. For each
state transition a delta should be written recording the new state. A
new method is also provided which is able to retrieve a previous
channel state based on a state update #.

At the moment no measures has been taken to optimize the space
utilization of each update on disk. There are several low-hanging
fruits which can be addressed at a later point. Ultimately the update
log itself should be implemented with an append-only flat file at the
storage level. In any case, the high level abstraction should be able
to maintained independent of differences in the on-disk format itself.
2016-09-12 19:06:44 -07:00
Olaoluwa Osuntokun
bdb6566076
channeldb: only store public keys for multi-sig + commitment keys
This commit removes the storage+encryption of private keys within
channeldb. We no longer need to encrypt these secrets before storing as
the base wallet is now expected to retain full control of these secrets
rather than the database.

As a result, we now only store public keys within the database.
2016-09-08 12:25:12 -07:00
Olaoluwa Osuntokun
dc00514c42 channeldb: remove EncryptorDecryptor interface
This commit removes the EncryptorDecryptor interface, and all related
usage within channeldb. This interface is no longer needed as wallet
specific secrets such as private keys are no longer stored within the
database.
2016-09-08 12:22:12 -07:00
Olaoluwa Osuntokun
504c8bf5f3
channeldb: bucket not found during .Wipe() is no longer an error
This commit changes the current behavior around channeldb.Wipe().
Previously if a channel had never been closed, and a wipe was
attempted, then wipe operation would fail and the transaction would be
rolled back.

This commit fixes this behavior by checking for bolt.ErrBucketNotFound
error, and handling the specific error as a noop.
2016-07-21 16:16:28 -07:00
Olaoluwa Osuntokun
56fb414767
channeldb: add method to sync revocation state to disk 2016-07-12 17:26:53 -07:00
Olaoluwa Osuntokun
48491a7fee
channeldb: fix commit key state corruption bug
This commit fixes a bug caused by overriding the prefix key for storing
commitment keys with the first few bytes of a channel’s channel point.

Once a channel was deleted, then all future channels would result in a
panic due to a nil pointer deference since the prefix key was mutated,
causing all future stores/gets to fail.
2016-07-12 17:00:08 -07:00
Olaoluwa Osuntokun
4590364d04
channeldb: return nil for error if openChanBucket not created
This avoids an unnecessary panic in the case that the channeldb has
been wiped independently while a new peer connects.
2016-07-09 16:21:32 -07:00
Olaoluwa Osuntokun
1f7d9b6425
channeldb: also store their current revocation hash
We now also store their current revocation hash which is given to us
along with the revocation key once an initial HTLC is added to a
commitment transaction.
2016-07-05 16:48:31 -07:00
Olaoluwa Osuntokun
06af4b130f
channeldb: remove theirCommitTx, store latest commit sig 2016-07-05 16:44:52 -07:00
Olaoluwa Osuntokun
582b83ada3
channeldb: update channel state to revocation keys + new elkrem API
This commit updates the stored on-disk channel state to store a current
revocation key rather than a revocation hash. This change coincides
with the new commitment transaction format which uses revocation keys
rather than hashes.

Additionally, this commit updates the decoding/encoding of local+remote
elkrem trees to the latest changes in the elkrem API.
2016-06-30 11:40:12 -07:00
Olaoluwa Osuntokun
1a48db3039
channeldb: stored un-revoked hashes are now 32 bytes 2016-06-26 22:59:02 -07:00
Olaoluwa Osuntokun
b88b2d4c91
channeldb: implement snapshots for active channels 2016-06-22 22:05:53 -07:00
Olaoluwa Osuntokun
07bc7bbd42
channeldb: sent internal db pointer in FetchOpenChannels
This commit fixes a bug which would potentially cause a panic if a
channel returned from FetchOpenChannels attempted to access the
internal pointer to the database.

To fix this bug, the pointer is now properly set once the channel has
been loaded from the database.
2016-06-22 22:01:32 -07:00
Olaoluwa Osuntokun
210c32d890
channeldb: add doc strings to finalize funcs/structs 2016-06-22 17:15:52 -07:00
Olaoluwa Osuntokun
f1f27b2046
channeldb: add ability to close active channels
This commit introduces the concept of “closing” an already active
channel. Closing a channel causes all the channel state to be purged
from the database, and also triggers the creation of a small “summary”
kept concerning details of the previously open channel.

This commit also updates the previous test case(s), and includes the
close channel bucket in the database deletion in the .Wipe() method.
2016-06-22 16:16:43 -07:00
Olaoluwa Osuntokun
c5f97a17d5
channeldb: introduce FetchOpenChannels
This commit introduces/re-writes a method FetchOpenChannels for
channeldb, which returns all currently active/open channels for a
particular peerID.
2016-06-21 13:12:47 -07:00
Olaoluwa Osuntokun
c2818a549b
channeldb: modify schema to multiple-channels-per-peer
This commit overhauls the current schema for storing active channels in
order to support tracking+updating multiple open channels for a
particular peer.

Channels are now uniquely identified by an output (txid:index) rather
than an arbitrary hash value. As a result, the funding transaction is
no longer stored, as only the txin is required to lookup the original
transaction, and to sign for new commitment states.

A new bucket, nested within the bucket for a node’s Lightning ID has
been created. This new bucket acts as an index to the active channels
for a particular peer by storing all the active channel points as keys
within the bucket. This bucket can then be scanned in a linear fashion,
or queried randomly in order to retrieve channel information.

The split between top-level, and channel-level keys remains the same.
The primary modification comes in using the channel ID (the funding
outpoint) as the key suffix for all top-level and channel-level keys.
2016-06-21 13:12:43 -07:00
Olaoluwa Osuntokun
fcff17c336
multi: change all imports to roasbeef's forks
This commit will allow the general public to build lnd without jumping
through hoops setting up their local git branches nicely with all of
our forks.
2016-05-15 17:22:37 +03:00
Olaoluwa Osuntokun
20f5ff56eb channeldb: fix imports 2016-05-03 20:02:46 -07:00
Tadge Dryja
71231c477c make imports lowercase 2016-04-30 20:24:57 -07:00
Olaoluwa Osuntokun
fa1e7a332f channeldb: remove hardcoded netparams 2016-04-24 12:35:52 -07:00
Olaoluwa Osuntokun
3f5664b46d channeldb: unify Create/Open into simply Open
Open will now create, and initialize the db if it does not yet exist.
2016-03-24 14:31:46 -07:00
Olaoluwa Osuntokun
afb45ffe8b channeldb: temporary hack...
Need to add a channeldb.NewOpenChannel() method to once again allow a
channel’s db pointer to be private.
2016-03-24 00:00:22 -07:00
Olaoluwa Osuntokun
87603e780f channeldb: re-design OpenChannel schema, update tests
The state of OpenChannel on disk has now been partitioned into several
buckets+keys within the db. At the top level, a set of prefixed keys
stored common data updated frequently (with every channel update).
These fields are stored at the top level in order to facilities prefix
scans, and to avoid read/write amplification due to
serialization/deserialization with each read/write.

Within the active channel bucket, a nested bucket keyed on the node’s
ID stores the remainder of the channel.

Additionally OpenChannel now uses elkrem rather than shachain, delivery
scripts instead of addresses, stores the total net fees, and splits the
csv delay into the remote vs local node’s.

Several TODO’s have been left lingering, to be visited in the near
future.
2016-03-23 22:39:52 -07:00
Olaoluwa Osuntokun
631e76519e channeldb: switch to Open/Create methods rather than New
Commit includes basic tests for Open/Create. Additionally, rather than
relying on btcwallet’s addmgr for encryption/decryption, this package
now exposes a simple crypto system interface.
2016-03-23 22:11:57 -07:00
Olaoluwa Osuntokun
6155fd5d01 channeldb: move node ID/addr funds to node.go 2016-03-22 18:46:54 -07:00
Olaoluwa Osuntokun
3e3948a04f channeldb: switch to bolt.DB instead of walletdb.DB
This decouples channeldb from btcwallet, and also allows us access to
bolt’s Batch() call.
2016-03-22 18:46:30 -07:00
Olaoluwa Osuntokun
8feb86c4aa multi: create lnd's logging infrastructure 2016-03-22 18:43:10 -07:00
Tadge Dryja
2815afebb7 update imports to github 2016-01-16 10:45:54 -08:00
Olaoluwa Osuntokun
c4af4017da channeldb: revocation hashes are 20 bytes
* Although, why not 32?
2016-01-14 23:56:08 -08:00
Tadge Dryja
8504362c5e trying to store id pkh. Doesn't work; wallet creation never completes 2016-01-14 23:56:08 -08:00
Tadge Dryja
e70c6aa367 trying to store private identity key. Doesn't work. 2016-01-14 23:56:08 -08:00
Olaoluwa Osuntokun
1b9d34f614 channeldb: don't store commit sig separately, expected to be within the txin of the commit tx 2015-12-29 00:00:29 -06:00
Olaoluwa Osuntokun
4fdb2763e6 channeldb: create new channeldb package, update lnwallet to use new API
* Initial draft of brain dump of chandler. Nothing yet set in stone.
* Will most likely move the storage of all structs to a more “column”
oriented approach. Such that, small updates like incrementing the total
satoshi sent don’t result in the entire struct being serialized and
written.
* Some skeleton structs for other possible data we might want to store
are also included.
* Seem valuable to record as much data as possible for record keeping,
visualization, debugging, etc. Will need to set up a time+space+dirty
cache to ensure performance isn’t impacted too much.
2015-12-26 12:35:15 -06:00