Commit Graph

745 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
480ec3bbca
channeldb: ensure restored channels can't be mutated 2019-01-28 20:24:48 -08:00
Olaoluwa Osuntokun
b2b57314fa
channeldb: add new RestoreChannelShells method
In this commit, we add a new type (ChannelShell) along with a new
method, RestoreChannelShells which allows a caller to insert a series of
channel shells into the database. These channel shells will allow a
restored node to initiate the DLP protocol and recover their set of
existing channels.

When we insert a channel shell, we re-create the original link node, and
also add the outgoing edge to the channel graph. This way we can be sure
that upon start up, we attempt to connect to the remote peers, and that
the normal graph query commands will operate as expected.
2019-01-28 20:24:41 -08:00
Olaoluwa Osuntokun
fa30af0475
channeldb: when fetching/inserting commits check for ChanStatusRestored
If the ChanStatusRestored flag is set, then we don't need to write or
read the set of commits for a channel as they won't exist. This will be
the case when we restore a channel from an SCB.
2019-01-28 20:24:40 -08:00
Olaoluwa Osuntokun
c656788b0b
channeldb: add new methods to allow adding a new edge+policy w/ existing db transaction 2019-01-28 20:24:38 -08:00
Olaoluwa Osuntokun
11c6887ffa
channeldb: refactor syncPending to expose new syncNewChannel function
The new syncNewChannel function will allow callers to insert a new
channel given the OpenChannel struct, and set of addresses for the
channel peer. This new method will also create a new LinkNode for the
peer if one doesn't already exist.
2019-01-28 20:24:37 -08:00
Olaoluwa Osuntokun
7a88f580ea
channeldb: add new FetchChannelMethod
In this commit, we add a new method, FetchChannel which is required in
order to implement the new chanbackup.LiveChannelSource interface.
2019-01-28 20:24:36 -08:00
Olaoluwa Osuntokun
aaf6456e12
channeldb: add HasChanStatus and ApplyChanState methods to OpenChannel
These methods allow callers to properly query if a channel is in a
particular flag, and also modify the channel status in a thread safe
manner.
2019-01-28 20:24:35 -08:00
Olaoluwa Osuntokun
a410262dda
channeldb: modify the String() method of ChannelStatus reflect all flags
In this commit, we modify the String() method of the ChannelStatus type
to reflect the fact that it's a flag set. With these new changes, we'll
now print the variable name of each assigned bit with a bar delimiting
them all.
2019-01-28 20:24:33 -08:00
Olaoluwa Osuntokun
f57f40e767
channeldb: add prefix naming to ChannelStatus enum variables
In this commit, we add a prefix naming scheme to the ChannelStatus enum
variables. We do this as it enables outside callers to more easily
identify each individual enum variable as a part of the greater
enum-like type.
2019-01-28 20:24:32 -08:00
Olaoluwa Osuntokun
f4054d2a66 channeldb: add new AddrsForNode method
In this commit, we add a new AddrsForNode method. This method will allow
a wrapper sturct to implement the new chanbackup.LiveChannelSource
method which is required to implement the full SCB feature set.
2019-01-28 20:24:22 -08:00
Johan T. Halseth
7d34ce9d08
lnwire+multi: define HasMaxHtlc helper on msgFlags 2019-01-22 08:42:30 +01:00
Johan T. Halseth
01e679786d
channeldb/graph_test: add TestEdgePolicyMissingMaxHtcl 2019-01-22 08:42:28 +01:00
Johan T. Halseth
6fcc76fc68
channeldb/channel: ignore ChannelEdgePolicy with missing max_htlc
If the max_htlc field is not found when fetching a ChannelEdgePolicy
from the DB, we treat this as an unknown policy.

This is done to ensure we won't propagate invalid data further. The data
will be overwritten with a valid one when we receive an update for this
channel.

It shouldn't be very common, but old data could be lingering in the DB
added before this field was validated.
2019-01-22 08:42:28 +01:00
Johan T. Halseth
8dd074ee57
channeldb/graph_test: assert MaxHTLC field correctness
Co-authored-by: Valentine Wallace <valentine.m.wallace@gmail.com>
2019-01-22 08:42:27 +01:00
Valentine Wallace
69d4bf051f
channeldb/graph: add max HTLC to ChannelEdgePolicy
Adding this field will allow us to persist an edge's
max HTLC to disk, thus preserving it between restarts.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:27 +01:00
Johan T. Halseth
b9c5248915
channeldb/graph: extract ChannelEdgePolicy serialization 2019-01-22 08:42:27 +01:00
Johan T. Halseth
91c9e45031
channeldb/graph: check correct bucket for nilness 2019-01-22 08:42:27 +01:00
Valentine Wallace
0fd6004958
multi: partition lnwire.ChanUpdateFlag into ChannelFlags and MessageFlags
In this commit:

* we partition lnwire.ChanUpdateFlag into two (ChanUpdateChanFlags and
ChanUpdateMsgFlags), from a uint16 to a pair of uint8's

* we rename the ChannelUpdate.Flags to ChannelFlags and add an
additional MessageFlags field, which will be used to indicate the
presence of the optional field HtlcMaximumMsat within the ChannelUpdate.

* we partition ChannelEdgePolicy.Flags into message and channel flags.
This change corresponds to the partitioning of the ChannelUpdate's Flags
field into MessageFlags and ChannelFlags.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:26 +01:00
Johan T. Halseth
8af2473644
channeldb/addr_test: add tests for invalid onion addresses 2019-01-17 14:16:15 +01:00
Johan T. Halseth
6a3e1423d2
channeldb/addr: sanity check onion address length before writing to db 2019-01-17 14:16:15 +01:00
Johan T. Halseth
e2e00f9dd2
channeldb/addr_test: add invalid TCP cases
These are invalid length IP addresses that earlier would not fail to
serialize.
2019-01-17 14:16:15 +01:00
Johan T. Halseth
02b7bb356a
channeldb/addr: validate ip before writing 2019-01-17 14:16:12 +01:00
Johan T. Halseth
6f605b2de2
channeldb/addr_test: inspect error string
Lets us match on more than static errors.
2019-01-17 14:14:24 +01:00
Wilmer Paulino
b951f06456
multi: move CsvDelay into ChannelConstraints 2019-01-11 16:58:15 -08:00
Olaoluwa Osuntokun
2e2d5fcf54
Merge pull request #2354 from joostjager/invoice-state
channeldb+lnrpc: invoice state
2019-01-04 14:48:17 -08:00
Joost Jager
5515713b88
channeldb: convert settled boolean to state
This commit is a preparation for the addition of new invoice
states. A database migration is not needed because we keep
the same field length and values.
2019-01-04 07:53:55 +01:00
Wilmer Paulino
70d3fc640a channeldb/db: prevent filtering channels with unconfirmed close txs
In this commit, we address an issue with the FetchWaitingCloseChannels
method where it would not properly return channels that are unconfirmed
and also have an unconfirmed closing transaction because they were
filtered out. We fix this by fetching channels that remain unconfirmed
that are also waiting for a confirmed closing transaction.

This will allow the recently added test TestFetchWaitingCloseChannels to
pass.
2019-01-03 20:48:28 -05:00
Wilmer Paulino
409efd1361 channeldb/channel_test: add TestFetchWaitingCloseChannels
In this commit, we add a test case for FetchWaitingCloseChannels to
ensure it behaves as intended. Currently, this test fails due to not
fetching channels which are pending to be closed but are also pending to
be opened. This will be fixed in the following commit and should allow
the test to pass.
2019-01-03 20:48:28 -05:00
Wilmer Paulino
f6c02aec20 channeldb/channel_test: use random outpoint when creating new test channels
In this commit, we use random outpoints when creating new test channels
to ensure we can uniquely identify them.
2019-01-03 20:33:33 -05:00
Johan T. Halseth
70aeda1589
channeldb/graph: don't create node bucket in PruneGraph
Instead return ErrSourceNodeNotSet directly. This would be returned from
the call pruneGraphNodes -> sourceNode anyway.
2018-12-06 10:22:16 +01:00
Johan T. Halseth
9d467d3534
channeldb/graph: reuse nodes bucket in deleteLightningNode
Lets us avoid passing the tx from pruneGraphNodes
2018-12-06 10:22:16 +01:00
Johan T. Halseth
1b9ca06563
channeldb/graph: reuse nodes bucket when fetchin source node 2018-12-06 10:22:16 +01:00
Johan T. Halseth
86d40636cd
channeldb/graph: don't create bucket in delEdgeUpdateIndexEntry
We are deleting what's in the bucket, so if it doesn't exist, just
return early with a non-error.
2018-12-06 10:22:16 +01:00
Johan T. Halseth
84de318553
channeldb/graph: don't create bucket in UpdateEdgePolicy
instead return ErEdgeNotFound, which would be returned anyway when
querying the edgeIndex for the channel.
2018-12-06 10:22:15 +01:00
Johan T. Halseth
9722323161
channeldb/graph: don't create node bucket in DeleteChannelEdge
Instead return ErrGraphNodeNotFound directly. If the node bucket was
created it would be empty, and the call delChannelByEdge ->
fetchChanEdgePolicies -> fetchChanEdgePolicy ->
deserializeChanEdgePolicy -> fetchLightningNode would return this error
anyway.
2018-12-06 10:22:15 +01:00
Johan T. Halseth
351f4aab79
channeldb/graph: don't create buckets in DeleteChannelEdge
Instead return ErrEdgeNotFound if buckets don't exist. This would be
returned anyway, when the chanIndex is checked for the channel point in
question.
2018-12-06 10:22:15 +01:00
Johan T. Halseth
5d8d99b7bc
channeldb/graph: don't create bucket in UpdateChannelEdge
It would return ErrEdgeNotFound when edge not found in bucket anyway.
2018-12-06 10:22:15 +01:00
Johan T. Halseth
81fe6e73ed
router+graph: return ErrGraphNodesNotFound if no nodes to prune
Avoids creating a bucket unneccessarily.
2018-12-06 10:22:15 +01:00
Johan T. Halseth
3d7bbd6d72
channeldb/graph: write max 80 sig len for nodes 2018-12-05 09:31:26 +01:00
Olaoluwa Osuntokun
1fd3aac925
multi: switch from bolt packge to bbolt package for all imports 2018-11-29 20:33:49 -08:00
Johan T. Halseth
b1a35fc8f4
channeldb/migrations_test: add TestMigrateOptionalChannelCloseSummaryFields 2018-11-21 13:36:42 +01:00
Johan T. Halseth
da3f515f12
channeldb/db_test: add TestFetchClosedChannelForID 2018-11-21 10:28:56 +01:00
Johan T. Halseth
45b132a954
channeldb/db: define FetchClosedChannelForID
FetchClosedChannelForID is used to find the channel close summary given
only a channel ID.
2018-11-21 10:28:56 +01:00
Johan T. Halseth
0b9a323fcb
channeldb/channel: add LastChanSync field to CloseChannelSummary
This commit adds an optional field LastChanSyncMsg to the
CloseChannelSummary, which will be used to save the ChannelReestablish
message for the channel at the point of channel close.
2018-11-21 10:28:43 +01:00
Johan T. Halseth
46c961aa17
channeldb/migrations: migrate ChannelCloseSummary 2018-11-21 10:27:58 +01:00
Johan T. Halseth
28b15dcbbb
channeldb/channel: write boolean to indicate presence of ChannelCloseSummary fields 2018-11-21 10:27:58 +01:00
Johan T. Halseth
149a8ce94f
channeldb/legacy_serialization: add deserializeCloseChannelSummaryV6
This commit adds a new file legacy_serialization.go, where a copy of the
current deserializeCloseChannelSummary is made, called
deserializeCloseChannelSummaryV6.

The rationale is to keep old deserialization code around to be used
during migration, as it is hard maintaining compatibility with the old
format while changing the code in use.
2018-11-21 10:12:05 +01:00
Wilmer Paulino
e795f7fce4
channeldb/graph: add method to determine if a node is public
In this commit, we add a method to the ChannelGraph struct that
determines whether a node is seen as public based on graph's source
node's point of view.
2018-10-24 17:28:53 -07:00
Johan T. Halseth
9cba3e813a
Merge pull request #1992 from halseth/listinvoices-offset
Make reversed listinvoices consistent in edge cases
2018-10-24 17:25:54 -07:00
Wilmer Paulino
28cf413055
channeldb: add method to retrieve a node's node announcement 2018-10-23 18:42:25 -07:00
Olaoluwa Osuntokun
4c0ca37174
Merge pull request #1389 from Bluetegu/noprivate-describegraph-1037
Add --noprivate flag to describegraph rpc to filter out private channels
2018-10-16 20:01:35 -07:00
Olaoluwa Osuntokun
3fda1029c0
Merge pull request #1786 from Roasbeef/golang-1.1
build+docs: bump to golang 1.11
2018-10-08 13:26:07 +09:00
Olaoluwa Osuntokun
d8e6085c17
multi: update to go 1.11 gofmt 2018-10-07 14:37:38 +09:00
Conner Fromknecht
59b459674d
multi: init subsystem loggers via build pkg 2018-10-05 13:04:45 +09:00
bluetegu
bd75a56855 Add unit test for db Wipe. 2018-10-04 13:35:42 -04:00
Olaoluwa Osuntokun
13802b4218
Merge pull request #1963 from wpaulino/create-chan-buckets-once
channeldb: ensure channel buckets are only created once
2018-09-28 17:06:39 -07:00
Johan T. Halseth
22d0e5b43e
channeldb/invoice test: add more tests for edge cases 2018-09-28 13:00:57 +02:00
Johan T. Halseth
0c51e31d1c
channeldb/invoices+test: don't return invoices on reversed query at index 1
Previously a call to QueryInvoices with reversed=true and index_offset=1
would make the cursor point to the first available invoice (num 1) that
would be returned as part of the response. This is inconsistent with the
othre indexes, so we instead just return an empty list in this case.

A test case for this situation is also added.
2018-09-28 13:00:57 +02:00
Olaoluwa Osuntokun
6afee3d099
Merge pull request #1856 from maurycy/typos
multi: fix various typos in comments
2018-09-27 20:38:10 -07:00
Wilmer Paulino
f13c1d2787
channeldb: ensure channel buckets are only created once
In this commit, we ensure that we only create the sub-bucket for
channels once: at the time of creation. We do this as otherwise it's
possible that a method that mutates a channel's state is called after it
has already been closed on-chain, leading to the channel bucket being
recreated.
2018-09-21 17:14:59 -07:00
Joost Jager
ab67b9a4de
rpcserver+lnrpc+lncli: add AbandonChannel rpc call
Using AbandonChannel, a channel can be abandoned. This means
removing all state without any on-chain or off-chain action.
A close summary is the only thing that is stored in the db after
abandoning.

A specific close type Abandoned is added. Abandoned channels
can be retrieved via the ClosedChannels RPC.
2018-09-18 12:20:27 -07:00
Conner Fromknecht
974ec02d24
channeldb/migrations: improve performance of edge update migration
This commit modifies the edge update index
migration to avoid repetitive calls to
Cursor.First(). Instead, we construct a set
of all edge update keys to remove, and then do
a second pass to remove each from the bucket.
Additional log messages are included to notify
users on progress, as some have reported long
migration times.
2018-09-17 00:05:24 -07:00
Olaoluwa Osuntokun
3b2c807288
channeldb: fix bug in migration from 0.4 to 0.5
In this commit, we fix a bug in the latest database migration when
migrating from 0.4 to 0.5. There's an issue in bolt db where if one
deletes a bucket that has a key with a nil value, it thinks that's a sub
bucket and attempts a bucket deletion. This will fail as it's not
actually a sub-bucket.  We get around this by using a cursor to manually
delete items in the
bucket.

Fixes #1907.
2018-09-14 12:39:44 -07:00
Olaoluwa Osuntokun
9e99f13f8d
Merge pull request #1898 from Roasbeef/fix-migration
channeldb: don't use KeyN in latest migration
2018-09-13 14:12:52 -07:00
Wilmer Paulino
7e6eb44cf8
channeldb/invoice_test: refactor TestQueryInvoices and add reverse test cases 2018-09-12 22:11:57 -07:00
Wilmer Paulino
0fe35e0014
channeldb/invoices: extend invoice queries to allow backwards pagination 2018-09-12 22:11:56 -07:00
Olaoluwa Osuntokun
034198ffb7
channeldb: don't use KeyN in latest migration
In this commit, we fix a bug in the latest migration that could cause
the migration to end in a panic. Additionally, we modify the migration
to exit early if the bucket wasn't found, as in this case, no migration
is required.

Fixes #1874.
2018-09-12 22:06:11 -07:00
Olaoluwa Osuntokun
09924f3f2c
channeldb: relax bucket assumptions for latest db migration
In this commit, we no longer assume that the bucket hierarchy has been
created properly when applying the latest DB migration. On older nodes
that never obtained a channel graph, or updated _before_ the query sync
stuff was added, then they're missing buckets that the migration expects
them to have.

We fix this by simply creating the buckets as we go, if needed.
2018-09-11 16:09:25 -07:00
Olaoluwa Osuntokun
3e67321aa7
channeldb: fix bucket creation hierarchy in createChannelDB
In this commit, we fix a bug in the bucket creation code in
createChannelDB. This bug can cause migrations on older nodes to fail,
as we expect the bucket to already have been created. With this commit,
we ensure that all the buckets under the main node and edges bucket are
properly created. Otherwise, a set of the newer migrations will fail to
apply for nodes updating from 0.4.
2018-09-09 15:49:28 -07:00
Olaoluwa Osuntokun
874002022f
channeldb: remove unused fwdPackageLogBucket key
In this commit, we move the declaration of the key for an unused bucket.
In the past, this bucket was used to store the revocation forwarding
package log. However, this has been moved under the key
`fwdPackagesKey`.
2018-09-09 15:47:52 -07:00
maurycy
ac24b12bf2
multi: fix various typos in comments 2018-09-07 06:51:49 +02:00
Olaoluwa Osuntokun
d2a7d910b7
Merge pull request #1825 from Roasbeef/extra-gossip-message-data
channeldb+discovery: ensure we store, validate and propagate announcements with opaque data
2018-09-05 17:53:02 -07:00
Olaoluwa Osuntokun
785efcfaa2
channeldb: also ignore the EOF error when trying to read ExtraOpaqueBytes
In this commit, we account for the additional case wherein the
announcement hasn't yet been written with the extra zero byte to
indicate that there aren't any remaining bytes to be read. Before this
commit, we accounted for the case where the announcement was written
with the extra byte, but now we ensure that legacy nodes that upgrade
will be able to boot properly.
2018-09-05 16:42:23 -07:00
Olaoluwa Osuntokun
16b5a67c3a
Merge pull request #1842 from wpaulino/dedup-chan-updates-horizon
channeldb: dedup channel edges returned from ChanUpdatesInHorizon
2018-09-04 21:24:40 -07:00
Olaoluwa Osuntokun
48072f8e82
channeldb: add limit on the max number of opaque bytes per announcement
In this commit, we add a new limit on the largest number of extra opaque
bytes that we'll allow to be written per vertex/edge. We do this in
order to limit the amount of disk space that we expose, as it's possible
that nodes may start to pad their announcements adding an additional
externalized cost as nodes may need to continue to store and relay these
large announcements.
2018-09-04 20:52:45 -07:00
Olaoluwa Osuntokun
a35bdd4233
channeldb: extend set of graph tests to include opque data where pertinent 2018-09-04 20:52:43 -07:00
Olaoluwa Osuntokun
f5c582d8d7
channeldb: add new ExtraOpaqueData to edge policy+update and node ann
In this commit, we add a mirror set of fields to the ones we recently
added to the set of gossip wire messages. With these set of fields in
place, we ensure that we'll be able to properly store and re-validate
gossip messages that contain a set of extra/optional fields.
2018-09-04 20:52:43 -07:00
Wilmer Paulino
3f58c2dea4
channeldb/graph: dedup channel edges returned from ChanUpdatesInHorizon
In this commit, we ensure that we de-duplicate the set of channel edges
returned from ChanUpdatesInHorizon. Other subsystems within lnd use this
method to retrieve and send all the channels with updates within a time
series to network peers. However, since the method looks at the edge
update index, which can include up to two entries per edge for each
policy, it's possible that we'd send channel announcements and updates
twice, causing extra bandwidth.
2018-09-04 18:48:21 -07:00
Wilmer Paulino
d3cf3168d2
channeldb/graph_test: ensure policies for an edge have different
timestamps

In this commit, we ensure policies for edges we create in
TestChanUpdatesInHorizon have different update timestamps. This ensures
that there are two entries per edge in the edge update index. Because of
this, the test will fail because ChanUpdatesInHorizon will return
duplicate channel edges due to looking at all the entries within the
edge update index. This will be addressed in a future commit to allow
the set of tests to pass once again.
2018-09-04 18:36:25 -07:00
Wilmer Paulino
85ea08fd17
channeldb: add migration to properly prune edge update index
In this commit, we introduce a migration to fix some of the recent
issues found w.r.t. the edge update index. The migration attempts to fix
two things:

1) Edge policies include an extra byte at the end due to reading an
extra byte for the node's public key from the serialized node info.

2) Properly prune all stale entries within the edge update index.

As a result of this migration, nodes will have a slightly smaller in
size channeldb. We will also no longer send stale edges to our peers in
response to their gossip queries, which should also fix the fetching
channel announcement for closed channels issue.
2018-09-04 18:33:43 -07:00
Wilmer Paulino
c1633da252
channeldb/graph_test: extend prune edge update index test to update edges
In this commit, we extend TestChannelEdgePruningUpdateIndexDeletion test
to include one more update for each edge. By doing this, we can
correctly determine whether old entries were properly pruned from the
index once a new update has arrived.
2018-09-04 18:33:41 -07:00
Wilmer Paulino
8dec659e10
channeldb/graph_test: properly check entries within edge update index
Due to entries within the edge update index having a nil value, the
tests need to be modified to account for this. Previously, we'd assume
that if we were unable to retrieve a value for a certain key that the
entry was non-existent, which is why the improper pruning bug was not
caught. Instead, we'll assert the number of entries to be the expected
value and populate a lookup map to determine whether the correct entries
exist within it.
2018-09-04 18:33:40 -07:00
Wilmer Paulino
2f22e6c35f
channeldb/graph: properly determine old update timestamp for an edge
In this commit, we fix a lingering issue within the edge update index
where entries were not being properly pruned due to an incorrect
calculation of the offset of an edge's last update time. Since the
offset is being determined from the end to the start, we need to
subtract all the fields after an edge policy's last update time from the
total amount of bytes of the serialized edge policy to determine the
correct offset. This was also slightly off as the edge policy included
an extra byte, which has been fixed in the previous commit.

Instead of continuing the slicing approach however, we'll switch to
deserializing the raw bytes of an edge's policy to ensure this doesn't
happen in the future when/if the serialization methods change or extra
data is included.
2018-09-04 18:33:38 -07:00
Wilmer Paulino
492d581df6
channeldb/graph: fix off-by-one public key slice
In this commit, we fix an off-by-one error when slicing the public key
from the serialized node info byte slice. This would cause us to write
an extra byte to all edge policies. Even though the values were read
correctly, when attempting to calculate the offset of an edge's update
time going backwards, we'd always be incorrect, causing us to not
properly prune the edge update index.
2018-09-04 18:31:38 -07:00
Wilmer Paulino
06344da62e
channeldb/graph: refactor UpdateEdgePolicy to use existing db transaction 2018-09-04 18:31:37 -07:00
Wilmer Paulino
aa3e2b6ba4
channeldb/graph: identify edge chan id on failure 2018-09-04 18:31:36 -07:00
Olaoluwa Osuntokun
4f20905ac1
Merge pull request #1719 from cfromknecht/duplicate-payments
Control Tower: Sender-side checks for duplicate payments #2
2018-08-22 21:22:38 -07:00
Olaoluwa Osuntokun
9f4685228e
Merge pull request #1763 from valentinewallace/listinvoices-pagination
channeldb+rpc: add pagination of listinvoices
2018-08-21 20:14:18 -07:00
Conner Fromknecht
86b347c996
channeldb/payments: make payment status helper methods
This commit splits FetchPaymentStatus and
UpdatePaymentStatus, such that they each invoke
helper methods that can be composed into different
db txns. This enables us to improve performance on
send/receive, as we can remove the exclusive lock
from the control tower, and allow concurrent calls
to utilize Batch more effectively.
2018-08-21 19:23:25 -07:00
Conner Fromknecht
0865ac7cf6
channeldb/migrations_test: assert locally-sourced circuits...
in the circuit map are marked StatusInFlight. We also
check that hashes contained in forwarded circuits are
not updated.
2018-08-21 19:23:25 -07:00
Conner Fromknecht
11bb5685f9
channeldb/migrations: mark locally-sourced payments as InFlight...
by reading the payment hash from the circuit map.
2018-08-21 19:23:25 -07:00
Conner Fromknecht
e7c0f4c5dc
channeldb/payments: touch up docs 2018-08-21 19:23:23 -07:00
Conner Fromknecht
3a579f3305
channeldb/migrations_test: touch up docs 2018-08-21 19:23:23 -07:00
Conner Fromknecht
93e5f9a545
channeldb/migrations: touch up documentation 2018-08-21 19:23:23 -07:00
Conner Fromknecht
9f46727507
channeldb/meta_test: restore migration helper + godocs 2018-08-21 19:23:23 -07:00
Conner Fromknecht
090e97cd3b
channeldb/db_test: move migration helper back to meta_test 2018-08-21 19:23:23 -07:00
Conner Fromknecht
79a4203346
channeldb/db: add comment describing migration 2018-08-21 19:23:23 -07:00
Vadym Popov
41e31e0909
channeldb: applying payment statuses migration to current database 2018-08-21 19:23:22 -07:00
Vadym Popov
24e3310f13
channeldb: payment statuses migration test 2018-08-21 19:23:21 -07:00
Vadym Popov
f405376b8b
channeldb: add migration for payment statuses for completed payments 2018-08-21 19:23:21 -07:00
Vadym Popov
7f6fbd4533
channeldb: test of payment statuses transitions 2018-08-21 19:23:21 -07:00
Vadym Popov
7296cfb425
channeldb: add payment statuses: ground, in flight, completed 2018-08-21 19:23:18 -07:00
Wilmer Paulino
f315b5b0d1
channeldb: support querying for invoices within a specific time range
In this commit, we introduce support for querying the database for invoices
that occurred within a specific add index range. The query format includes an
index to start with and a limit on the number of returned results.

Co-authored-by: Valentine Wallace <valentine.m.wallace@gmail.com>
2018-08-21 18:00:08 -07:00
Conner Fromknecht
af6c4e5174
channeldb/channel: adds readLogKey for chanids
Adds helper method to parse short chanids used as keys in the forwarding package.
2018-08-21 00:30:26 -07:00
Conner Fromknecht
81778664a7
channeldb/channel: expose AckAddHtlc and AckSettleFail 2018-08-21 00:30:26 -07:00
Conner Fromknecht
215a47e6bf
channeldb/forwarding_package: loosen Add/SettleFailRef acking
This commit loosens the fwdpkg reference acking to be more tolerant
of prior deletions. Specifically, we won't fail if certain channels
are not found or fwdpkgs do not exist. This will make us more
tolerant to future changes where we:
 - remove fwdpkgs on channel close
 - defensively cleanup stray responses
2018-08-21 00:30:23 -07:00
Olaoluwa Osuntokun
9353ab00b1
channeldb: ensure that we remove update index entries for nodes upon deletion 2018-08-16 18:22:55 -07:00
Olaoluwa Osuntokun
06d03b55a3
channeldb: add new TestNodePruningUpdateIndexDeletion test
In this commit, we add a new test to expose a lurking bug within the
graph database code. As is, when we go to delete a node from the
database, we don't also remove the entries within the update index. As a
result, if a user attempted to call NodeUpdatesInHorizon (or typically
as part of the p2p handshake), we would error out, as we would try to
read a node that no longer existed in the graph, as it was pruned.
2018-08-16 18:20:58 -07:00
Olaoluwa Osuntokun
90cdc9da8f
channeldb: add String() method for EdgePoint
In this commit we fix a minor logging artifact. After the switch to
EdgePoint, the FilteredChainView implementations will try to log the
struct directly, as prior they would have an outpoint object. We restore
this behavior by adding a String() method to EdgePoint which will simply
proxy through to the outpoint so we can log that directly.
2018-08-15 21:43:39 -07:00
Olaoluwa Osuntokun
26032f5956
channeldb: extend TestFetchClosedChannels to populate local chan cfg properly 2018-08-14 19:11:41 -07:00
Olaoluwa Osuntokun
d5863d03ae
Merge pull request #1660 from lightningnetwork/fetch-allll-channels
channeldb/db: return all channels if multiple chains exist
2018-08-09 20:59:28 -07:00
Olaoluwa Osuntokun
0f3c0ccf85
channeldb: add new FetchOtherNode method to ChannelEdgeInfo
In this commit, we add a new method to the ChannelEdgeInfo that will
allow the path finding logic to get the node opposite the pivot node
without first creating a new db transaction. The new method is able to
use an existing db transaction, or create a new one if needed.
2018-08-09 20:47:01 -07:00
Olaoluwa Osuntokun
ce7bfae4f7
channeldb: ensure we always check error from fetchChanEdgePolicy 2018-08-09 20:46:04 -07:00
Olaoluwa Osuntokun
0109672835
channeldb: add new db field for ChannelEdgeInfo to allow new methods 2018-08-09 20:45:39 -07:00
Joost Jager
6c918a1806 channeldb: store unknown policies in database
The commit ensures that for every channel, there will always
be two entries in the edges bucket. If the policy from one or
both ends of the channel is unknown, it is marked as such.

This allows efficient lookup of incoming edges. This is
required for backwards payment path finding.
2018-08-09 20:23:40 -07:00
Olaoluwa Osuntokun
8379bbaa9b
Merge pull request #1673 from cfromknecht/prevent-db-reversions
channeldb: fail Open on db reversion
2018-08-07 15:47:02 -07:00
Conner Fromknecht
cf2c371042
multi: fix linting errors 2018-08-02 18:20:50 -07:00
Conner Fromknecht
1ded697e8d
multi: sort import paths with gofmt 2018-08-02 18:20:49 -07:00
Conner Fromknecht
dff7706d04
channeldb/meta_test: adds TestMigrationReversion
Adds a test asserting that we will fail to open
a channeldb with a version higher than the
highest known version to the current build of
lnd.
2018-08-02 18:18:49 -07:00
Conner Fromknecht
e23fc40d63
channeldb/db: check for db reversions on startup 2018-08-02 16:46:17 -07:00
Conner Fromknecht
ee34b9c7b0
channeldb/error: adds ErrDBReversion error
Adds an error for signaling that we detected
the case where a user is trying to revert
to prior version before a db migration.
2018-08-02 16:45:03 -07:00
Olaoluwa Osuntokun
d129e7c890
channeldb: update ChannelView to return new EdgePoint struct
In this commit, we update the ChannelView method to be compatible with
the new set of interfaces that require the script to be passed in in
addition to the outpoint. In order to do this, we introduce a new
EdgePoint struct which packages together a channel point along with the
funding pkScript. Along the way, we've copied over a utility method from
the lnwallet package to avoid having to deal with an import cycle.
2018-07-31 21:28:54 -07:00
Johan T. Halseth
6cdf0e2d6e
channeldb/channel: methods for marking borked+dataloss commitPoint in db 2018-07-31 15:16:22 +02:00
Johan T. Halseth
ea6aca26a5
channeldb: make chanStatus unexported
Since the ChanStatus field can be changed from concurrent callers, we
make it unexported and add the method ChanStatus() for safe retrieval.
2018-07-31 15:07:30 +02:00
Conner Fromknecht
5aa96758e4
channeldb/db: return all channels if multiple chains exist 2018-07-30 23:39:40 -07:00
Olaoluwa Osuntokun
ef4c7d2a78
channeldb: add new TestAddChannelEdgeShellNodes test 2018-07-22 21:05:52 -07:00
Olaoluwa Osuntokun
2e75499787
channeldb+routing: move adding shell nodes into db txn of AddChannelEdge
In this commit, we fix a slight race condition that can occur when we go
to add a shell node for a node announcement, but then right afterwards,
a new block arrives that causes us to prune an unconnected node. To
ensure this doesn't happen, we now add shell nodes within the same db
transaction as AddChannelEdge. This ensures that the state is fully
consistent and shell nodes will be added atomically along with the new
channel edge.

As a result of this change, we no longer need to add shell nodes within
the ChannelRouter, as the database will take care of this operation as
it should.
2018-07-22 21:02:53 -07:00
Olaoluwa Osuntokun
8519ea869d
channeldb: add logging for nodes pruned from the channel graph 2018-07-22 21:00:51 -07:00
Olaoluwa Osuntokun
6bf15e8f2b
channeldb: fix bug in pruneGraphNodes by switching to using ref counting
In this commit, we fix an existing bug in the pruneGraphNodes method.
Before this commit, if a node was involved in a channel, but only one of
the edges was advertised, then either it, or the other node would be
erroneously pruned from the graph. They shouldn't be pruned as there's
still an edge connecting the, although only 1/2 of the edge is actually
advertised.

In order to fix this, we'll now do two passes: the first pass will
populate a ref count map of all known nodes in the graph, the second
pass will increment the ref count each time a node is found in the
graph. With this two pass method, we ensure that nodes are only deleted
if there are absolutely no edges pointing to them within the graph.
2018-07-22 18:59:36 -07:00
Olaoluwa Osuntokun
131eea4960
channeldb: modify TestPruneGraphNodes to test nodes w/ single edge
In this commit, we extend the TestPruneGraphNodes test to also test the
case of when a node is involved in a channel, but only a single edge for
that channel has been advertised. In order to test this, we add an
additional node to the graph, and also a new channel. However, this
channel will only have a single edge advertised. As result, when we
prune the set of edges, the only node remaining should be the node that
didn't have any edges at all.
2018-07-22 18:57:18 -07:00
Olaoluwa Osuntokun
0645261e5e
channeldb: add nwe PruneGraphNodes method to prune unconnected nodes 2018-07-21 19:49:59 -07:00
Olaoluwa Osuntokun
3a465c64b5
channeldb: modify pruneGraphNodes to prune nodes if no edges exist 2018-07-21 19:49:16 -07:00
Olaoluwa Osuntokun
bca926d6af
Merge pull request #1371 from wpaulino/prune-link-nodes
server: prune link nodes without any open channels
2018-07-21 18:55:38 -07:00
Wilmer Paulino
0aa1f39af8
channeldb+server: prune link nodes on startup
In this commit, we extend the server's functionality to prune link nodes
on startup. Since we currently only decide whether to prune a link node
from the database based on a channel close, it's possible that we have
link nodes lingering from before this functionality was added on.
2018-07-17 17:35:31 -07:00
Wilmer Paulino
0e4d350e56
channeldb: prune nodes with no open channels left from the graph 2018-07-17 17:35:29 -07:00
Wilmer Paulino
4578eec8a1
channeldb: garbage collect link nodes with no open channels remaining 2018-07-17 17:35:27 -07:00
Wilmer Paulino
959618d596
channeldb: refactor methods to allow using existing db transaction 2018-07-17 17:35:26 -07:00
Wilmer Paulino
044e81bbd4
channeldb: add DeleteLinkNode method 2018-07-17 17:35:25 -07:00
Olaoluwa Osuntokun
5ee368f0f9
channeldb: ensure that we return an invoice on duplicate settles
In this commit, we fix an existing bug related to duplicate invoice
settle.s Before this commit, the second (and later) times an invoice was
settled we would return a nil pointer. This would result in the new
invoiceRegistry panicing as it would go to attempt to notify with a nil
invoice.

We fix this by returning the invoice on disk (unmodified) for each
settle after the initial one.

Fixes #1568.
2018-07-17 16:28:08 -07:00
Olaoluwa Osuntokun
509998db86
channeldb: add new test for duplicate invoice settles
In this commit, we add a new test to ensure that duplicate invoice
settles work as expected. At the present time, this test will fail as
the second to last assertion fails as we'll return a nil invoice the
second time around.
2018-07-17 16:27:04 -07:00
Olaoluwa Osuntokun
6f60f139f4 multi: switch over import paths from roasbeef/* to btcsuite/* 2018-07-13 17:05:39 -07:00
Olaoluwa Osuntokun
19d84dd1cc
channeldb: add new migration to finalize invoice migration for outgoing payments
In this commit, we migrate the database away from a partially migrated
state. In a prior commit, we migrated the database in order to update
the Invoice struct with three new fields: add index, settle index, paid
amt.  However, it was overlooked that the OutgoingPayment struct also
embedded an Invoice within it. As a result, nodes that upgraded to the
first migration found themselves unable to start up, or call
listpayments, as the internal invoice within the OutgoignPayment hadn't
yet been updated.  This would result in an OOM typically as we went to
allocate a slice with a integer that should have been small, but may
have ended up actually being a set of random bytes, so a very large
number.

In this commit, we finish the DB migration by also migrating the
internal invoice within each OutgoingPayment.

Fixes #1538.
Fixes #1546.
2018-07-12 00:22:28 -07:00
Olaoluwa Osuntokun
515ba7a4d0
channeldb: fix formatting for invoice migration log 2018-07-11 17:05:56 -07:00
Olaoluwa Osuntokun
db4a09d3b5
channeldb: skip sub-buckets during invoice time series migration
In this commit, we fix an existing bu gin the invoice time series
migration code. Before this commit, the migration would fail as we would
try to migrate an empty invoice. We now detect this case and skip all
empty invoices.

We also add a bit more logging on both the info and trace logging level.
2018-07-06 16:45:02 -07:00
Olaoluwa Osuntokun
7aeed0b58f
channeldb: AddInvoice now returns the addIndex of the new invoice 2018-07-06 12:22:01 -07:00
Olaoluwa Osuntokun
e5c579120e
channeldb: modify SettleInvoice to also return the invoice being settled 2018-07-06 12:22:00 -07:00
Olaoluwa Osuntokun
2097564307
channeldb: add new migration to upgrade old databases to new invoice time series index 2018-07-06 12:21:58 -07:00
Olaoluwa Osuntokun
24d97b5ee5
channeldb: add new test to exercise invoice time series index 2018-07-06 12:21:58 -07:00
Olaoluwa Osuntokun
532975588d
channeldb: update existing invoice tests to due to recent field additions 2018-07-06 12:21:57 -07:00
Olaoluwa Osuntokun
24ae13d3a5
channeldb: add two methods to allow seeking into the invoice time series
In this commit, we add two new methods: InvoicesAddedSince and
InvoicesSettledSince. These methods will be used by higher level
sub-systems that implement notifications to deliver any notifications
backlog based on the last add index, and last settle index that the
client knows of.

It's important to note that care has been taken to ensure that this new
API can be used in a backwards compatible manner. If a client specifies
and index of 0 for either of the methods, then no backlog will be sent.
This is due to the fact that current users of the API don't expect any
backlog notifications to be sent. Additionally, the index actually
starts at 1, instead of 0.
2018-07-06 12:21:57 -07:00
Olaoluwa Osuntokun
5d20c02ea8
channeldb: add new add+settle index to invoice database
In this commit, we add two new indexes to the invoice database: the add
index, and the settle index. These to indexes essentially form a time
series index on top of the existing primary index bucket. Each time an
invoice is added, we'll advance the addIndex seqno, and then create a
mapping from seqNo -> invoiceNum. Each time an invoice is settled, we'll
do the same, but within the settle index.

This change is required in order to allow callers to effectively seek
into the current invoice database in order to obtain notifications for
any invoices they may have missed out on while they were disconnected.
This will allow us to implement robust streaming invoice notifications
within lnd to ensure that clients never miss an event.
2018-07-06 12:21:56 -07:00
Olaoluwa Osuntokun
fc0f0d33b2
channeldb: add new AmtPaid field to the Invoice struct
In this commit, in order to allow the caller to specify the amount that
was ultimately accepted for an invoice, the SettleInvoice method has
gained a new parameter: amtPaid. SettleInvoice will now populate the
final amount paid in the database upon db commit.
2018-07-06 12:21:56 -07:00
Olaoluwa Osuntokun
dbf7e4525a
lntest: ensure we always cancel active streaming notifications 2018-07-06 12:21:54 -07:00
Conner Fromknecht
75d7436f7a
channledb/codec: add net.Addr and []net.Addr to codec 2018-07-03 17:07:10 -07:00
Conner Fromknecht
4cb7953f66
channledb/addr: remove silent encoding failure for unknown types 2018-07-03 17:07:10 -07:00
Conner Fromknecht
c6c442f6b6
channledb/addr_test: add negative test for unknown types 2018-07-03 17:07:10 -07:00
Conner Fromknecht
21674c05e5
channeldb/codec: add concrete error for unknown types 2018-07-03 17:07:09 -07:00
Conner Fromknecht
1d5189bd25
channeldb/forwarding_log: use public Read/WriteElements 2018-07-03 17:07:09 -07:00
Conner Fromknecht
30ff91913d
channeldb/channel: use public Read/WriteElements 2018-07-03 17:07:09 -07:00
Conner Fromknecht
ec029ac7fd
channeldb/codec: exposes Read/WriteElements 2018-07-03 17:07:09 -07:00
Olaoluwa Osuntokun
e5f802c33c Revert "channeldb: explicitly store the FinalCltvDelta within the ContractTerm struct"
This reverts commit 8dcfeeaef507ebe02c60e34022f614a65a18b050.
2018-06-29 12:40:42 -07:00
Olaoluwa Osuntokun
8dcfeeaef5 channeldb: explicitly store the FinalCltvDelta within the ContractTerm struct
In this commit, we move to explicitly storing a bit more information
within the invoice. Currently this information is already stored in the
payment request, but by storing it at this level, callers that may not
be in the state to fully decode a payment request can obtain this data.

We avoid a database migration by appending this data to the end of an
invoice. When decoding, we'll try to read out this extra information,
and simply return what we have if it isn't found.
2018-06-26 19:49:49 -07:00
Conner Fromknecht
56e5eed037
channeldb/channel: update short chan id for fwd packager 2018-06-19 13:48:08 +01:00
Conner Fromknecht
639c9875b2
channeldb/channel_test: test packager source updated 2018-06-19 13:48:08 +01:00
Conner Fromknecht
086b44c1f4
channeldb/waitingproof: improve external consistency of store
This commit synchronizes the in-memory cache with the
on-disk state to ensure the waiting proof store is
externally consistent. Currently, there are scenarios
where the in-memory state is updated, and not reverted
if the write fails. The general fix is to wait to apply
modifications until the write succeeds, and use a
read/write lock to synchronize access with db operations.
2018-06-05 21:33:47 -07:00
Olaoluwa Osuntokun
991c9fb7dd
Merge pull request #1332 from Roasbeef/delete-update-index-items
channeldb: ensure items from the update index are actually deleted
2018-06-05 19:34:33 -07:00
Olaoluwa Osuntokun
1d994439e2
Merge pull request #1323 from Roasbeef/accept-duplicate-waiting-proofs
channeldb: don't reject duplicate waiting proofs
2018-06-05 19:20:29 -07:00
Olaoluwa Osuntokun
dd4cab054d
channeldb: ensure items from the update index are actually deleted
In this commit, we fix an existing bug in the new graph query sync
feature. Before this commit, when a block is pruned, we would never
actually delete the update index entries. This is due to the fact that
we would attempt to delete the entries from the update index _after_ we
had already removed the edges from the update index.

We fix this by simply swapping the order: first we delete from the
update index, then we delete the edges themselves. A test ensuring that
the entires are cleared (which failed before this commit), has been
added.
2018-06-05 17:30:56 -07:00
Wilmer Paulino
2e0484be19
multi: ensure addresses are no longer assumed to be TCP addresses only
In this commit, we go through the codebase looking for TCP address
assumptions and modifying them to include the recently introduced onion
addresses. This enables us to fully support onion addresses within the
daemon.
2018-06-04 20:41:49 -07:00
Wilmer Paulino
5f1d2524b8
channeldb: add address serialization tests 2018-06-04 20:41:34 -07:00
Wilmer Paulino
3738e68ae2
channeldb: implement serialization of onion addresses
Co-Authored-By: Eugene <crypt-iq@users.noreply.github.com>
2018-06-04 20:41:33 -07:00
Olaoluwa Osuntokun
ece5a29374
channeldb: don't reject duplicate waiting proofs
In this commit, we modify the waiting proof slightly to acept dupliacte
waiting proofs, rather than reject them. Otherwise, it's possible that
the remote node first sends us their half of the waiting proof (before
we do), we write that to disk, then upon restart, we'll try to add it
again, but be rejected by the system.

Fixes #1315.
2018-06-04 16:37:33 -07:00
Olaoluwa Osuntokun
70dffe7e99
channeldb: ensure that when we delete a channel we delete entry in edge update index
In this commit, we ensure that all indexes for a particular channel have
any relevant keys deleted once a channel is removed from the database.
Before this commit, if we pruned a channel due to closing, then its
entry in the channel update index would ever be removed.
2018-05-31 16:30:59 -07:00
Olaoluwa Osuntokun
12e73f55e9
channeldb: add database migration for new node+edge update indexes
In this commit, we add a new database migration required to update old
database to the version of the database that tracks the update index for
the nodes and edge policies. The migration is straight forward, we
simply need to populate the new indexes for the all the nodes, and then
all the edges.
2018-05-31 16:30:54 -07:00
Olaoluwa Osuntokun
baed4d1f47
channeldb: add new methods required to implement new discovery.ChannelGraphTimeSeries interface
In this commit, we add a series of methods, and a new database index
that we'll use to implement the new discovery.ChannelGraphTimeSeries
interface interface. The primary change is that we now maintain two new
indexes tracking the last update time for each node, and the last update
time for each edge. These two indexes allow us to implement the
NodeUpdatesInHorizon and ChanUpdatesInHorizon methods. The remaining
methods added simply utilize the existing database indexes to allow us to
respond to any peer gossip range queries.

A set of new unit tests has been added to exercise the added logic.
2018-05-31 16:30:54 -07:00
Olaoluwa Osuntokun
fc3d711cf0
Merge pull request #1248 from halseth/close-channel-fix
[bugfix] Wait for confirmation before marking channel cooperatively closed
2018-05-24 18:28:31 -07:00
Johan T. Halseth
72d9726e7f
channeldb/channel: update IsPending godoc
The pending state definitin in ChannelCloseSummary was slightly changed
in such a way that channels that has had their commitment broadcasted
now is no longer considered "pending close". They now instead stay in
the open chan bucket with the ChanStatus "CommitmentBroadcasted" until
their commitment is confirmed. This commit updates the IsPending godoc
to reflect this.
2018-05-23 12:07:35 +02:00
Olaoluwa Osuntokun
71d7d4952f
channeldb: remove premature buf.Grow optimization
This optimization isn't needed as a bytes.Buffer already comes
pre-allocated with 64-bytes as a fast path.
2018-05-22 17:52:45 -07:00
Olaoluwa Osuntokun
3f2aa1c368
channeldb: modify updateChanBucket to no longer auto-create buckets
In this commit, we modify the existing updateChanBucket function to no
longer auto-create buckets if they don't exist. We do this in order to
fix a class of bug that could arise wherein after a channel has actually
be closed (and the parent buckets removed) a method that mutates the
channel state is called, which then re-creates the relevant set of
buckets. As a result, subsequent calls to any RPCs which need to read
all the channels will fail as most of the fields won't actually be
populated.

After this commit, the fullSync method is the only one that's able to
create the full bucket hierarchy.
2018-05-22 16:32:36 -07:00
Conner Fromknecht
f963f91a3c
multi: use mutex-guarded ShortChanID() on OpenChannel 2018-05-09 16:06:49 -07:00
Conner Fromknecht
19b5eaa4dc
channeldb/channel_test: adds RefreshShortChanID test 2018-05-09 16:06:49 -07:00
Conner Fromknecht
7ad56943c7
channeldb/channel: adds RefreshShortChanID 2018-05-09 16:06:49 -07:00
Johan T. Halseth
f26ec38a74
channeldb: return ErrNoPendingCommit in case of no channel bucket 2018-05-08 13:41:52 +02:00
Olaoluwa Osuntokun
1ec5dc3c2d
channeldb: additionally store static channel information in CloseChannelSummary
In this commit, we extend the CloseChannelSummary by also storing: the
current unrevoked revocation for the remote party, the next pending
unused revocation, and also the local channel config. We move to store
these as the provide an extra level of defense against bugs as we'll
always store information required to derive keys for any current and
prior states.
2018-05-03 21:29:23 -07:00
Conner Fromknecht
1b6101b0c0
channeldb/channel: add NextLocalHtlcIndex 2018-05-02 01:12:17 -07:00
Johan T. Halseth
0735b8e0b7
channeldb: add method for fetching channels waiting for closing tx
This commit adds a new method FetchWaitingCloseChannels to the database,
used for fetching OpenChannels that have a ChanStatus != Default. These
are channels that are borked, or have had a commitment broadcasted, and
is now waiting for it to confirm.

The fetchChannels method is rewritten to return channels exclusively
based on wheter they are pending or waitingClose.
2018-04-25 09:37:25 +02:00
Johan T. Halseth
c51f6352c3
channeldb test: rename ForceClose->RemoteForceClose 2018-04-25 09:37:24 +02:00
Johan T. Halseth
cde862e7e0
channeldb/codec: add ChannelStatus 2018-04-25 09:37:24 +02:00
Johan T. Halseth
b7bb53a8b8
channeldb/channel: add property ChanStatus in place of IsBorked to OpenChannel
This commit changes the bool `IsBorked` in OpenChannel to a `ChanStatus`
struct, of type ChannelStatus. This is used to indicated that a channel
that is technically still open, is either borked, or has had a
commitment broadcasted, but is not confirmed on-chain yet.

The ChannelStatus type has the value 1 for the status Borked, meaning it
is backwards compatible with the old database format.
2018-04-25 09:37:23 +02:00
Dimitris Tsapakidis
4009f7f874 multi: fix typos in comments 2018-04-17 19:03:27 -07:00
practicalswift
663c396235 multi: fix a-vs-an typos 2018-04-17 19:02:04 -07:00
Olaoluwa Osuntokun
7a13378671
channeldb+contractcourt: add additional logging around co-op channel closes 2018-04-12 17:13:37 -07:00
Conner Fromknecht
f594a242ea
channeldb/channel_test: test ShortChanID is updated in-mem
Modifies TestFetchPendingChannels to verify that calls to
MarkAsOpen also modify the in-memory state. Previously we
only tested the persistent state loaded immediately after.
2018-03-27 17:07:15 -07:00
Conner Fromknecht
9fd73af589
channeldb: update chanid in-mem during MarkAsOpen
Modifies the MarkAsOpen operation to also update the
ShortChanID and IsPending fields in-memory. Before,
only the on-disk representation was updated, which
may have lead to stale data channel states being
passed in-memory.
2018-03-27 17:05:22 -07:00
Conner Fromknecht
5df8b52dae
multi: set initiator funding txn 2018-03-11 15:06:22 -07:00
Conner Fromknecht
db88c82169
channeldb/channel: add serialization for single funder funding txn 2018-03-11 13:58:40 -07:00
Olaoluwa Osuntokun
800eea931f
build+multi: switch from bolt to bbolt
In this commit, we switch from boltbd/bolt to coreos/bbolt as the
former is no longer being actively maintained.
2018-03-10 19:01:13 -08:00
Conner Fromknecht
7a93c7530c
channeldb/invoices: add idempotency to SettleInvoice 2018-03-09 14:45:30 -08:00
Conner Fromknecht
e4d2958f68
channeldb/channel_test: init with Pacakager and construct FwdPkgs 2018-03-09 14:45:30 -08:00
Conner Fromknecht
2df9fb5510
channeldb/channel: adds fwding package to channeldb 2018-03-09 14:45:30 -08:00
Conner Fromknecht
d18c317220
channeldb/forwarding_package_test: test PkgFilter 2018-03-09 14:45:29 -08:00
Conner Fromknecht
41c40a9560
channeldb/forwarding_package: initial fwdpkg 2018-03-09 14:45:29 -08:00
Olaoluwa Osuntokun
705661a39e
channeldb: replace raw keys in ChannelConfig with keychain.KeyDescriptor
In this commit, we remove references to raw keys from the main
ChannelConfig struct and instead replace it with usage of
keychain.KeyDescriptor. We do this, as the ChannelConfig as it stands
is a near complete static description of a channel. In the future, it
will be possible to export these static descriptions as backups. We
prefer the KeyDescriptor of a plain PublicKey, as the KeyLocator
portion of the struct allows a stateless signer to re-derive the keys
as needed when signing.
2018-03-06 16:04:01 -05:00
Olaoluwa Osuntokun
744cfd2ce5
channeldb: add a set of initial tests for the ForwardingLog 2018-03-06 13:56:09 -05:00
Olaoluwa Osuntokun
f2cd668bcf
channeldb: add new ForwardingLog storage namespace
In this commit, we add a new storage namespace to channeldb: the
ForwardingLog. This log will be used by higher level sub-systems to log
each successfully completed HTLC. Each payment circuit will be
summarized as a “ForwardingEvent”. A series of events can then be
queried via a time slice query. In a time slice query, the caller
specifies a time range, a number of events to skip, and the max number
of events to return. Each query will return the index of the final
item. As we have a max number of events we’ll return in a response,
callers may need to use this last offset index to seek further by
skipping that number of entries. Combining these fields, callers are
able to query the time series, skipping an arbitrary amount of events,
and capping the max number of returned events.
2018-03-06 13:56:06 -05:00
Johan T. Halseth
004563b20b
channeldb: add todo making feerate explicit type 2018-02-26 22:42:27 +01:00
practicalswift
b8e1351cf3 multi: fix some recently introduced typos 2018-02-18 15:27:29 -08:00
Olaoluwa Osuntokun
3e422fedd3
channeldb: further GC optimizations during path finding
In this commit, we made a series of modification to the way we handle
reading edges and vertexes from disk, in order to reduce the amount of
garbage generated:
  1. Properly use PubKeyBytes are required rather than PubKey()
  2. Return direct structs rather than pointers, and leave it to the
runtime to perform escape analysis.
  3. In-line the former readSig() method when reading sigs from disk.
2018-02-12 16:17:14 -08:00
Johan T. Halseth
65723387fa
channeldb: update ChannelConstraints godoc
This commit changes the definition of the
constraints in the ChannelConstraints struct
to specify that these are all constraints the
*owner* of the set of constraints must *never
violate*.

This is done to make it easier to check that
a particular node is not violating any
constraint for a gien update, as before it
could violate constraints found both in its
local and the remote contraints.
2018-02-08 18:35:24 -05:00
Olaoluwa Osuntokun
e578cea375
channeldb: fix linter errors 2018-02-06 20:14:34 -08:00
Olaoluwa Osuntokun
5e9166e478
channeldb: use raw pub keys and signatures directly in vertex/edge structs
In this commit, we make an API change that’s meant to reduce the amount
of garbage we generate when doing pathfinding or syncing nodes with our
latest graph state. Before this commit, we would always have to fully
decode the public key and signatures when reading a edge or vertex
struct. For the edges, we may need several EC operations to fully
decode all the pubkeys. This has been seen to generate a ton of
garbage, as well as slow down path finding a good bit.

To remedy this, we’ll now only ever read the *raw* bytes from disk. In
the event that we actually need to verify a signature (or w/e), only
*then* will we fully decode everything.
2018-02-06 20:14:31 -08:00
Daniel McNally
8543497dcc multi: fixing it's/its typos in comments 2018-02-06 19:13:07 -08:00
practicalswift
a93736d21e multi: comprehensive typo fixes across all packages 2018-02-06 19:11:11 -08:00
MeshCollider
4b1cc98808 multi: apply roasbeef diff to support incoming socks.ProxiedAddr 2018-02-05 17:37:46 -08:00
MeshCollider
36219427ad channeldb: use string comparison instead of deepequal to avoid array length mismatches 2018-02-05 17:37:46 -08:00
MeshCollider
d8f453d9dc channeldb: remove address resolution from channeldb 2018-02-05 17:37:46 -08:00
nsa
e2142c778f multi: Added Tor support
This commit adds Tor support. Users can set the --TorSocks flag
to specify which port Tor's SOCKS5 proxy is listening on so that
lnd can connect to it. When this flag is set, ALL traffic gets
routed over Tor including DNS traffic. Special functions for
DNS lookups were added, and since Tor doesn't natively support
SRV requests, the proxySRV function routes connects us to
a DNS server via Tor and SRV requests can be issued directly
to the DNS server.

Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
2018-02-05 17:37:46 -08:00
Olaoluwa Osuntokun
a94648e9dc
channeldb: properly use a read-transaction in FetchChannelEdgesByOutpoint
Before this commit, we’d unnecessarily use a write transaction within
the FetchChannelEdgesByOutpoint. This is wasteful as the function only
actually reads items from the database, and doesn’t attempt any
mutations at all.
2018-01-28 14:48:57 -08:00
Andrew Naoum
0f161c5033 channeldb+lnrpc: store the payment pre-image not rhash
Fixes #481.

Prior to this commit, payments stored in the channel DB only kept a
record of the payment hash. This is a problem as the preimage is what
serves as proof of payment and a user should be able to look up this
value in the future (not just immediately after payment).

Instead of storing both the payment hash and the preimage, we store the
preimage only since the hash can be derrived from this using a SHA256.

In the RPC listpayments command, we now give the preimage in addition to
the payment hash.
2018-01-27 18:36:33 -08:00
Conner Fromknecht
926949ef0f
channeldb/db: adds FetchClosedChannel
This commit adds a FetchClosedChannel method to the
channeldb, which allows querying based on a known
channel point. This will be used in the nursery to
load channel close summaries, which can be used to
provide more accurate height hints when recovering
from failures.
2018-01-24 15:38:50 -08:00
Olaoluwa Osuntokun
ebb4c84b32
channeldb: add new LatestCommitments and RemoteRevocationStore methods
These methods will allow the chainWatcher to ensure it has the latest
channel state before attempting to construct any resolution objects.
2018-01-22 19:19:51 -08:00
Olaoluwa Osuntokun
850ff1d970
channeldb: add new WitnessCache structure
In this commit, we add the WitnessCache sub-storage system of the
greater database. The WitnessCache is a persistent cache of all
witnesses we’ve encountered on the network. We’ll use this cache to
share any on-chain discoveries between active channels. Eventually
we’ll also use this to enforce the variant that a preimage is only to
be used ONCE on the network.
2018-01-22 19:19:37 -08:00
Olaoluwa Osuntokun
b4fc34a93a
channeldb: add ShortChanID to close channel summary 2018-01-22 19:19:35 -08:00
Olaoluwa Osuntokun
5d7119fe9c
channeldb: expose methods to serialize HTLC's on-disk
These new methods will be used by the contract court package to
properly serialize the state of HTLC’s that have yet to be fully
resolved on-chain.
2018-01-22 19:19:35 -08:00
Conner Fromknecht
95b788e9a6
channeldb/db: exposes Path method, useful for testing 2018-01-05 13:47:17 -08:00
Conner Fromknecht
1d69526874
channeldb/channel: adds IsBorked to OpenChannel 2018-01-05 13:47:17 -08:00
Matt Drollette
adf0d98194 multi: fix several typos in godoc comments 2017-12-17 18:40:05 -08:00
Johan T. Halseth
48e22219c0 channeldb: add ChannelFlags to OpenChannel struct
This commit adds the ChannelFlags field, of type
lnwire.FundingFlags, to the OpenChannel struct,
including serialization for database storage.
This is done to preserve the flags that were
sent during channel opening, currently used
to determine whether a channel should be made
public or not after opening.
2017-12-17 18:35:34 -08:00
Micah Lerner
cf6f313cff channeldb: update channeldb to set and store SettleDate for invoices 2017-12-08 16:50:04 -08:00
Micah Lerner
1295bfe096 channeldb: fix channel_test scoping error 2017-12-03 16:34:52 -08:00
Olaoluwa Osuntokun
9b53d7bd95
channeldb: treat Flags field in ChannelEdgePolicy as a bitfield 2017-11-30 22:41:54 -08:00
Olaoluwa Osuntokun
12e5951434
channeldb: extend ChannelConfig with new HtlcBasePoint key 2017-11-16 20:00:01 -08:00
Olaoluwa Osuntokun
6f51b941df
multi: fix linter errors 2017-11-10 19:51:13 -08:00
Olaoluwa Osuntokun
1fb05e0436
channeldb: properly craft key for reading/writing channel commitments
In this commit, we fix an existing bug that arose due to incorrectly
crafting the key we use to store channel commitments. Before this
commit, we tried to copy to a slice that hadn’t been allocated yet. As
a result, the key would only have the 0x00 or 0x01 as its value. We fix
this by properly crafting the key using the built-in append function.
2017-11-10 19:51:12 -08:00
Olaoluwa Osuntokun
699e5327e1
channeldb: don't delete the chainHash bucket in CloseChannel
In this commit, we fix an existing bug wherein if we closed two
channels, then we were unable to read the channel state afterwards as
we deleted the enclosing bucket.
2017-11-10 19:51:12 -08:00
Olaoluwa Osuntokun
3875754e0f
channeldb: also update chanInfo when updating commitment
In this commit, we fix an existing bug wherein we failed to update the
channels state once we accepted a new commitment. As a result, after a
state transition, if the channel state was read from disk, values like
TotalMSatSent wouldn’t be properly updated.
2017-11-10 19:51:11 -08:00
Olaoluwa Osuntokun
1d6e5ad1ef
channeldb: add missing mutex interaction to new methods 2017-11-10 19:51:11 -08:00
Olaoluwa Osuntokun
29f2f88abc
channeldb: expand TestChannelStateTransition to expertise new state transition related methods
In this commit we’ve extended the TestChannelStateTransition method to
exercise the new state transition related messages. This includes
ensuring that when we add a new dangling commitment, and then the
remote party revokes it, then the on-disk state is update accordingly.
2017-11-10 19:50:50 -08:00
Olaoluwa Osuntokun
06e6649da5
channeldb: update channel closing methods to respect new on-disk structure 2017-11-10 19:50:50 -08:00
Olaoluwa Osuntokun
385818307b
channeldb: update FetchOpenChannel to traverse new chain bucket 2017-11-10 19:50:49 -08:00
Olaoluwa Osuntokun
da7a5f7c4e
channeldb: update tests to respect new API's 2017-11-10 19:50:49 -08:00
Olaoluwa Osuntokun
0178920cba
channeldb: update CloseChannel to adhere to new disk structure
In this commit, we update the CloseChannel method to respect the new
on-disk bucket based structure. Additionally, we now ensure that we
delete the new chainBucket.
2017-11-10 19:50:48 -08:00
Olaoluwa Osuntokun
3cf4ac8237
channeldb: add a new ChainHash field to ChannelCloseSummary 2017-11-10 19:50:48 -08:00
Olaoluwa Osuntokun
da22078995
channeldb: modify FindPreviousState to return a ChannelCommitment 2017-11-10 19:50:47 -08:00
Olaoluwa Osuntokun
5bdff9585b
channeldb: modify RevocationLogTail to return a ChannelCommitment 2017-11-10 19:50:47 -08:00
Olaoluwa Osuntokun
374cab7467
channeldb: rename AppendToRevocationLog to AdvanceCommitChainTail
In this commit, in addition to the renaming we’ve modified the behavior
of AdvanceCommitChainTail as follows: this method now will simply
atomically advance the commitment tail, set the new commitment to the
prior dangling commitment, and update the on-disk revocation log.

The macho expects the new revocation state to already be stored within
the channel. This method is to be called once the remote party revokes
their current commitment state.
2017-11-10 19:50:47 -08:00
Olaoluwa Osuntokun
b89c14616c
channeldb: add new RemoteCommitChainTip method
In this commit, we add a new method: RemoteCommitChainTip. This method
allows callers to poll the database state to check if we have an
un-acked commitment for the remote party. If so, then it should be
retransmitted once a communication channel has been re-established with
the channel peer. This method will return ErrNoPendingCommit if we
don’t currently have a dangling commitment.
2017-11-10 19:50:46 -08:00
Olaoluwa Osuntokun
893cda3ac2
channeldb: add new AppendRemoteCommitChain method
In this commit, we add a new method AppendRemoteCommitChain. This
method is meant to be used once we extend a new state to the remote
party, but before we actually transmit the CommitSig message. With this
method, we store a fully valid CommitDiff on disk which can be used in
the case that we need to retransmit the state to the party as they
didn’t fully receive it.
2017-11-10 19:50:46 -08:00
Olaoluwa Osuntokun
10a54a45fd
channeldb: expand the CommitDiff struct with the addition of LogUpdate
In this commit we finalized the structure of the CommitDiff struct by
adding a set of LogUpdates, and also a valid CommitSig message.

The LogUpdate struct houses a messages that were transmitted and
locked-in with the new commitment state. We include the LogIndex along
with the wire messages itself in order to be able to properly
reconstruct the update log upon restart.

The CommitSig message included should be transmitted after the set of
LogUpdates, and fully covers the new commitment state and any new (or
already present) HTLC’s that were included as part of the state.
2017-11-10 19:50:45 -08:00
Olaoluwa Osuntokun
3e3969eb66
channeldb: modify UpdateCommitment to accept a ChannelCommitment
In this commit, we modify the UpdateCommitment method to accept a full
ChannelCommitment rather than a new transaction, the sig, and a
ChannelDelta. This new structure of this method also takes advantage of
the new bucket structure of the storage schema. Additionally, this
method will now atomically swap in the new passed commitment to point
to the LocalCommitment value within the struct.
2017-11-10 19:50:45 -08:00
Olaoluwa Osuntokun
2dcd82e779
channeldb: add new MarkAsOpen method to OpenChannel struct
In this commit we add a new MarkAsOpen method to the OpenChannel
struct. This method replaces the existing MarkChannelAsOpen method
which targeted the database struct itself.
2017-11-10 19:50:44 -08:00
Olaoluwa Osuntokun
4ae0b008ed
channeldb: modify storage of OpenChannel struct to use new codec.go
In this commit we comptely overhaul the existing storage of the
OpenChannel struct to use the new common serialization defined within
the codec.go file. Additionally, we’ve modified the structure of the
channel database on disk. Rather then use the existing prefix based
segmentation, everything is now bucket based. This has resulted in much
simpler and easier to follow code. The bucket progression is:
openChannelBucket -> nodeBucket -> chainBucket -> channelBucket. We add
a chainBucket as it’s possible that in the future we may have several
channels on distinct chains with a given node.

With the above changes, we’re able to delete much of the existing code
within the file, drastically reducing its size.
2017-11-10 19:50:44 -08:00
Olaoluwa Osuntokun
040c6bdf22
channeldb: add new HtlcIndex and LogIndex fields to the HTLC struct
By adding these two fields, it is now possible to fully reconstruct the
channel’s update log from the set of HTLC’s stored on disk, as we now
properly note both the log index and HTLC index. Prior to this commit
we would simply start the new log index based on the amount of HTLC’s
that were present in the prior state.
2017-11-10 19:50:43 -08:00
Olaoluwa Osuntokun
fb8ff256d8
channeldb: restructure the OpenChannel struct to use two ChannelCommitments
In this commit, we restructure the OpenChannel struct to used two
distinct ChannelCommitments: one for the remote party, and one for the
local party. With this, we now partition the local state and the remote
state. Previously, we weren’t properly tracking the state of the remote
party. As a result, there were certain classes of failures we were
unable to properly recover from. With this separation, we can now
handle them, and the OpenChannel struct is slimmer and more
understandable.
2017-11-10 19:50:43 -08:00
Olaoluwa Osuntokun
4bc317fbbe
channeldb: add new ChannelCommitment struct
In this commit we’ve added a new struct to the package:
ChannelCommitment. This sturct houses all the common data the comprises
a particular commitment state. This will soon replace the open fielded
commitmetn fields within the OpenChannel struct.
2017-11-10 19:50:35 -08:00
Olaoluwa Osuntokun
e3cdbcac0c
channeldb: add new codec.go file which houses common serialization funcs
In this commit, we add a new files to the channeldb package: codec.go.
This file is similar to the way that serialization is done within the
lnwire package. The goal of this file is to reduce all the duplication
within the common serialization methods across the package.
2017-11-09 19:44:37 -08:00
Andrey Samokhvalov
9247168c5d not finished index persistence 2017-11-09 16:38:58 -08:00
Andrey Samokhvalov
28dd6e5d84 channeldb+lnwallet: add remote pending commiment persistance 2017-11-09 16:38:58 -08:00
Andrey Samokhvalov
7aa7cb1caf channeldb: add persistance of the onion blob
In this commit htlc channeldb representation have been augmented
with onion blob field, and (de)serialisaion functions have been changed
to make the onion blob persistant.
2017-11-09 16:38:58 -08:00
Johan T. Halseth
b26560e0f4
channeldb: add DisconnectBlockAtHeight
This commit adds the method DisconnectBlockAtHeight to the channel
graph database, making it possible to "rewind" the database in case
a block is disconnected from the main chain. To accomplish this,
a prune log is introduced, making it possible to keep track of the
point in time where the database was pruned. This is necessary for
the case where lnd might wake up on a stale branch, and must "walk
backwards" on this branch after it finds a common block fro the
graph database and the new main chain.
2017-11-03 00:05:19 +01:00
Jim Posen
a13ad0a339 multi: Fix various typos. 2017-10-25 13:20:54 -07:00
Jim Posen
9fd77a6e40 multi: Update lnd to use new feature vector API. 2017-10-17 22:47:20 -07:00
Olaoluwa Osuntokun
5eb3406b19
channeldb: add new Database method to ChannelGraph 2017-10-10 22:19:26 -07:00
Olaoluwa Osuntokun
81cd954cfc
discovery: don't prune *our* channels during retransmission tick
This commit modifies the recently modified logic for self-channel
retransmission to exclude pruning *our* channels which haven’t been
updated since the broadcastInterval. Instead, we only re-broadcast
channels of ours that haven’t been updated in 24 hours.
2017-10-04 20:46:08 -07:00
Laura Cressman
6253419ce5 channeldb: use binary.Read/Write in waitingproof.go
Use binary.Read/Write when reading and writing booleans
in Encode and Decode functions that operate on the
WaitingProof struct.
2017-10-02 23:13:47 -07:00
Laura Cressman
3f6a58f967 channeldb: use binary.Read/Write in invoices.go
Use binary.Read/Write in when reading and writing
booleans in functions to serialize and deserialize
invoices.
2017-10-02 23:13:47 -07:00
Laura Cressman
156772d04a channeldb: use binary.Read/Write with bools in channel.go
Use binary.Read/Write in functions to serialize and deserialize
channel close summary and HTLC boolean data, as well as in
methods to put and fetch channel funding info. Remove lnd
implementations of readBool and writeBool as they are no
longer needed. Also fix a few minor typos.
2017-10-02 23:13:47 -07:00
Johan T. Halseth
5ed31b1030
channeldb: store payment request with invoices 2017-09-27 12:26:04 +02:00
Olaoluwa Osuntokun
5ff82c92da
channeldb: add commitment transaction fee to ChannelSnapshot 2017-09-24 20:09:28 -07:00
halseth
8693e8babc channeldb: check chanIndex bucket for nilness instead of edges 2017-09-19 05:40:29 +02:00
Dave Collins
916ab454c1 channeldb: Fix payment serialization tests.
This modifies the tests that deal serializing the Invoice type to limit
the creation date to seconds since Go1.9 added the concept of a
monotonic component to times which does not round trip through
MarshalBinary and UnmarshalBinary and therefore causes the tests to fail.

In particular, it modifies the creation dates in the randInvoice,
makeFakePayment, makeRandomFakePayment, and TestInvoiceWorkflow
functions.

This results in allowing TestOutgoingPaymentSerialization,
TestOutgoingPaymentWorkflow, and TestInvoiceWorkflow to pass.
2017-08-30 18:57:39 -07:00
Conner Fromknecht
6ffe33f01a lnw+ba+utxon+cdb: reverts OutPoint and TxOut encoding 2017-08-25 17:56:50 -07:00
Conner Fromknecht
4cdce1fc0a breacharbiter+channeldb: resolves rebase conflicts 2017-08-25 17:56:50 -07:00
Philip Hayes
f0aa186a56 channeldb+utxonursery+lnwire: use lnwire's OutPoint,TxOut serialization 2017-08-25 17:56:50 -07:00
Olaoluwa Osuntokun
460c2b0ec1
channeldb: update test to match recent API changes 2017-08-22 00:52:28 -07:00
Olaoluwa Osuntokun
76302a136b
channeldb: in node.ForEachChannel return incoming/outgoing edges
This commit modifies the node.ForEachChannel to *also* return the
incoming edge as well as the outgoing edge. We make this modification
as when we’re doing path finding, we need to return the incoming edge
as well, since we need to use the to properly compute the time lock and
fees for transit on that edge.
2017-08-22 00:52:25 -07:00
Olaoluwa Osuntokun
35d9dc8092
channeldb: add ChainHash to ChannelEdgeInfo to match BOLT-0007 2017-08-22 00:52:22 -07:00
Olaoluwa Osuntokun
59aae249dc
channeldb: use lnwire.MilliSatoshi within the ChannelEdgePolicy struct 2017-08-22 00:52:20 -07:00
Olaoluwa Osuntokun
74470cf7d0
channeldb: update ChannelConstraints+OpenChannel to use lnwire.MilliSatoshi 2017-08-22 00:52:17 -07:00
Olaoluwa Osuntokun
251f8d650f
channeldb: update the Invoice struct to use lnwire.MilliSatoshi 2017-08-22 00:52:15 -07:00
Olaoluwa Osuntokun
063525c6e0
channeldb: update the OutgoingPayment struct to use lnwire.MilliSatoshi 2017-08-22 00:52:12 -07:00
Olaoluwa Osuntokun
8abba7eafc
channeldb: add TODO to add storage for onion adds 2017-08-10 21:15:55 -07:00
Johan T. Halseth
849d0b93b1 fundingManager: persist state in opening process.
Persists the state of a channel opening process after funding
transaction is confirmed. This tracks the messages sent to
the peer such that the process can be continued in case of a
restart. Also introduces that the receiver side forgets about
channel if funding transaction is not confirmed in 48hrs.
2017-08-10 13:14:25 -07:00
Johan T. Halseth
bd0465ee1d channeldb: support adding partial LightningNodes to graph.
Adds a HaveNodeAnnouncement field to the LightningNode
struct, which is used to indicate if we have gotten
all the necessary information to fill the remaining
fields in the struct. If we haven't gotten a node
announcement for this specific node, then we only
know the pubkey, and can only fill that field in
the struct. Still, we should be able to add it to the
channel graph and use it for routes, as long as we
know about channels to this node.
2017-08-02 15:58:58 -07:00
Olaoluwa Osuntokun
88c15abeef
channeldb: remove RevocationDelay field from the HTLC struct
This commit removes the RevocationDelay field from the HTLC struct as
with the latest commitment transaction scheme, it is no longer needed.
This is due to the fact the the delay is now observed when an on-chain
HTLC claim is attempted, rather than from Shane the HTLC itself has
been broadcast.
2017-07-30 17:48:32 -07:00
Olaoluwa Osuntokun
5ad6c23848
channeldb: update tests to reflect latest OpenChannel/HTLC API changes 2017-07-30 17:48:26 -07:00
Olaoluwa Osuntokun
5854ffb644
channeldb: add InsertNextRevocation method to OpenChannel
This commit adds a new method to the OpenChannel struct:
InsertNextRevocation. This functionality is required in order to
properly implement the new funding workflow and commitment transaction
state machine.

In the prior state machine, we utilized the “initial revocation window”
which would provide both sides with the necessary revocation keys that
they needed to create new commitment states. The version of the
commitment state machine within the specification has been simplified a
bit, and instead essentially incorporates a revocation window of 1. The
window is initially filled at the _opening_ of the commitment
transaction, then never extended, only “slid down” from there on.
2017-07-30 17:48:22 -07:00
Olaoluwa Osuntokun
a14072a47e
channeldb: store the second-level HTLC signature along with HTLC's
This commit modifies the HTLC struct within the channeldb struct in
order to accommodate the additional storage that’s now required as a
result of the change to 2-of-2 HTLC covenant transactions within the
commitment transaction.
2017-07-30 17:48:19 -07:00
Olaoluwa Osuntokun
9811b404e4
channeldb: integrate ChannelConfig and ChannelConstraints into OpenChannel
This commit is the next phase in a rather major overhaul of the
OpenChannel struct. With this commit we remove the old ours vs theirs
semantics with channel related state variables, and instead encapsulate
all local vs remote state into a ChannelConfig structure for each party
within the channel.
2017-07-30 17:48:16 -07:00
Olaoluwa Osuntokun
5780992f8d
channeldb: eliminate ChanID in favor of FundingOutpoint
This commit eliminates a bit of redundancy within the OpenChannel
struct by eschewing the ChanID field, in favor of simply the
FundingOutpoint field.
2017-07-30 17:48:13 -07:00
Olaoluwa Osuntokun
a092b0d82c
channeldb: introduce ChannelConfig and ChannelConstraints
This commit introduces two new structs to the channeled package:
ChannelConfig and ChannelConstraints.

The ChannelConfig struct will eliminate many of the fields on the
OpenChannel struct (ours vs theirs), in favor of having a single config
for both sides (local and remote).

The ChannelConstraints struct will house the channel flow control and
HTLC policies for both sides. The fields of this struct will be used to
control the flow of HTLC’s within the channel. In the occasion that any
of these constraints are violated, either the connection, or the
channel entirely is to be failed.
2017-07-30 17:48:09 -07:00
Olaoluwa Osuntokun
1fb9d2e44e
channeldb: reset database version back to zero
As we removed the prior migration in an earlier commit, we now reset
the version of the database back to zero (no migrations needed).
2017-07-30 17:48:04 -07:00
Olaoluwa Osuntokun
c15cd12dcf
channeldb: fix type in ErrSourceNodeNotSet comment 2017-07-30 17:48:01 -07:00
Olaoluwa Osuntokun
d4916ba9c6
channeldb: remove prior migrations as their no longer needed
This commit removes a database migration that was added to fix a bug
introduced within the first alpha release. This migration is no longer
needed as the underlying database scheme has changed since then.
2017-07-30 17:47:58 -07:00
Johan T. Halseth
ebe05f6568 lnwallet: add update_fee message support.
This commit adds the possibility for the initiator of a
channel to send the update_fee message, as specified
in BOLT#2. After the message is sent and both parties
have committed to the updated fee, all new commitment
messages in the channel will use the specified fee.
2017-07-14 16:39:15 -07:00
Andrey Samokhvalov
8fa2b95c12 lnd: remove seelog logger
The btclog package has been changed to defining its own logging
interface (rather than seelog's) and provides a default implementation
for callers to use.

There are two primary advantages to the new logger implementation.

First, all log messages are created before the call returns.  Compared
to seelog, this prevents data races when mutable variables are logged.

Second, the new logger does not implement any kind of artifical rate
limiting (what seelog refers to as "adaptive logging").  Log messages
are outputted as soon as possible and the application will appear to
perform much better when watching standard output.

Because log rotation is not a feature of the btclog logging
implementation, it is handled by the main package by importing a file
rotation package that provides an io.Reader interface for creating
output to a rotating file output.  The rotator has been configured
with the same defaults that btcd previously used in the seelog config
(10MB file limits with maximum of 3 rolls) but now compresses newly
created roll files.  Due to the high compressibility of log text, the
compressed files typically reduce to around 15-30% of the original
10MB file.
2017-06-25 14:19:56 +01:00
Olaoluwa Osuntokun
1be4d67ce4
multi: run all test instances in parallel 2017-06-17 01:00:07 +02:00
Olaoluwa Osuntokun
4b8d052afc
channeldb: replace opening height will short chan ID in OpenChannel
This commit modifies the OpenChannel struct to include the full short
channel ID rather than simply the opening height. This new field will
be needed by an upcoming change to uniformly switch to using short
channel ID’s when forwarding HTLC’s due to the change in per-hop
payloads.
2017-06-16 22:28:58 +02:00
Olaoluwa Osuntokun
66ba2e862c
channeldb: store the funding transaction broadcast height for channels
This commit expands the field within the OpenChannel struct in order to
start tracking the height that the funding transaction was initially
broadcast. Other sub-systems within lnd can now use this data to give a
more accurate height hint to the ChainNotifier, or to use during the
funding workflow to decide if a channel should be forgotten after it
fails to confirm for N blocks.
2017-06-05 19:15:44 -07:00
Olaoluwa Osuntokun
75d5396445
channeldb: change MinFeePerKb to FeePerKw in OpenChannel struct
This commit modifies the name of a field in the OpenChannel struct to
better reflect its actual usage within this protocol. The FeePerKw
represents the amount of satoshi to be paid as fees per kilo-weight.
This field is set at the opening of a transaction and will be able to
be updated properly via the usage of the update_fee method.
2017-05-16 18:44:01 -07:00
bryanvu
320bed7e6b lnwallet: add CommitFee field to OpenChannel
In order to cleanly handle shutdowns and restarts during state machine operation, the fee for the current
commitment transaction must be persisted. This allows the fee to be
reapplied when the current state is reloaded.
2017-05-15 20:26:11 -07:00
Olaoluwa Osuntokun
68dbbb046b
channeldb: add total time locked balance to CloseChannelSummary
This commit adds the total time locked balance of a closed channel at
closure time to the CloseChannelSummary struct. With this information,
we now provide sub-systems within the daemon further knowledge which
can aide them in properly handling querying for the state of pending
close transactions, or if they should watch a channel for closure or
not.
2017-05-14 19:03:08 -07:00
Olaoluwa Osuntokun
a5113c2439
channeldb+funding: track opening height using MarkChannelAsOpen
This commit modifies the OpenChannel structure on-disk to also track
that opening height of a channel. This change is being made in order to
make and more light client friendly. A follow up commit will modify
several areas of the codebase to use this new functionality.
2017-05-11 15:20:33 -07:00
Olaoluwa Osuntokun
373a1192ce
channeldb: fix panic bug in channeldb, check proper bucket for nil-ness 2017-05-11 15:20:31 -07:00
Olaoluwa Osuntokun
333373f78f
channeldb: add new ChannelView method to the ChannelGraph
This commit introduces a new method to the ChannelGraph struct:
ChannelView. This struct returns all the outpoints that represent the
set of active channels within the network. The set of items returned by
this new method will possibly shrink with each call to `PruneGraph`,
and possibly be expanded by each call to `AddChannelEdge`.

The graph pruning tests have been updated to ensure the description
above holds true.
2017-05-11 15:20:23 -07:00
Andrey Samokhvalov
a3eee453c3 channeldb: add waiting proof storage
In this commit boltdb persistent storage have been added, which allow
as to not loose announcement proofs inside gossiper subsystem.
2017-05-05 17:02:49 -07:00
Olaoluwa Osuntokun
18a2c8a455
channeldb: fix linter errors 2017-05-04 17:46:17 -07:00
Olaoluwa Osuntokun
35fd800083
channeldb: add test for new channel closure persistence 2017-05-04 17:39:05 -07:00
Olaoluwa Osuntokun
826dbbd1f7
channeldb: add new method to db, MarkChanFullyClosed
This commit adds a new method to the database which allows callers to
mark a channel as fully closed once certain conditions have been
reached. If a channel was cooperatively closed, then it can be marked
as fully closed as soon as the channel has been confirmed. If the
channel was marked as force closed, then it should be marked as closed
as soon as _all_ the funds in limbo have been swept.
2017-05-04 17:39:03 -07:00
Olaoluwa Osuntokun
f2f3774cec
channeldb: add new method to db, FetchClosedChannels
This commit builds upon the prior commit by adding a new method to
allows callers to query the state of all closed (fully) and pending
closed channel within the database.
2017-05-04 17:39:00 -07:00
Olaoluwa Osuntokun
bafea35a32
channeldb: expand channel close summaries to include additional data
This commit expands the close summaries within the database to include
additional information such as: how the channel was closed, if the
channel if fully closed, and other balance information. This new
information will be used in an upcoming expansion of the
pendingChannels RPC to display more comprehensive data w.r.t where the
node’s funds are and how soon they’ll be fully available.

The data stored in the closeChannel bucket has been expanded to include
the above. Additionally the CloseChannel method on the OpenChannel
struct now takes in the additional information that details how and who
it was closed by.

This commit borrows heavily from the initial work in #179 by @halseth.
2017-05-04 17:38:58 -07:00
Olaoluwa Osuntokun
b96b180b0b
channeldb: modify ForEachNode/ForEachChannel to accept a db txn
This commit modifies the ForEachNode on the ChannelGraph and
ForEachChannel on the LightningNode struct to accept a database
transaction as its first argument. With this change, it’ll now be
possible to implement graph traversals that typically required a nested
loop with all the vertex loaded into memory using the callback API
instead:
c.ForEachNode(nil, func(tx, node) {
    node.ForEachChannel(tx, func(…) {
    })
})
2017-04-14 13:14:09 -07:00
Olaoluwa Osuntokun
fe3c3642e2
channeldb: use the Batch method when writing payment details
This commit implements an easy optimization by using bolt db’s Batch
method when writing payment details to disk. The AddPaymnent method can
be concurrently called by thousands of grouting due to the way the
payment dispatch pipeline is architected. With this commit, we shave of
a significant amount of running time when users are sending thousands
of payments a second as what would’ve been thousands of writes can now
be coalesced into one or two writes!
2017-04-11 22:02:09 -07:00
Andrey Samokhvalov
07076cce21 routing+channeldb: add AddProof method
Originally we adding the edge without proof in order to able to use it
for payments path constrcution. This method will allow us to populate
the announcement proof after the exchange of the half proofs and
constrcutrion of full proof finished.
2017-03-29 19:49:05 -07:00
Andrey Samokhvalov
19174ebdfd channeldb: add storing of node signature and add edge signature
In order to properly announce the channel the announcements proofs
should be persistent in boltdb.
2017-03-29 19:49:05 -07:00
Andrey Samokhvalov
2105a06a26 channeldb: add ability to store empty auth proof
In case if the channel shouldn't be announced to the rest of the network
the proof, which is needed to announce the channel, will not be
populated, fot that reason the ability to store the empty proof has
been added.
2017-03-29 19:49:05 -07:00
Andrey Samokhvalov
7a9a52c7b9 channeldb: add ErrEdgeAlreadyExist error 2017-03-29 19:49:05 -07:00
bryanvu
085b7333cb lnwire: add support for Features in NodeAnnouncement
Add support for Features in NodeAnnouncment according to spec.
2017-03-29 12:03:43 -07:00
bryanvu
654c5ea61a config: added support, tests for --externalip config option
Minor change to server.go to add ExternalIPs to
channeldb.LightningNode. Also, added a test that utilizes this
functionality and exercises multiple addresses in NodeAnnouncement.
2017-03-29 12:03:43 -07:00
bryanvu
9ffac9eae1 lnwire: update NodeAnnouncement to handle multiple addresses
This commit modifies address handling in the NodeAnnouncement struct,
switching from net.TCPAddr to []net.Addr. This enables more flexible
address handling with multiple types and multiple addresses for each
node. This commit addresses the first part of issue #131 .
2017-03-29 12:03:43 -07:00
Olaoluwa Osuntokun
6b3a258e86
multi: fix formatting issues in packge README's 2017-03-27 16:25:25 -07:00
Olaoluwa Osuntokun
c9511da22f
channeldb: add new method to channel state, RevocationLogTail
This commit adds a new method to the channel state: RevocationLogTail.
This new method will return the information concerning the latest
revoked state of the remote party’s commitment chain.

This new data can be used to properly initialize the states of the
in-memory commitment chains on node start up.
2017-03-24 16:09:14 -07:00
Olaoluwa Osuntokun
f217093c00
multi: replace usage of fastsha256 with crypto/sha256
This commit removes all instances of the fastsha256 library and
replaces it with the sha256 library in the standard library. This
change should see a number of performance improvements as the standard
library has highly optimized assembly instructions with use vectorized
instructions as the platform supports.
2017-03-15 18:56:41 -07:00
Andrey Samokhvalov
61991a1c89 lnd: fix latest goclean.sh lint warning 2017-03-13 16:30:23 -07:00
Andrey Samokhvalov
ee2379775c lnd: fix golint warning which requires to add additional comments 2017-03-13 16:30:23 -07:00
Andrey Samokhvalov
fd97a4bd19 lnd: partially fix golint warnings 2017-03-13 16:30:23 -07:00
Andrey Samokhvalov
143a6e01bb lnd: fix unconvert warnings 2017-03-13 16:30:23 -07:00
Andrey Samokhvalov
f2843dd4c9 lnd: fix gofmt warnings 2017-03-13 16:30:23 -07:00
Andrey Samokhvalov
8fb54782e2 lnd: fix gosimple warnings 2017-03-13 16:30:23 -07:00
Olaoluwa Osuntokun
7bdf02bc9e
routing: modify path finding routines use new EdgeInfo/EdgePolicy
This commit modifies the path finding routines to properly use the new
channel edge related API exposed by the database. Additionally, a new
type `ChannelHop` has been introduced which couples an edges routing
policy with the capacity and origin chain of the channel.
2017-03-08 14:25:05 -08:00
Olaoluwa Osuntokun
39e145f037
channeldb: update tests to recent API change (edge struct split) 2017-03-05 19:22:33 -06:00
Olaoluwa Osuntokun
deceae8fe4
channeldb: PruneGraph now returns the channels closed
This commit alters the return value of PrunedGraph to me a bit more
useful: the function now returns all the channels that were closed when
processing the block (slice of spent outpoints). With this information,
callers gain greater visibility into exactly which channels were
closed. This can be used in higher levels to present detailed summaries
of how blocks affect closed channels.
2017-03-05 19:20:57 -06:00
Olaoluwa Osuntokun
2c29a20a8d
channeldb: split ChannelEdge into two structs, info and policy
This commit splits the prior ChannelEdge struct into two distinct
structs: ChannelEdgeInfo and ChannelEdgePolicy. The info structs stores
the _full_ information that was used to advertise the channel, while
the policy struct contains the information that’s needed in order to
use the information for routing purposes.

With this split we can eliminate a number of hacks within the rest of
the codebase that were added as a result of data unavailability if one
or neither edge was present.

Finally a bit of field renaming has taken place (Exipiry ->
TimeLockDelta), etc.
2017-03-05 19:18:34 -06:00
Olaoluwa Osuntokun
9adc5f6484
lnwallet+channeldb: update callers to shachain API changes 2017-02-24 16:31:45 -08:00
bryanvu
59763ebc7e channeldb: persist numConfsRequired in OpenChannel.
When a pending channel is persisted and then reloaded upon system startup
it's necessary to also persist the number of confirmations that will be required
before the pending channel can be opened.
2017-02-24 11:37:33 -08:00
bryanvu
3e02ea11ef channeldb: added isPending flag and queries
In order to facilitate persistence during the funding process, added
the isPending flag to channels so that when the daemon restarts, we can
properly re-initialize the chain notifier and update the state of
channels that were going through the funding process.
2017-02-24 11:37:33 -08:00
Andrey Samokhvalov
f86557c3e4 channeldb+lnwallet: replace elkrem with shachain
In this commit the initial implementation of revocation hash
generation 'elkrem' was replaced with 'shachain' Rusty Russel
implementation which currently enshrined in the spec. This alghoritm has
the same asymptotic characteristics but has more complex scheme
to determine wish hash we can drop and what needs to be stored
in order to be able to achive full compression.
2017-02-22 16:49:29 -08:00
Christopher Jämthagen
97aa478ce1 channeldb: change [4]byte obsfucator to [6]byte in test 2017-02-22 15:46:59 -08:00
Christopher Jämthagen
873211c02f Use 48-bit commitment transaction numbers
Fix SetStateNumHint and GetStateNumHint to properly
set and get the stateNumHints using the lower 24 bits
of the locktime of the commitment transaction as the
lower 24 bits of the obfuscated state number and the
lower 24 bits of the sequence field as the higher 24
bits.
2017-02-22 15:46:59 -08:00
Olaoluwa Osuntokun
bcde70e74a
channeldb: fix bug when writing revocation log states
This commit fixes a bug that was introduced when we moved to using
64-bit integers for storing the revocation log state. When we made this
change, we forgot to increase the size of the buffer which stores the
key for the particular channel state from 40 to 44 bytes to account for
the 4 additional bytes in the new 64-bit integer.

This bug has been fixed by properly sizing the key buffer. We’ve also
added an additional test to ensure that we retrieve the proper state
after multiple state updates.
2017-02-21 01:43:31 -08:00
Olaoluwa Osuntokun
db8c6d4cf4
channeldb: modify the NumUpdates field in ChannelDelta to be a uint64
This commit modifies the running update count within all ChannelDelta’s
to track the number of updates using a uint64 rather than a uint32.
This change reflects the fact that the obsfucated commitment hints are
to be encoded using a 48-bit integer, rather than a 32-bit integer.
2017-02-21 01:42:44 -08:00
Olaoluwa Osuntokun
59615b3cb2
channedb: add db migration for databases before delivery script bug fix 2017-02-08 13:05:45 -08:00
Olaoluwa Osuntokun
e8e53a4b91
channeldb: fix channel corruption bug upon channel closure
This commit fixes a bug which would previously lead to corruption of
the channel state when a node had one or more channels open and one of
them was closed either forcibly or cooperatively. The source of the bug
itself as a typo: rather than using the construed `deliveryKey`
variable to fetch/put/delete the delivery scripts, `deliveryScriptsKey`
(the key prefix itself) as used. This bug would cause the database to
be unable to read _any_ channel from the database after one was
deleted, as each channel would actually be reading/writing-to the
_exact same_ delivery script.

The fix for the bug itself is simple: eliminate the typo.
2017-02-07 19:54:51 -08:00
Olaoluwa Osuntokun
7f98e8e5f1
channeldb: fix warnings found by go vet 2017-02-07 16:52:04 -08:00
Olaoluwa Osuntokun
5889331422
channeldb: improve error messages when unable to read channel 2017-02-07 16:41:20 -08:00
Olaoluwa Osuntokun
d87e795e2a
channeldb: ensure all state is deleted for a channel by CloseChannel
This commit addresses some lingering TODO’s which ensure that related
state to a channel is properly deleted by the CloseChannel method.
Previously the values for the respective dust-limits of either side,
the on-disk HTLC’s, and any entries the revocation log for the channel
weren’t being properly deleted.

Additionally, we now modify the checks within the unit tests to ensure
that we can still read the channel from disk w/o running into an error
(thought the slice will be blank), and also the the revocation log is
properly garbage collected.
2017-02-07 16:40:46 -08:00
Olaoluwa Osuntokun
aeb7cce7c3
channeldb: move serialization of outpoint into db txn in CloseChannel 2017-02-02 16:59:22 -08:00
Olaoluwa Osuntokun
f623729b64
channeldb: fix panic in query for nonexistent channel ID
This commit fixes a panic that would arise when the daemon attempts to
query for a channel that doesn’t currently exist. The bug was the
result of a typo which checked for the nil existence of the incorrect
variable.
2017-01-22 14:28:36 -08:00
Trevin Hofmann
40c7bac3aa multi: fix a variety of typos throughout the repo 2017-01-17 17:02:56 -08:00
Olaoluwa Osuntokun
7a36fb4562
channeldb: fix assumption that both channel edges will always be advertised
This commit fixes a prior bug in the graph database due to an invalid
assumption that both channel edges would _always_ be advertised. This
assumption is invalid, as it’s up to a node’s policy if the advertise
their direction of the channel.

The fix for this assumption is straight forward: ErrEdgeNotFound is no
longer a critical error, instead a nil pointer will now be passed into
the passed callback function.
2017-01-17 13:01:19 -08:00