Commit Graph

330 Commits

Author SHA1 Message Date
Joost Jager
6c918a1806 channeldb: store unknown policies in database
The commit ensures that for every channel, there will always
be two entries in the edges bucket. If the policy from one or
both ends of the channel is unknown, it is marked as such.

This allows efficient lookup of incoming edges. This is
required for backwards payment path finding.
2018-08-09 20:23:40 -07:00
Olaoluwa Osuntokun
8379bbaa9b
Merge pull request #1673 from cfromknecht/prevent-db-reversions
channeldb: fail Open on db reversion
2018-08-07 15:47:02 -07:00
Conner Fromknecht
cf2c371042
multi: fix linting errors 2018-08-02 18:20:50 -07:00
Conner Fromknecht
1ded697e8d
multi: sort import paths with gofmt 2018-08-02 18:20:49 -07:00
Conner Fromknecht
dff7706d04
channeldb/meta_test: adds TestMigrationReversion
Adds a test asserting that we will fail to open
a channeldb with a version higher than the
highest known version to the current build of
lnd.
2018-08-02 18:18:49 -07:00
Conner Fromknecht
e23fc40d63
channeldb/db: check for db reversions on startup 2018-08-02 16:46:17 -07:00
Conner Fromknecht
ee34b9c7b0
channeldb/error: adds ErrDBReversion error
Adds an error for signaling that we detected
the case where a user is trying to revert
to prior version before a db migration.
2018-08-02 16:45:03 -07:00
Olaoluwa Osuntokun
d129e7c890
channeldb: update ChannelView to return new EdgePoint struct
In this commit, we update the ChannelView method to be compatible with
the new set of interfaces that require the script to be passed in in
addition to the outpoint. In order to do this, we introduce a new
EdgePoint struct which packages together a channel point along with the
funding pkScript. Along the way, we've copied over a utility method from
the lnwallet package to avoid having to deal with an import cycle.
2018-07-31 21:28:54 -07:00
Johan T. Halseth
6cdf0e2d6e
channeldb/channel: methods for marking borked+dataloss commitPoint in db 2018-07-31 15:16:22 +02:00
Johan T. Halseth
ea6aca26a5
channeldb: make chanStatus unexported
Since the ChanStatus field can be changed from concurrent callers, we
make it unexported and add the method ChanStatus() for safe retrieval.
2018-07-31 15:07:30 +02:00
Olaoluwa Osuntokun
ef4c7d2a78
channeldb: add new TestAddChannelEdgeShellNodes test 2018-07-22 21:05:52 -07:00
Olaoluwa Osuntokun
2e75499787
channeldb+routing: move adding shell nodes into db txn of AddChannelEdge
In this commit, we fix a slight race condition that can occur when we go
to add a shell node for a node announcement, but then right afterwards,
a new block arrives that causes us to prune an unconnected node. To
ensure this doesn't happen, we now add shell nodes within the same db
transaction as AddChannelEdge. This ensures that the state is fully
consistent and shell nodes will be added atomically along with the new
channel edge.

As a result of this change, we no longer need to add shell nodes within
the ChannelRouter, as the database will take care of this operation as
it should.
2018-07-22 21:02:53 -07:00
Olaoluwa Osuntokun
8519ea869d
channeldb: add logging for nodes pruned from the channel graph 2018-07-22 21:00:51 -07:00
Olaoluwa Osuntokun
6bf15e8f2b
channeldb: fix bug in pruneGraphNodes by switching to using ref counting
In this commit, we fix an existing bug in the pruneGraphNodes method.
Before this commit, if a node was involved in a channel, but only one of
the edges was advertised, then either it, or the other node would be
erroneously pruned from the graph. They shouldn't be pruned as there's
still an edge connecting the, although only 1/2 of the edge is actually
advertised.

In order to fix this, we'll now do two passes: the first pass will
populate a ref count map of all known nodes in the graph, the second
pass will increment the ref count each time a node is found in the
graph. With this two pass method, we ensure that nodes are only deleted
if there are absolutely no edges pointing to them within the graph.
2018-07-22 18:59:36 -07:00
Olaoluwa Osuntokun
131eea4960
channeldb: modify TestPruneGraphNodes to test nodes w/ single edge
In this commit, we extend the TestPruneGraphNodes test to also test the
case of when a node is involved in a channel, but only a single edge for
that channel has been advertised. In order to test this, we add an
additional node to the graph, and also a new channel. However, this
channel will only have a single edge advertised. As result, when we
prune the set of edges, the only node remaining should be the node that
didn't have any edges at all.
2018-07-22 18:57:18 -07:00
Olaoluwa Osuntokun
0645261e5e
channeldb: add nwe PruneGraphNodes method to prune unconnected nodes 2018-07-21 19:49:59 -07:00
Olaoluwa Osuntokun
3a465c64b5
channeldb: modify pruneGraphNodes to prune nodes if no edges exist 2018-07-21 19:49:16 -07:00
Olaoluwa Osuntokun
bca926d6af
Merge pull request #1371 from wpaulino/prune-link-nodes
server: prune link nodes without any open channels
2018-07-21 18:55:38 -07:00
Wilmer Paulino
0aa1f39af8
channeldb+server: prune link nodes on startup
In this commit, we extend the server's functionality to prune link nodes
on startup. Since we currently only decide whether to prune a link node
from the database based on a channel close, it's possible that we have
link nodes lingering from before this functionality was added on.
2018-07-17 17:35:31 -07:00
Wilmer Paulino
0e4d350e56
channeldb: prune nodes with no open channels left from the graph 2018-07-17 17:35:29 -07:00
Wilmer Paulino
4578eec8a1
channeldb: garbage collect link nodes with no open channels remaining 2018-07-17 17:35:27 -07:00
Wilmer Paulino
959618d596
channeldb: refactor methods to allow using existing db transaction 2018-07-17 17:35:26 -07:00
Wilmer Paulino
044e81bbd4
channeldb: add DeleteLinkNode method 2018-07-17 17:35:25 -07:00
Olaoluwa Osuntokun
5ee368f0f9
channeldb: ensure that we return an invoice on duplicate settles
In this commit, we fix an existing bug related to duplicate invoice
settle.s Before this commit, the second (and later) times an invoice was
settled we would return a nil pointer. This would result in the new
invoiceRegistry panicing as it would go to attempt to notify with a nil
invoice.

We fix this by returning the invoice on disk (unmodified) for each
settle after the initial one.

Fixes #1568.
2018-07-17 16:28:08 -07:00
Olaoluwa Osuntokun
509998db86
channeldb: add new test for duplicate invoice settles
In this commit, we add a new test to ensure that duplicate invoice
settles work as expected. At the present time, this test will fail as
the second to last assertion fails as we'll return a nil invoice the
second time around.
2018-07-17 16:27:04 -07:00
Olaoluwa Osuntokun
6f60f139f4 multi: switch over import paths from roasbeef/* to btcsuite/* 2018-07-13 17:05:39 -07:00
Olaoluwa Osuntokun
19d84dd1cc
channeldb: add new migration to finalize invoice migration for outgoing payments
In this commit, we migrate the database away from a partially migrated
state. In a prior commit, we migrated the database in order to update
the Invoice struct with three new fields: add index, settle index, paid
amt.  However, it was overlooked that the OutgoingPayment struct also
embedded an Invoice within it. As a result, nodes that upgraded to the
first migration found themselves unable to start up, or call
listpayments, as the internal invoice within the OutgoignPayment hadn't
yet been updated.  This would result in an OOM typically as we went to
allocate a slice with a integer that should have been small, but may
have ended up actually being a set of random bytes, so a very large
number.

In this commit, we finish the DB migration by also migrating the
internal invoice within each OutgoingPayment.

Fixes #1538.
Fixes #1546.
2018-07-12 00:22:28 -07:00
Olaoluwa Osuntokun
515ba7a4d0
channeldb: fix formatting for invoice migration log 2018-07-11 17:05:56 -07:00
Olaoluwa Osuntokun
db4a09d3b5
channeldb: skip sub-buckets during invoice time series migration
In this commit, we fix an existing bu gin the invoice time series
migration code. Before this commit, the migration would fail as we would
try to migrate an empty invoice. We now detect this case and skip all
empty invoices.

We also add a bit more logging on both the info and trace logging level.
2018-07-06 16:45:02 -07:00
Olaoluwa Osuntokun
7aeed0b58f
channeldb: AddInvoice now returns the addIndex of the new invoice 2018-07-06 12:22:01 -07:00
Olaoluwa Osuntokun
e5c579120e
channeldb: modify SettleInvoice to also return the invoice being settled 2018-07-06 12:22:00 -07:00
Olaoluwa Osuntokun
2097564307
channeldb: add new migration to upgrade old databases to new invoice time series index 2018-07-06 12:21:58 -07:00
Olaoluwa Osuntokun
24d97b5ee5
channeldb: add new test to exercise invoice time series index 2018-07-06 12:21:58 -07:00
Olaoluwa Osuntokun
532975588d
channeldb: update existing invoice tests to due to recent field additions 2018-07-06 12:21:57 -07:00
Olaoluwa Osuntokun
24ae13d3a5
channeldb: add two methods to allow seeking into the invoice time series
In this commit, we add two new methods: InvoicesAddedSince and
InvoicesSettledSince. These methods will be used by higher level
sub-systems that implement notifications to deliver any notifications
backlog based on the last add index, and last settle index that the
client knows of.

It's important to note that care has been taken to ensure that this new
API can be used in a backwards compatible manner. If a client specifies
and index of 0 for either of the methods, then no backlog will be sent.
This is due to the fact that current users of the API don't expect any
backlog notifications to be sent. Additionally, the index actually
starts at 1, instead of 0.
2018-07-06 12:21:57 -07:00
Olaoluwa Osuntokun
5d20c02ea8
channeldb: add new add+settle index to invoice database
In this commit, we add two new indexes to the invoice database: the add
index, and the settle index. These to indexes essentially form a time
series index on top of the existing primary index bucket. Each time an
invoice is added, we'll advance the addIndex seqno, and then create a
mapping from seqNo -> invoiceNum. Each time an invoice is settled, we'll
do the same, but within the settle index.

This change is required in order to allow callers to effectively seek
into the current invoice database in order to obtain notifications for
any invoices they may have missed out on while they were disconnected.
This will allow us to implement robust streaming invoice notifications
within lnd to ensure that clients never miss an event.
2018-07-06 12:21:56 -07:00
Olaoluwa Osuntokun
fc0f0d33b2
channeldb: add new AmtPaid field to the Invoice struct
In this commit, in order to allow the caller to specify the amount that
was ultimately accepted for an invoice, the SettleInvoice method has
gained a new parameter: amtPaid. SettleInvoice will now populate the
final amount paid in the database upon db commit.
2018-07-06 12:21:56 -07:00
Olaoluwa Osuntokun
dbf7e4525a
lntest: ensure we always cancel active streaming notifications 2018-07-06 12:21:54 -07:00
Conner Fromknecht
75d7436f7a
channledb/codec: add net.Addr and []net.Addr to codec 2018-07-03 17:07:10 -07:00
Conner Fromknecht
4cb7953f66
channledb/addr: remove silent encoding failure for unknown types 2018-07-03 17:07:10 -07:00
Conner Fromknecht
c6c442f6b6
channledb/addr_test: add negative test for unknown types 2018-07-03 17:07:10 -07:00
Conner Fromknecht
21674c05e5
channeldb/codec: add concrete error for unknown types 2018-07-03 17:07:09 -07:00
Conner Fromknecht
1d5189bd25
channeldb/forwarding_log: use public Read/WriteElements 2018-07-03 17:07:09 -07:00
Conner Fromknecht
30ff91913d
channeldb/channel: use public Read/WriteElements 2018-07-03 17:07:09 -07:00
Conner Fromknecht
ec029ac7fd
channeldb/codec: exposes Read/WriteElements 2018-07-03 17:07:09 -07:00
Olaoluwa Osuntokun
e5f802c33c Revert "channeldb: explicitly store the FinalCltvDelta within the ContractTerm struct"
This reverts commit 8dcfeeaef507ebe02c60e34022f614a65a18b050.
2018-06-29 12:40:42 -07:00
Olaoluwa Osuntokun
8dcfeeaef5 channeldb: explicitly store the FinalCltvDelta within the ContractTerm struct
In this commit, we move to explicitly storing a bit more information
within the invoice. Currently this information is already stored in the
payment request, but by storing it at this level, callers that may not
be in the state to fully decode a payment request can obtain this data.

We avoid a database migration by appending this data to the end of an
invoice. When decoding, we'll try to read out this extra information,
and simply return what we have if it isn't found.
2018-06-26 19:49:49 -07:00
Conner Fromknecht
56e5eed037
channeldb/channel: update short chan id for fwd packager 2018-06-19 13:48:08 +01:00
Conner Fromknecht
639c9875b2
channeldb/channel_test: test packager source updated 2018-06-19 13:48:08 +01:00
Conner Fromknecht
086b44c1f4
channeldb/waitingproof: improve external consistency of store
This commit synchronizes the in-memory cache with the
on-disk state to ensure the waiting proof store is
externally consistent. Currently, there are scenarios
where the in-memory state is updated, and not reverted
if the write fails. The general fix is to wait to apply
modifications until the write succeeds, and use a
read/write lock to synchronize access with db operations.
2018-06-05 21:33:47 -07:00