Commit Graph

43 Commits

Author SHA1 Message Date
maurycy
ac24b12bf2
multi: fix various typos in comments 2018-09-07 06:51:49 +02:00
Olaoluwa Osuntokun
d2a7d910b7
Merge pull request #1825 from Roasbeef/extra-gossip-message-data
channeldb+discovery: ensure we store, validate and propagate announcements with opaque data
2018-09-05 17:53:02 -07:00
Olaoluwa Osuntokun
16b5a67c3a
Merge pull request #1842 from wpaulino/dedup-chan-updates-horizon
channeldb: dedup channel edges returned from ChanUpdatesInHorizon
2018-09-04 21:24:40 -07:00
Olaoluwa Osuntokun
a35bdd4233
channeldb: extend set of graph tests to include opque data where pertinent 2018-09-04 20:52:43 -07:00
Wilmer Paulino
d3cf3168d2
channeldb/graph_test: ensure policies for an edge have different
timestamps

In this commit, we ensure policies for edges we create in
TestChanUpdatesInHorizon have different update timestamps. This ensures
that there are two entries per edge in the edge update index. Because of
this, the test will fail because ChanUpdatesInHorizon will return
duplicate channel edges due to looking at all the entries within the
edge update index. This will be addressed in a future commit to allow
the set of tests to pass once again.
2018-09-04 18:36:25 -07:00
Wilmer Paulino
c1633da252
channeldb/graph_test: extend prune edge update index test to update edges
In this commit, we extend TestChannelEdgePruningUpdateIndexDeletion test
to include one more update for each edge. By doing this, we can
correctly determine whether old entries were properly pruned from the
index once a new update has arrived.
2018-09-04 18:33:41 -07:00
Wilmer Paulino
8dec659e10
channeldb/graph_test: properly check entries within edge update index
Due to entries within the edge update index having a nil value, the
tests need to be modified to account for this. Previously, we'd assume
that if we were unable to retrieve a value for a certain key that the
entry was non-existent, which is why the improper pruning bug was not
caught. Instead, we'll assert the number of entries to be the expected
value and populate a lookup map to determine whether the correct entries
exist within it.
2018-09-04 18:33:40 -07:00
Olaoluwa Osuntokun
06d03b55a3
channeldb: add new TestNodePruningUpdateIndexDeletion test
In this commit, we add a new test to expose a lurking bug within the
graph database code. As is, when we go to delete a node from the
database, we don't also remove the entries within the update index. As a
result, if a user attempted to call NodeUpdatesInHorizon (or typically
as part of the p2p handshake), we would error out, as we would try to
read a node that no longer existed in the graph, as it was pruned.
2018-08-16 18:20:58 -07:00
Joost Jager
6c918a1806 channeldb: store unknown policies in database
The commit ensures that for every channel, there will always
be two entries in the edges bucket. If the policy from one or
both ends of the channel is unknown, it is marked as such.

This allows efficient lookup of incoming edges. This is
required for backwards payment path finding.
2018-08-09 20:23:40 -07:00
Olaoluwa Osuntokun
d129e7c890
channeldb: update ChannelView to return new EdgePoint struct
In this commit, we update the ChannelView method to be compatible with
the new set of interfaces that require the script to be passed in in
addition to the outpoint. In order to do this, we introduce a new
EdgePoint struct which packages together a channel point along with the
funding pkScript. Along the way, we've copied over a utility method from
the lnwallet package to avoid having to deal with an import cycle.
2018-07-31 21:28:54 -07:00
Olaoluwa Osuntokun
ef4c7d2a78
channeldb: add new TestAddChannelEdgeShellNodes test 2018-07-22 21:05:52 -07:00
Olaoluwa Osuntokun
131eea4960
channeldb: modify TestPruneGraphNodes to test nodes w/ single edge
In this commit, we extend the TestPruneGraphNodes test to also test the
case of when a node is involved in a channel, but only a single edge for
that channel has been advertised. In order to test this, we add an
additional node to the graph, and also a new channel. However, this
channel will only have a single edge advertised. As result, when we
prune the set of edges, the only node remaining should be the node that
didn't have any edges at all.
2018-07-22 18:57:18 -07:00
Olaoluwa Osuntokun
0645261e5e
channeldb: add nwe PruneGraphNodes method to prune unconnected nodes 2018-07-21 19:49:59 -07:00
Wilmer Paulino
0e4d350e56
channeldb: prune nodes with no open channels left from the graph 2018-07-17 17:35:29 -07:00
Olaoluwa Osuntokun
6f60f139f4 multi: switch over import paths from roasbeef/* to btcsuite/* 2018-07-13 17:05:39 -07:00
Olaoluwa Osuntokun
dd4cab054d
channeldb: ensure items from the update index are actually deleted
In this commit, we fix an existing bug in the new graph query sync
feature. Before this commit, when a block is pruned, we would never
actually delete the update index entries. This is due to the fact that
we would attempt to delete the entries from the update index _after_ we
had already removed the edges from the update index.

We fix this by simply swapping the order: first we delete from the
update index, then we delete the edges themselves. A test ensuring that
the entires are cleared (which failed before this commit), has been
added.
2018-06-05 17:30:56 -07:00
Olaoluwa Osuntokun
baed4d1f47
channeldb: add new methods required to implement new discovery.ChannelGraphTimeSeries interface
In this commit, we add a series of methods, and a new database index
that we'll use to implement the new discovery.ChannelGraphTimeSeries
interface interface. The primary change is that we now maintain two new
indexes tracking the last update time for each node, and the last update
time for each edge. These two indexes allow us to implement the
NodeUpdatesInHorizon and ChanUpdatesInHorizon methods. The remaining
methods added simply utilize the existing database indexes to allow us to
respond to any peer gossip range queries.

A set of new unit tests has been added to exercise the added logic.
2018-05-31 16:30:54 -07:00
Dimitris Tsapakidis
4009f7f874 multi: fix typos in comments 2018-04-17 19:03:27 -07:00
practicalswift
663c396235 multi: fix a-vs-an typos 2018-04-17 19:02:04 -07:00
Olaoluwa Osuntokun
800eea931f
build+multi: switch from bolt to bbolt
In this commit, we switch from boltbd/bolt to coreos/bbolt as the
former is no longer being actively maintained.
2018-03-10 19:01:13 -08:00
Olaoluwa Osuntokun
5e9166e478
channeldb: use raw pub keys and signatures directly in vertex/edge structs
In this commit, we make an API change that’s meant to reduce the amount
of garbage we generate when doing pathfinding or syncing nodes with our
latest graph state. Before this commit, we would always have to fully
decode the public key and signatures when reading a edge or vertex
struct. For the edges, we may need several EC operations to fully
decode all the pubkeys. This has been seen to generate a ton of
garbage, as well as slow down path finding a good bit.

To remedy this, we’ll now only ever read the *raw* bytes from disk. In
the event that we actually need to verify a signature (or w/e), only
*then* will we fully decode everything.
2018-02-06 20:14:31 -08:00
practicalswift
a93736d21e multi: comprehensive typo fixes across all packages 2018-02-06 19:11:11 -08:00
Johan T. Halseth
b26560e0f4
channeldb: add DisconnectBlockAtHeight
This commit adds the method DisconnectBlockAtHeight to the channel
graph database, making it possible to "rewind" the database in case
a block is disconnected from the main chain. To accomplish this,
a prune log is introduced, making it possible to keep track of the
point in time where the database was pruned. This is necessary for
the case where lnd might wake up on a stale branch, and must "walk
backwards" on this branch after it finds a common block fro the
graph database and the new main chain.
2017-11-03 00:05:19 +01:00
Jim Posen
9fd77a6e40 multi: Update lnd to use new feature vector API. 2017-10-17 22:47:20 -07:00
Olaoluwa Osuntokun
460c2b0ec1
channeldb: update test to match recent API changes 2017-08-22 00:52:28 -07:00
Johan T. Halseth
bd0465ee1d channeldb: support adding partial LightningNodes to graph.
Adds a HaveNodeAnnouncement field to the LightningNode
struct, which is used to indicate if we have gotten
all the necessary information to fill the remaining
fields in the struct. If we haven't gotten a node
announcement for this specific node, then we only
know the pubkey, and can only fill that field in
the struct. Still, we should be able to add it to the
channel graph and use it for routes, as long as we
know about channels to this node.
2017-08-02 15:58:58 -07:00
Olaoluwa Osuntokun
1be4d67ce4
multi: run all test instances in parallel 2017-06-17 01:00:07 +02:00
Olaoluwa Osuntokun
333373f78f
channeldb: add new ChannelView method to the ChannelGraph
This commit introduces a new method to the ChannelGraph struct:
ChannelView. This struct returns all the outpoints that represent the
set of active channels within the network. The set of items returned by
this new method will possibly shrink with each call to `PruneGraph`,
and possibly be expanded by each call to `AddChannelEdge`.

The graph pruning tests have been updated to ensure the description
above holds true.
2017-05-11 15:20:23 -07:00
Olaoluwa Osuntokun
b96b180b0b
channeldb: modify ForEachNode/ForEachChannel to accept a db txn
This commit modifies the ForEachNode on the ChannelGraph and
ForEachChannel on the LightningNode struct to accept a database
transaction as its first argument. With this change, it’ll now be
possible to implement graph traversals that typically required a nested
loop with all the vertex loaded into memory using the callback API
instead:
c.ForEachNode(nil, func(tx, node) {
    node.ForEachChannel(tx, func(…) {
    })
})
2017-04-14 13:14:09 -07:00
Andrey Samokhvalov
19174ebdfd channeldb: add storing of node signature and add edge signature
In order to properly announce the channel the announcements proofs
should be persistent in boltdb.
2017-03-29 19:49:05 -07:00
bryanvu
085b7333cb lnwire: add support for Features in NodeAnnouncement
Add support for Features in NodeAnnouncment according to spec.
2017-03-29 12:03:43 -07:00
bryanvu
654c5ea61a config: added support, tests for --externalip config option
Minor change to server.go to add ExternalIPs to
channeldb.LightningNode. Also, added a test that utilizes this
functionality and exercises multiple addresses in NodeAnnouncement.
2017-03-29 12:03:43 -07:00
bryanvu
9ffac9eae1 lnwire: update NodeAnnouncement to handle multiple addresses
This commit modifies address handling in the NodeAnnouncement struct,
switching from net.TCPAddr to []net.Addr. This enables more flexible
address handling with multiple types and multiple addresses for each
node. This commit addresses the first part of issue #131 .
2017-03-29 12:03:43 -07:00
Olaoluwa Osuntokun
f217093c00
multi: replace usage of fastsha256 with crypto/sha256
This commit removes all instances of the fastsha256 library and
replaces it with the sha256 library in the standard library. This
change should see a number of performance improvements as the standard
library has highly optimized assembly instructions with use vectorized
instructions as the platform supports.
2017-03-15 18:56:41 -07:00
Andrey Samokhvalov
fd97a4bd19 lnd: partially fix golint warnings 2017-03-13 16:30:23 -07:00
Olaoluwa Osuntokun
39e145f037
channeldb: update tests to recent API change (edge struct split) 2017-03-05 19:22:33 -06:00
Olaoluwa Osuntokun
5affed38fc
multi: update btcsuite API's to latest upstream changes
This commit makes a large number of minor changes concerning API usage
within the deamon to match the latest version on the upstream btcsuite
libraries.

The major changes are the switch from wire.ShaHash to chainhash.Hash,
and that wire.NewMsgTx() now takes a paramter indicating the version of
the transaction to be created.
2017-01-05 13:56:34 -08:00
Olaoluwa Osuntokun
1103e252be
channeldb: add method to lookup a channel ID by it's funding outpoint
This commit adds a utility method whcih utilizes the edge index bucket
and allows caller to look up the channel ID of a channel by its funding
outpoint. This can be used to populate RPC’s with additional
information and also to provide users with an additional query
interface to build channel explorers, etc.
2016-12-27 16:44:13 -08:00
Olaoluwa Osuntokun
000c334e63
channeldb: add HasLightningNode+HasChannelEdge methods to ChannelGraph
This commit adds to new functions to the ChannelGraph struct which
allow the callers to query for the existence or non-existence of a
vertex (node) or edge (channel) within the graph. In addition to
returning whether the edge exists, the functions will also return the
last time the state has been modified for the edge or vertex. This will
allow callers to ensure that only the most up to date state is
committed to disk.
2016-12-27 16:43:29 -08:00
Olaoluwa Osuntokun
587bde5636
channeldb: ensure the cleanUp func is always run during tests 2016-12-27 16:43:17 -08:00
Olaoluwa Osuntokun
12538ea922
channeldb: add support for channel graph pruning
This commit adds support for channel graph pruning, which is the method
used to keep the channel graph in sync with the current UTXO state. As
the channel graph is essentially simply a subset of the UTXO set, by
evaluating the channel graph with the set of outfits spent within a
block, then we’re able to prune channels that’ve been closed by
spending their funding outpoint. A new method `PruneGraph` has been
provided which implements the described functionality.

Upon start up any upper routing layers should sync forward in the chain
pruning the channel graph with each newly found block. In order to
facilitate such channel graph reconciliation a new method `PruneTip`
has been added which allows callers to query current pruning state of
the channel graph.
2016-12-27 16:43:12 -08:00
Olaoluwa Osuntokun
d5423f007d
channeldb: add option to re-use existing db transaction for graph traversals
This commit modifies the LightningNode.ForEachChannel method to give
the caller the option of re-using an existing database transaction
instead of always creating a new db transaction with each invocation.
Internally boltdb will run into an error/dead-lock if a nested
transaction is attempted.

Such an action might be attempted if one were to use the traversal
functions in  a path finding algorithm. Therefore in order to avoid
that after, we now allow the re-use of transactions to facilitate
nested calls to ForEachChannel.
2016-12-27 16:43:00 -08:00
Olaoluwa Osuntokun
e39dc9eec1
channeldb: add storage of an on-disk directed channel graph
This commit introduces a new capability to the database: storage of an
on-disk directed channel graph. The on-disk representation of the graph
within boltdb is essentially a modified adjacency list which separates
the storage of the edge’s existence and the storage of the edge
information itself.

The new objects provided within he ChannelGraph carry an API which
facilitates easy graph traversal via their ForEach* methods. As a
result, path finding algorithms will be able to be expressed in a
natural way using the range methods as a for-range language extension
within Go.

Additionally caching will likely be added either at this layer or the
layer above (the RoutingManager) in order keep queries and outgoing
payments speedy. In a future commit a new set of RPC’s to query the
state of a particular edge or node will also be added.
2016-12-07 22:50:19 -08:00