Commit Graph

3891 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
ad522a72c1
htlcswitch: add new ticker in switch to batch log forwarding events
In this commit, we extend the switch as is, to record details
concerning settled payment circuits. To do this, we introduce a new
interface to the package: the ForwardingLog. This is a tiny interface
that simply lets us abstract away the details of the storage backing of
the forwarding log.

Each time we receive a successful HTLC settle, we’ll log the full
details (chans, fees, time) as a pending forwarding log entry. Every 15
seconds, we’ll then batch flush out these entries to disk. When we’re
exiting, we’ll try to flush out all entries to ensure everything gets
recorded to disk.
2018-03-06 13:56:11 -05:00
Olaoluwa Osuntokun
6f11fee1a4
htlcswitch: when forwarding htlcs, set the incomingHtlcAmt 2018-03-06 13:56:10 -05:00
Olaoluwa Osuntokun
d377ffafdd
htlcswitch: populate the incoming+outgoing in the payment circuit 2018-03-06 13:56:10 -05:00
Olaoluwa Osuntokun
157e052958
htlcswitch: add the incoming payment amount to the htlcPacket
We’ll need this value within the link+switch in order to fully populate
the forwarding event that will be generated if this HTLC circuit is
successfully completed.
2018-03-06 13:56:10 -05:00
Olaoluwa Osuntokun
2b8cad2f08
htlcswitch: add incoming+outgoing amounts to the PaymentCircuit
In this commit, we add the incoming+outgoing amounts if the HTLC’s that
the payment circuit consists of. With these new fields, we’ll be able
to populate the forwarding event log once the payment circuit has been
successfully completed.
2018-03-06 13:56:09 -05:00
Olaoluwa Osuntokun
744cfd2ce5
channeldb: add a set of initial tests for the ForwardingLog 2018-03-06 13:56:09 -05:00
Olaoluwa Osuntokun
f2cd668bcf
channeldb: add new ForwardingLog storage namespace
In this commit, we add a new storage namespace to channeldb: the
ForwardingLog. This log will be used by higher level sub-systems to log
each successfully completed HTLC. Each payment circuit will be
summarized as a “ForwardingEvent”. A series of events can then be
queried via a time slice query. In a time slice query, the caller
specifies a time range, a number of events to skip, and the max number
of events to return. Each query will return the index of the final
item. As we have a max number of events we’ll return in a response,
callers may need to use this last offset index to seek further by
skipping that number of entries. Combining these fields, callers are
able to query the time series, skipping an arbitrary amount of events,
and capping the max number of returned events.
2018-03-06 13:56:06 -05:00
Olaoluwa Osuntokun
e7e4cdcf49
discovery: avoid always validating ECDSA sigs by asking router if item is fresh
In this commit, we reduce the amount of unnecessary work that the
gossiper can carry out. When CPU profiling some nodes, I noticed that
we’d spend a lot of time validating the signatures for an announcement,
only to realize that the router already had it.

To remedy this, we’ll use the new methods added to the channel router
in order to avoid unnecessarily validating an announcement that is
actually stale. This should reduce memory usage (since it uses big
int’s under the scenes), and also idle CPU usage.
2018-03-06 13:34:51 -05:00
Olaoluwa Osuntokun
aa0410c90a
routing: add new methods to check the freshness of an edge/node
In this commit, we add a set of new methods to check the freshness of
an edge/node. This will allow callers to skip expensive validation in
the case that the router already knows of an item, or knows of a
fresher version of that time.

A set of tests have been added to ensure basic correctness of these new
methods.
2018-03-06 13:34:48 -05:00
vegardengen
a1fb22eb8d rpc: properly populate fee field in listpayments 2018-03-06 13:21:03 -05:00
Olaoluwa Osuntokun
61a7556931
Merge pull request #719 from Roasbeef/rpc-seed
lnrpc+walletunlocker: extend wallet creation to allow user generated entropy + entropy restore (BIP 39)
2018-03-05 15:56:41 -05:00
Olaoluwa Osuntokun
3356a370c7
cmd/lncli: extend initial wallet creation with aezeed seed support
In this commit, we extend the `lncli create` command to allow users to
specify their own side (if they want). In the case that the user
*doesn’t* specify their own seed, we’ll return the entropy generated by
the wallet in a 24-word mnemonic format for easy backup.

With this change, it’s now possible for users to restore an existing lnd
wallet seed.
2018-03-05 11:07:07 -05:00
Olaoluwa Osuntokun
d8ce90306d
lnwallet/btcwallet: during initial creation catch the case of an existing wallet
In this commit, due to the recent changes within lnd itself, it may be
possible that a wallet already exists when the wallet has been signaled
to be created. As a result, *always* open the wallet ourselves, but
allow an existing wallet to already be in place.
2018-03-05 11:07:07 -05:00
Olaoluwa Osuntokun
6289a2fb84
lnd: move seed generation outside wallet to allow for user supplied entropy
In this commit, we extend the initial wallet creation set up case with
the goal of giving the user the ability to restore a prior wallet from
seed, or obtain the mnemonic for a newly generated wallet.

As the WalletUnlocker has been extended to allow passing a user source
of entropy, if this is detected, then we’ll use BIP39 to covert it into
an HD wallet seed. Otherwise, we’ll generate our own entropy, then
convert that into the wallet seed.

In order to make this change, we’ll now manually create the default
wallet ourselves. In the case that the user didn’t provide their own
seed, we’ll send the seed we generated back to the user. This will allow
frontends to display the newly generated seed to the end user.
2018-03-05 11:07:06 -05:00
Olaoluwa Osuntokun
428693cb6e
walletunlocker: modify service to implement new 2-step wallet creation
In this commit, we extend the UnlockerService to account for the new
changes in the lnrpc definition. Setting up the daemon for the first
time is now two step process: first the user will generate a new seed
via the GenSeed method, then the user will present this new seed (and
optional pass) to the InitWallet method which then will finalize the
wallet creation.

This two step process ensures that we don't commit the wallet changes
in the case that the user doesn't actually "ACK" the new seed.

In the case that the user already has an existing seed, they can
re-enter it and skip straight to the InitWallet step.

We also update the tests to account for the new API changes.
2018-03-05 11:07:06 -05:00
Olaoluwa Osuntokun
7f953bf0b9
lnrpc: revamp the WalletUnlocker service to have a two-stage init
In this commit, we revamp the WalletUnlocker service to now have a
two-stage init process.

The first (optional) is for the user instantiating a new lnd instance to
call the GenSeed method with an optional aezeed passphrase. The response
to this will be a freshly generated aezeed mnemonic along with the
original enciphered seed.

The second step will be the actual wallet initaliztion. By separating
this step from seed generation, UI's will be able to ensure that the
user has written down the seed, before proceeding and committing the
seed to the internal wallet. The new method InitWallet accepts a wallet
passphrase, the aezeed mnemonic, and the optional passphrase.
2018-03-05 11:07:01 -05:00
Olaoluwa Osuntokun
294447fe2f
Merge pull request #803 from cfromknecht/glide-crypto-x-version
glide: add explicit version to x/crypto deps
2018-03-05 10:29:33 -05:00
Conner Fromknecht
9ffd7bb7ce
glide: add explicit version to x/crypto deps
This commit resolves an issue where glide would try to install an older
version of x/crypto, which for me resulted in blake2b not exposing the
interfaces required by the new aezeed package. Using the explicit
version has seemed to resolve this entirely.
2018-03-04 21:34:37 -05:00
Alex Gessner
9d34fa8c1e docs/grpc/ruby: fix incorrect documentation in ruby.md value to amt (#799) 2018-03-02 19:16:22 -08:00
Olaoluwa Osuntokun
1ba399267b
Merge pull request #773 from Roasbeef/aezeed
aezeed: add new package implementing the aezeed cipher seed scheme
2018-03-01 17:53:56 -08:00
Olaoluwa Osuntokun
d65fe83a59
build: add aez and bstream to glide 2018-03-01 17:10:54 -08:00
Olaoluwa Osuntokun
120cebef77
aezeed: add a set of benchmarks
In this commit we add a set of benchmarks to be able to measure the
enciphering and deciphering speed of the current scheme with the
current scrypt parameters.

On my laptop I get about 100ms per attempt:
⛰ go test -run=XXX -bench=.

goos: darwin
goarch: amd64
pkg: github.com/lightningnetwork/lnd/aezeed
BenchmarkToMnenonic-4     	      10	 102287840 ns/op
BenchmarkFromMnenonic-4   	      10	 105874973 ns/op
PASS
ok  	github.com/lightningnetwork/lnd/aezeed	3.036s
2018-03-01 17:10:54 -08:00
Olaoluwa Osuntokun
eb3b5196e1
aezeed: add comprehensive, mostly property-based tests 2018-03-01 17:10:53 -08:00
Olaoluwa Osuntokun
ffac0336e6
aezeed: add new package implementing the aezeed cipher seed scheme
In this commit, we add a new package implementing the aezeed cipher
seed scheme. This is a new scheme developed that aims to overcome the
two major short comings of BIP39: a lack of a version, and a lack of a
wallet birthday. A lack a version means that wallets may not
necessarily know *how* to re-derive addresses during the recovery
process. A lack of a birthday means that wallets don’t know how far
back to look in the chain to ensure that they derive *all* the proper
user addresses.

The aezeed scheme addresses these two drawbacks and adds a number of
desirable features. First, we start with the following plaintext seed:
{1 byte internal version || 2 byte timestamp || 16 bytes of entropy}.

The version field is for wallets to be able to know *how* to re-derive
the keys of the wallet.

The 2 byte timestamp is expressed in Bitcoin Days Genesis, meaning that
the number of days since the timestamp in Bitcoin’s genesis block. This
allow us to save space, and also avoid using a wasteful level of
granularity. With the currently, this can express time up until 2188.

Finally, the entropy is raw entropy that should be used to derive
wallet’s HD root.

Next, we’ll take the plaintext seed described above and encipher it to
procure a final cipher text. We’ll then take this cipher text (the
CipherSeed) and encode that using a 24-word mnemonic. The enciphering
process takes a user defined passphrase. If no passphrase is provided,
then the string “aezeed” will be used.

To encipher a plaintext seed (19 bytes) to arrive at an enciphered
cipher seed (33 bytes), we apply the following operations:
   * First we take the external version an append it to our buffer. The
external version describes *how* we encipher. For the first version
(version 0), we’ll use scrypt(n=32768, r=8, p=1) and aezeed.
  * Next, we’ll use scrypt (with the version 9 params) to generate a
strong key for encryption. We’ll generate a 32-byte key using 5 bytes
as a salt. The usage of the salt is meant to make the creation of
rainbow tables infeasible.
  * Next, the enciphering process. We use aezeed, modern AEAD with
nonce-misuse resistance properties. The important trait we exploit is
that it’s an *arbitrary input length block cipher*. Additionally, it
has what’s essentially a configurable MAC size. In our scheme we’ll use
a value of 4, which acts as a 32-bit checksum. We’ll encrypt with our
generated seed, and use an AD of (version || salt). We'll them compute a
checksum over all the data, using crc-32, appending the result to the
end.
  * Finally, we’ll encode this 33-byte cipher text using the default
world list of BIP 39 to produce 24 english words.

The `aezeed` cipher seed scheme has a few cool properties, notably:
   * The mnemonic itself is a cipher text, meaning leaving it in
plaintext is advisable if the user also set a passphrase. This is in
contrast to BIP 39 where the mnemonic alone (without a passphrase) may
be sufficient to steal funds.
   * A cipherseed can be modified to *change* the passphrase. This
means that if the users wants a stronger passphrase, they can decipher
(with the old passphrase), then encipher (with a new passphrase).
Compared to BIP 39, where if the users used a passphrase, since the
mapping is one way, they can’t change the passphrase of their existing
HD key chain.
  * A cipher seed can be *upgraded*. Since we have an external version,
offline tools can be provided to decipher using the old params, and
encipher using the new params. In the future if we change ciphers,
change scrypt, or just the parameters of scrypt, then users can easily
upgrade their seed with an offline tool.
  * We're able to verify that a user has input the incorrect passphrase,
and that the user has input the incorrect mnemonic independently.
2018-03-01 17:10:50 -08:00
Olaoluwa Osuntokun
a9dc4f80f2
chainntnfs/neutrinonotify: log height hint for spend notifications 2018-03-01 16:49:28 -08:00
Olaoluwa Osuntokun
9479b085e2
Merge pull request #785 from brndnmtthws/master
Add more details on using macaroons with GRPC.
2018-02-28 19:27:13 -08:00
t4sk
55f751fd37 routing/router Fix FindRoutes in router test
In router_test FindRoutes is passing DefaultFinalCLTVDelta in place
where numPaths is expected. This commit passes a default numPaths for
function calls to FindRoutes so that final cltv delta are correctly
passed.
2018-02-28 19:25:19 -08:00
Brenden Matthews
37aa9759db
docs: Add details to macaroon usage with GRPC. 2018-02-28 16:19:48 -05:00
Brenden Matthews
eed0e9ac67
docs: Show macaroon usage with Java GRPC example. 2018-02-28 16:19:48 -05:00
Olaoluwa Osuntokun
f7ec490f44
Merge pull request #762 from halseth/fee-estimator
Add types for representing fee rates
2018-02-26 17:22:50 -08:00
John Griffith
0468f7b308 fundingmanager: Added additional test to check a pending channel
initiated by the user doesn't timeout.

Split assertNumPendingChannels into assertNumPendingChannelsRemains,
and assertNumPendingChannelsRemains to prevent possible false
positives passing tests
2018-02-26 16:44:34 -08:00
John Griffith
63ee31b83f fundingmanager: Added additional check so that pending channels don't
timeout for the channel initiator
2018-02-26 16:44:34 -08:00
Olaoluwa Osuntokun
7f04d927a0
rpc: fix queryroutes bug that assumed precise number of returned routes
In this commit, we fix a bug in the query routes RPC that could at
times lead to a panic. This would happen if the number of returned
routes was less than the number of expected routes. To remedy this,
we’ll return the minimum of the number of requested routes, and the
number of routes actually returned.
2018-02-26 16:31:50 -08:00
Johan T. Halseth
2db5e56754
lnwire: add todo for changing to SatPerKWeight for fee rates 2018-02-26 22:42:28 +01:00
Johan T. Halseth
b9f09a666d
lnd test: calculate channel reserve based on balance before fees 2018-02-26 22:42:27 +01:00
Johan T. Halseth
24d30e6495
peer test: remove var casting 2018-02-26 22:42:27 +01:00
Johan T. Halseth
f42c16b612
lnd test: ensure static fee is not rounded down 2018-02-26 22:42:27 +01:00
Johan T. Halseth
004563b20b
channeldb: add todo making feerate explicit type 2018-02-26 22:42:27 +01:00
Johan T. Halseth
dd6872da31
lnwallet test: add TestFeeRateTypes and TestStaticFeeEstimator 2018-02-26 22:42:27 +01:00
Johan T. Halseth
7b30425111
multi test: update tests/mocks to use EstimateFeePerVSize 2018-02-26 22:42:27 +01:00
Johan T. Halseth
ba3f3e1942
rpcserver: express fee rates using types 2018-02-26 22:42:27 +01:00
Johan T. Halseth
b9d1eceda3
peer: use EstimateFeePerVSize 2018-02-26 22:42:26 +01:00
Johan T. Halseth
d4cedc2917
breacharbiter: use EstimateFeePerVSize 2018-02-26 22:42:26 +01:00
Johan T. Halseth
e48d51db92
pilot: express fee rates using types, use EstimateFeePerVSize 2018-02-26 22:42:26 +01:00
Johan T. Halseth
3dc9e3c7d4
server: express fee rates using fee rate types 2018-02-26 22:42:26 +01:00
Johan T. Halseth
5f267bcc5b
utxonursery: use EstimateFeePerVSize 2018-02-26 22:42:26 +01:00
Johan T. Halseth
2a9aa092d8
fundingmanager: express fee rates using types, use EstimateFeePerVSize 2018-02-26 22:42:26 +01:00
Johan T. Halseth
3e12f823b2
chancloser: express fee rates in sat/vbyte 2018-02-26 22:42:26 +01:00
Johan T. Halseth
6d5c0679f4
chainregistry: express fee rates in sat/vbyte 2018-02-26 22:42:25 +01:00
Johan T. Halseth
166637bb55
contractcourt: use vsize fee estimate and new fee rate types 2018-02-26 22:42:25 +01:00