Commit Graph

6138 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
7fcab83bb4
chanbackup: add functions to allow recovery of existing channel backups
In this commit, we add a series of functions that will allow users to
recover existing channel backups. We do this using two primary
interfaces: the ChannelRestorer, and the PeerConnector. The first
interfaces allows us to abstract away the details w.r.t exactly how a
channel is restored. Instead, we simply expect that the channel backup
will be inserted as a sort of "channel shell" which contains only the
data required to initiate the data loss protection protocol. The second
interface is how we instruct the Lightning node to connect out to the
channel peer given its known addresses.
2019-01-23 18:11:29 -08:00
Olaoluwa Osuntokun
1348c6b35b
chanbackup: add functionality to allow external callers to backup channels
In this commit, we introduce a series of interfaces and methods that
will allow external callers to backup either all channels, or a specific
channel identified by its channel point. In order to abstract away the
details w.r.t _how_ we obtain the set of open channels, or their storage
mechanisms, we introduce a new LiveChannelSource interfaces. This
interfaces allows us to fetch all channels, a channel by its channel
point, and also all the known addresses for a node as we'll need this in
order to connect out to the node in the case of a recovery attempt.
2019-01-23 18:11:28 -08:00
Olaoluwa Osuntokun
71df4b0545
chanbackup: introduce Multi, a multi-channel backup
In this commit, we introduce the Multi sturct. Multi is a series of
static channel backups. This type of backup can contains ALL the channel
backup state in a single packed blob. This is suitable for storing on
your file system, cloud storage, etc. Systems will be in place within
lnd to ensure that one can easily obtain the latest version of the Multi
for the node, and also that it will be kept up to date if channel state
changes.
2019-01-23 18:11:27 -08:00
Olaoluwa Osuntokun
ee71e16593
chanbackup: add implementation of Single Channel Backups (SCBs)
In this commit, we add the initial implementation of the SCB structure.
Given an SCB, and a user's seed, it will be possible to recover the
settled balanced of a channel in the event of total or partial data
loss. The SCB contains all information required to initiate the data
loss protection protocol once we restore the channel and connect to the
remote channel peer.

The primary way outside callers will interact with this package are via
the Pack and Unpack methods. Packing means writing a
serialized+encrypted version of the SCB to an io.Writer. Unpacking does
the opposite.

The encoding format itself uses the same encoding as we do on the wire
within Lightning. Each encoded backup begins with a version so we can
easily add or modify the serialization format in the future, if new
channel types appear, or we need to add/remove fields.
2019-01-23 18:11:26 -08:00
Olaoluwa Osuntokun
d98a7e8138
chanbackup: add logger 2019-01-23 18:11:25 -08:00
Olaoluwa Osuntokun
67c6d0d331
chanbackup: implement crypto operations for serialized static channel baackups
In this commit, we implement a series of new crypto operations that will
allow us to encrypt and decrypt a set of serialized channel backups.
Their various backups may have distinct encodings when serialized, but
to the functions defined in this file, we treat them as simple opaque
blobs.

For encryption, we utilize chacha20poly1305 with a random 24 byte nonce.
We use a larger nonce size as this can be safely generated via a CSPRNG
without fear of frequency collisions between nonces generated. To
encrypt a blob, we then use this nonce as the AD (associated data) and
prepend the nonce to the front of the ciphertext package.

For key generation, in order to ensure the user only needs their
passphrase and the backup file, we utilize the existing keychain to
derive a private key. In order to ensure that at we don't force any
hardware signer to be aware of our crypto operations, we instead opt to
utilize a public key that will be hashed to derive our private key. The
assumption here is that this key will only be exposed to this software,
and never derived as a public facing address.
2019-01-23 18:11:21 -08:00
Olaoluwa Osuntokun
eff9899d14 keychain: add new KeyFamily: KeyFamilyStaticBackup for SCB encryption keys 2019-01-23 18:11:11 -08:00
Olaoluwa Osuntokun
e2285732e4
Merge pull request #2460 from halseth/max-htlc-size-pickup
Support the max_htlc field from ChannelUpdates
2019-01-22 18:54:38 -08:00
Wilmer Paulino
9860df69fe
Merge pull request #2247 from wpaulino/grpc-proxy-endpoint
rpcserver: forward gRPC proxy requests to localhost when listening on all interfaces
2019-01-22 16:21:21 -08:00
Johan T. Halseth
7d34ce9d08
lnwire+multi: define HasMaxHtlc helper on msgFlags 2019-01-22 08:42:30 +01:00
Valentine Wallace
4fb1536f54
fundingmanager_test: verify max HTLC in ChannelUpdates
In this commit, we verify that ChannelUpdates for newly
funded channels contain the max HTLC that we expect.
We expect the max HTLC value of each ChannelUpdate to
equal the maximum pending msats in HTLCs required by
the remote peer.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:30 +01:00
Valentine Wallace
a66a1e113f
fundingmanager: when funding new channels, set a default max htlc
In this commit, we set a default max HTLC value in ChannelUpdates
sent out for newly funded channels. As a result, we also default
to setting `MessageFlags` equal to 1 in each new ChannelUpdate, since
the max HTLC field is an optional field and MessageFlags indicates
the presence of optional fields within the ChannelUpdate.

For a default max HTLC, we choose the maximum msats worth of
HTLCs that can be pending (or in-flight) on our side of the channel.
The reason for this is because the spec specifies that the max
HTLC present in a ChannelUpdate must be less than or equal to
both total channel capacity and the maximum in-flight amount set
by the peer. Since this in-flight value will always be less than
or equal to channel capacity, it is a safe spec-compliant default.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:30 +01:00
Johan T. Halseth
4f9de9bf1d
fundingmanager_test: add TestFundingManagerRejectInvalidMaxValueInFlight 2019-01-22 08:42:30 +01:00
Johan T. Halseth
4aa52d267f
funding+lnwallet: ensure max_htlc_value_in_flight smaller than capacity
Return an error to the remote if larger.
2019-01-22 08:42:29 +01:00
Valentine Wallace
f8e588e2e3
autopilot/graph: set max htlc in test edge policies
Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:29 +01:00
Valentine Wallace
cced754659
routing/router: when applying onion failure channel update, set max htlc
In this commit, we ensure that when we update an edge
as a result of a ChannelUpdate being returned from an
onion failure, the max htlc portion of the channel update
is included in the edge update.
2019-01-22 08:42:29 +01:00
Valentine Wallace
390b974793
server: set max htlc in ChannelUpdate createChannelUpdate
This method is called to convert an EdgePolicy to a ChannelUpdate. We
make sure to carry over the max_htlc value.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:29 +01:00
Valentine Wallace
207c4f030a
discovery/gossiper: include max HTLC when rebroadcasting stale channel updates
Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:29 +01:00
Valentine Wallace
19c403711e
discovery/chan_series+utils: include max htlc when syncing with peers
In this commit, we ensure that max HTLC is included when we're
synchronizing ChannelUpdates with remote peers.
2019-01-22 08:42:28 +01:00
Valentine Wallace
513ac23479
discovery/gossiper: persist remote channel policy updates' max htlc 2019-01-22 08:42:28 +01:00
Johan T. Halseth
01e679786d
channeldb/graph_test: add TestEdgePolicyMissingMaxHtcl 2019-01-22 08:42:28 +01:00
Johan T. Halseth
6fcc76fc68
channeldb/channel: ignore ChannelEdgePolicy with missing max_htlc
If the max_htlc field is not found when fetching a ChannelEdgePolicy
from the DB, we treat this as an unknown policy.

This is done to ensure we won't propagate invalid data further. The data
will be overwritten with a valid one when we receive an update for this
channel.

It shouldn't be very common, but old data could be lingering in the DB
added before this field was validated.
2019-01-22 08:42:28 +01:00
Johan T. Halseth
8dd074ee57
channeldb/graph_test: assert MaxHTLC field correctness
Co-authored-by: Valentine Wallace <valentine.m.wallace@gmail.com>
2019-01-22 08:42:27 +01:00
Valentine Wallace
69d4bf051f
channeldb/graph: add max HTLC to ChannelEdgePolicy
Adding this field will allow us to persist an edge's
max HTLC to disk, thus preserving it between restarts.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:27 +01:00
Johan T. Halseth
b9c5248915
channeldb/graph: extract ChannelEdgePolicy serialization 2019-01-22 08:42:27 +01:00
Johan T. Halseth
91c9e45031
channeldb/graph: check correct bucket for nilness 2019-01-22 08:42:27 +01:00
Valentine Wallace
15168c391e
discovery+routing: validate msg flags and max htlc in ChannelUpdates
In this commit, we alter the ValidateChannelUpdateAnn function in
ann_validation to validate a remote ChannelUpdate's message flags
and max HTLC field. If the message flag is set but the max HTLC
field is not set or vice versa, the ChannelUpdate fails validation.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:27 +01:00
Valentine Wallace
f316cc6c7e
discovery/gossiper_test: set ChannelUpdate max htlc
In this commit, we alter the gossiper test's helper method
that creates channel updates to include the max htlc field
in the ChannelUpdates it creates.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:26 +01:00
Valentine Wallace
7ab8900eb6
discovery/gossiper_test: mock AddEdge: set capacity
In this commit, we modify the mockGraphSource's `AddEdge`
method to set the capacity of the edge it's adding to be a large
capacity.

This will enable us to test the validation of each ChannelUpdate's
max HTLC, since future validation checks will ensure the specified
max HTLC is less than total channel capacity.
2019-01-22 08:42:26 +01:00
Valentine Wallace
b49637fbe9
lnwire: add HtlcMaximumMsat field to ChannelUpdate
In this commit, we add a field to the ChannelUpdate
denoting the maximum HTLC we support sending over
this channel, a field which was recently added to the
spec.

This field serves multiple purposes. In the short
term, it enables nodes to signal the largest HTLC
they're willing to carry, allows light clients who
don't verify channel existence to have some guidance
when routing HTLCs, and finally may allow nodes to
preserve a portion of bandwidth at all times.

In the long term, this field can be used by
implementations of AMP to guide payment splitting,
as it becomes apparent to a node the largest possible
HTLC one can route over a particular channel.

This PR was made possible by the merge of #1825,
which enables older nodes to properly retain and
verify signatures on updates that include new fields
(like this new max HTLC field) that they haven't yet
been updated to recognize.

In addition, the new ChannelUpdate fields are added to
the lnwire fuzzing tests.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:26 +01:00
Valentine Wallace
f0ba4b454c
lnwire/channel_update: add String method for ChanUpdate[Chan|Msg]Flags
In this commit, we fix the problem where it's annoying to parse a
bitfield printed out in decimal by writing a String method for the
ChanUpdate[Chan|Msg]Flags bitfield.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:26 +01:00
Valentine Wallace
0fd6004958
multi: partition lnwire.ChanUpdateFlag into ChannelFlags and MessageFlags
In this commit:

* we partition lnwire.ChanUpdateFlag into two (ChanUpdateChanFlags and
ChanUpdateMsgFlags), from a uint16 to a pair of uint8's

* we rename the ChannelUpdate.Flags to ChannelFlags and add an
additional MessageFlags field, which will be used to indicate the
presence of the optional field HtlcMaximumMsat within the ChannelUpdate.

* we partition ChannelEdgePolicy.Flags into message and channel flags.
This change corresponds to the partitioning of the ChannelUpdate's Flags
field into MessageFlags and ChannelFlags.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:26 +01:00
Wilmer Paulino
552b71c620
rpcserver: forward gRPC proxy requests to localhost when listening on all interfaces
This prevents certificate issues when accessing the gRPC REST proxy externally.
2019-01-21 16:07:19 -08:00
Olaoluwa Osuntokun
375be936ce
Merge pull request #2314 from wpaulino/chainnotifier-subserver
chainntnfs+lnrpc/chainrpc: add ChainNotifier RPC subserver
2019-01-21 15:19:12 -08:00
Wilmer Paulino
59e2be5306
config: add ChainRPC config 2019-01-21 14:02:00 -08:00
Wilmer Paulino
3d91aa2312
lnrpc/chainrpc: add ChainNotifier subserver registration driver 2019-01-21 14:01:22 -08:00
Wilmer Paulino
1f8eef2833
lnrpc/chainrpc: add ChainNotifier subserver implementation 2019-01-21 14:01:22 -08:00
Wilmer Paulino
5e98af8c99
lnrpc/chainrpc: add ChainNotifier subserver Config 2019-01-21 14:01:22 -08:00
Wilmer Paulino
d06f6c14f5
lnrpc/chainrpc: add ChainNotifier subserver protos 2019-01-21 14:01:20 -08:00
Wilmer Paulino
26d53c64f1
chainntnfs: dispatch block notification for current tip upon registration 2019-01-21 13:58:52 -08:00
Wilmer Paulino
568f8271a0
chainntnfs/neutrinonotify: track best block hash 2019-01-21 13:58:52 -08:00
Wilmer Paulino
060f2f7774
chainntnfs: add Done chan to events to signal reorg safety 2019-01-21 13:58:52 -08:00
Wilmer Paulino
52a80f2d37
chainntnfs/txnotifier: track reorg risk for transactions within ConnectTip 2019-01-21 13:58:52 -08:00
Wilmer Paulino
0c579b110c
chainntnfs: add support to cancel pending registered confirmation 2019-01-21 13:58:52 -08:00
Wilmer Paulino
be2c321c8c
chainntnfs: declare common ErrChainNotifierShuttingDown for all backends 2019-01-21 13:58:52 -08:00
Wilmer Paulino
50650d054e
chainntnfs: include transaction in confirmation details 2019-01-21 13:58:52 -08:00
Wilmer Paulino
5ab30cf7ea
chainntnfs/interface_test: add script dispatch test cases 2019-01-21 13:57:43 -08:00
Wilmer Paulino
83c7f204cd
chainntnfs: modify GetTestTxidAndScript to generate new P2PKH scripts
In this commit, we modify GetTestTxidAndScript to generate new P2PKH
scripts. This is needed to properly test confirmations and spends of
unique scripts on-chain within the set of interface-level test cases.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
52db5ed682
chainntnfs/neutrinonotify: support registration for script spends
In this commit, we extend the NeutrinoNotifier to support registering
scripts for spends notifications. Once the script has been detected as
spent within the chain, a spend notification will be dispatched through
the Spend channel of the SpendEvent returned upon registration.

For scripts that have been spent in the past, the rescan logic has been
modified to match on the script rather than the outpoint. A concurrent
queue for relevant transactions has been added to proxy notifications
from the underlying rescan to the txNotifier. This is needed for
scripts, as we cannot perform a historical rescan for scripts through
`GetUtxo`, like we do with outpoints.

For scripts that are unspent, a filter update is sent to the underlying
rescan to ensure that we match and dispatch on the script when
processing new blocks.

Along the way, we also address an issue where we'd miss detecting that
an outpoint/script has been spent in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always request the backend to notify us of the spend
once it detects it at tip, regardless of whether a historical rescan was
detected or not.
2019-01-21 13:57:43 -08:00
Wilmer Paulino
482f05a3bc
chainntnfs/bitcoindnotify: support registration for script spends
In this commit, we extend the BitcoindNotifier to support registering
scripts for spends notifications. Once the script has been detected as
spent within the chain, a spend notification will be dispatched through
the Spend channel of the SpendEvent returned upon registration.

For scripts that have been spent in the past, the rescan logic has been
modified to match on the script rather than the outpoint. This is done
by re-deriving the script of the output a transaction input is spending
and checking whether it matches ours.

For scripts that are unspent, a request to the backend will be sent to
alert the BitcoindNotifier of when the script was spent by a
transaction. To make this request we encode the script as an address, as
this is what the backend uses to detect the spend. The transaction will
then be proxied through the txUpdates concurrent queue, which will hand
it off to the underlying txNotifier and dispatch spend notifications to
the relevant clients.

Along the way, we also address an issue where we'd miss detecting that
an outpoint/script has been spent in the future due to not receiving a
historical dispatch request from the underlying txNotifier. To fix this,
we ensure that we always request the backend to notify us of the spend
once it detects it at tip, regardless of whether a historical rescan was
detected or not.
2019-01-21 13:57:43 -08:00