Commit Graph

7227 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
3b370fa08d
chanbackup: add new sub-system for keeping the on disk channels.backup up to date
In this commit, we introduce the chanbackup.SubSwapper interface. It
takes a regular Swapper implementation (defined by the
chanbackup.SubSwapper) interface along with a chanbackup.ChannelNotifier
implementation. Given these two interfaces, we're able to be notified
when a new channel is opened or closed, and then use the Swapper to
atomically replace the on-disk channel back up. As a result, a Lightning
daemon can ensure that they alwayts have a up to date channels.backup on
disk that can safely be copied away by users and be used to restore
channel funds in the event of partial/total data loss.
2019-01-23 18:11:33 -08:00
Olaoluwa Osuntokun
60999df08f
chanbackup: add new MultiFile struct for storing+updating Multis on disk
In this commit, we add a new MultiFile struct. We'll use this struct in
store the latest multi-channel backup on disk, swap it out atomically,
and finally extract+unpack the contents of the multi-file. The format
that's written to disk is the same as a regular Packed multi. The
contents of this new file are meant to be used to safely implement an
always up to date multi file on disk as a way for users to easily rsync
or fsnotiy (when it changes) the backup state of their channels.

We implement an atomic update and swap in the UpdateAndSwap. The method
uses relies on the underlying file system supporting an atomic rename
syscall. We first make a temporary backup file, write the latest
contents to that, then swap the temp file with the main file using
rename(2). This way, we ensure that we always have a single up to date
file, if the protocol aborts before the rename, then we can detect this,
remove the temp file, and attempt another swap.
2019-01-23 18:11:31 -08:00
Olaoluwa Osuntokun
7fcab83bb4
chanbackup: add functions to allow recovery of existing channel backups
In this commit, we add a series of functions that will allow users to
recover existing channel backups. We do this using two primary
interfaces: the ChannelRestorer, and the PeerConnector. The first
interfaces allows us to abstract away the details w.r.t exactly how a
channel is restored. Instead, we simply expect that the channel backup
will be inserted as a sort of "channel shell" which contains only the
data required to initiate the data loss protection protocol. The second
interface is how we instruct the Lightning node to connect out to the
channel peer given its known addresses.
2019-01-23 18:11:29 -08:00
Olaoluwa Osuntokun
1348c6b35b
chanbackup: add functionality to allow external callers to backup channels
In this commit, we introduce a series of interfaces and methods that
will allow external callers to backup either all channels, or a specific
channel identified by its channel point. In order to abstract away the
details w.r.t _how_ we obtain the set of open channels, or their storage
mechanisms, we introduce a new LiveChannelSource interfaces. This
interfaces allows us to fetch all channels, a channel by its channel
point, and also all the known addresses for a node as we'll need this in
order to connect out to the node in the case of a recovery attempt.
2019-01-23 18:11:28 -08:00
Olaoluwa Osuntokun
71df4b0545
chanbackup: introduce Multi, a multi-channel backup
In this commit, we introduce the Multi sturct. Multi is a series of
static channel backups. This type of backup can contains ALL the channel
backup state in a single packed blob. This is suitable for storing on
your file system, cloud storage, etc. Systems will be in place within
lnd to ensure that one can easily obtain the latest version of the Multi
for the node, and also that it will be kept up to date if channel state
changes.
2019-01-23 18:11:27 -08:00
Olaoluwa Osuntokun
ee71e16593
chanbackup: add implementation of Single Channel Backups (SCBs)
In this commit, we add the initial implementation of the SCB structure.
Given an SCB, and a user's seed, it will be possible to recover the
settled balanced of a channel in the event of total or partial data
loss. The SCB contains all information required to initiate the data
loss protection protocol once we restore the channel and connect to the
remote channel peer.

The primary way outside callers will interact with this package are via
the Pack and Unpack methods. Packing means writing a
serialized+encrypted version of the SCB to an io.Writer. Unpacking does
the opposite.

The encoding format itself uses the same encoding as we do on the wire
within Lightning. Each encoded backup begins with a version so we can
easily add or modify the serialization format in the future, if new
channel types appear, or we need to add/remove fields.
2019-01-23 18:11:26 -08:00
Olaoluwa Osuntokun
d98a7e8138
chanbackup: add logger 2019-01-23 18:11:25 -08:00
Olaoluwa Osuntokun
67c6d0d331
chanbackup: implement crypto operations for serialized static channel baackups
In this commit, we implement a series of new crypto operations that will
allow us to encrypt and decrypt a set of serialized channel backups.
Their various backups may have distinct encodings when serialized, but
to the functions defined in this file, we treat them as simple opaque
blobs.

For encryption, we utilize chacha20poly1305 with a random 24 byte nonce.
We use a larger nonce size as this can be safely generated via a CSPRNG
without fear of frequency collisions between nonces generated. To
encrypt a blob, we then use this nonce as the AD (associated data) and
prepend the nonce to the front of the ciphertext package.

For key generation, in order to ensure the user only needs their
passphrase and the backup file, we utilize the existing keychain to
derive a private key. In order to ensure that at we don't force any
hardware signer to be aware of our crypto operations, we instead opt to
utilize a public key that will be hashed to derive our private key. The
assumption here is that this key will only be exposed to this software,
and never derived as a public facing address.
2019-01-23 18:11:21 -08:00
Olaoluwa Osuntokun
eff9899d14 keychain: add new KeyFamily: KeyFamilyStaticBackup for SCB encryption keys 2019-01-23 18:11:11 -08:00
Olaoluwa Osuntokun
ecd5541d55
Merge pull request #2350 from halseth/autopilot-dryrun-only
[Autopilot] Add heuristic dry-run option
2019-01-23 17:59:00 -08:00
Conner Fromknecht
e0cf99c53d
Makefile: use go-acc for coverage 2019-01-23 17:42:52 -08:00
Olaoluwa Osuntokun
6c610d977f
Merge pull request #2075 from wpaulino/settle-invoice-on-chain-sweep
contractcourt: settle invoice when claiming HTLC on-chain
2019-01-23 16:50:54 -08:00
Olaoluwa Osuntokun
7a6b4e2fc5
Merge pull request #2532 from wpaulino/btcwallet-initial-sync
build: update btcwallet dependency to new initial sync definition
2019-01-23 16:41:03 -08:00
Johan T. Halseth
5306ed5b68
lncli: add autopilot query command
To query the autopilot heuristics for scores.
2019-01-23 09:05:43 +01:00
Johan T. Halseth
8874be764e
lnrpc/autopilotrpc: implement QueryScores RPC 2019-01-23 09:05:43 +01:00
Johan T. Halseth
a654be5884
lnrpc/autopilotrpc: add QueryScores API
Adds a new lnrpc API that lets the caller get ("dry run") the scores
that would be given to a set of node by the available autopilot
heuristics.
2019-01-23 09:05:43 +01:00
Johan T. Halseth
8f54a2bd6f
autopilot/manager: add QueryHeuristics
Proxies a query request to the active agent.
2019-01-23 09:05:43 +01:00
Johan T. Halseth
6c556fd92a
autopilot/agent: add queryHeuristics
This commit adds a method queryHeuristic that will query all simple
heuristics for node scores, in addition to the current active heuristic.
2019-01-23 09:05:43 +01:00
Johan T. Halseth
cf01879d96
autopilot/agent: protect chanState with mutex
Since we want to access the current channel state outside the main
autopilot agent goroutine, we make sure it is protected by a mutex.
2019-01-23 09:05:42 +01:00
Johan T. Halseth
8ab718368d
lnd+pilot+autopilot: use config to set active autopilot heuristics 2019-01-23 09:05:42 +01:00
Johan T. Halseth
36e7694e8e
config+pilot: define autopilot.heuristic
And validate the config set by the user.
2019-01-23 09:05:42 +01:00
Johan T. Halseth
c0fed861d2
autopilot/interface: add Name() to AttachmentHeuristic interface 2019-01-23 09:05:42 +01:00
Wilmer Paulino
016add6adc
lnd_test: extend chain claim test with invoice settled check
In this commit, we extend the remote/receiver chain claim integration
test to assert that the on-disk representation of the invoice on the
receiving side (Carol) is marked as settled due to the claiming the HTLC
on-chain.
2019-01-22 20:46:14 -08:00
Wilmer Paulino
41f638c7cf
contractcourt+server: settle invoice for on-chain HTLC sweep
In this commit, we extend the htlcSuccessResolver to settle the invoice,
if any, of the corresponding on-chain HTLC sweep. This ensures that the
invoice state is consistent as when claiming the HTLC "off-chain".
2019-01-22 20:46:14 -08:00
Wilmer Paulino
974e0f2df5
cnct: make original htlc amt available for contract resolvers
Previously, contract resolvers that needed to publish a second level tx,
did not have access to the original htlc amount.

This commit reconstructs this amount from data that is already persisted
in arbitrator log.

Co-authored-by: Joost Jager <joost.jager@gmail.com>
2019-01-22 20:47:17 -08:00
Wilmer Paulino
93754f8d37
lnwallet: add HtlcPoint() resolution helper methods
Co-authored-by: Joost Jager <joost.jager@gmail.com>
2019-01-22 20:47:13 -08:00
Wilmer Paulino
865f7568d6
cnct: extract relaunching of resolvers to method
Co-authored-by: Joost Jager <joost.jager@gmail.com>
2019-01-22 20:46:14 -08:00
Wilmer Paulino
ba2d9ff81a
build: update btcwallet with initial sync improvements
In this commit, we update our btcwallet dependency to point to the
latest version. This latest version redefines what the wallet will
consider as an initial sync. We'll now define it by determining if the
wallet has synced up to its birthday block, rather than looking at the
number of UTXOs in the wallet. This was needed, especially for light
clients, because it would cause unnecessary rescans to happen from the
wallet's birthday if the wallet had no UTXOs.
2019-01-22 19:53:33 -08:00
Olaoluwa Osuntokun
cebc4d8dba
Merge pull request #2405 from halseth/autopilot-weighted-heuristics-follow-up
[autopilot] Weighted combined attachment heuristic
2019-01-22 19:08:46 -08:00
Olaoluwa Osuntokun
e2285732e4
Merge pull request #2460 from halseth/max-htlc-size-pickup
Support the max_htlc field from ChannelUpdates
2019-01-22 18:54:38 -08:00
Wilmer Paulino
e684efa5a7
lnrpc: update ListInvoices documentation
Currently, the documentation includes details for `lncli`. This commit
modifies it to reflect the behavior of the RPC interface instead, as it
should.
2019-01-22 18:23:39 -08:00
Wilmer Paulino
9860df69fe
Merge pull request #2247 from wpaulino/grpc-proxy-endpoint
rpcserver: forward gRPC proxy requests to localhost when listening on all interfaces
2019-01-22 16:21:21 -08:00
Johan T. Halseth
33b2d92b80
lnd_test: don't shadow predErr
Would lead to the printed predErr not being set. We instead use the new
WaitNoError method to avoid this.
2019-01-22 13:41:51 +01:00
Johan T. Halseth
0714ffa0d7
lntest/harness: add WaitNoError 2019-01-22 13:38:05 +01:00
Johan T. Halseth
adb512e6bf
lntest/node: print amounts in same format 2019-01-22 13:23:30 +01:00
Johan T. Halseth
7d34ce9d08
lnwire+multi: define HasMaxHtlc helper on msgFlags 2019-01-22 08:42:30 +01:00
Valentine Wallace
4fb1536f54
fundingmanager_test: verify max HTLC in ChannelUpdates
In this commit, we verify that ChannelUpdates for newly
funded channels contain the max HTLC that we expect.
We expect the max HTLC value of each ChannelUpdate to
equal the maximum pending msats in HTLCs required by
the remote peer.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:30 +01:00
Valentine Wallace
a66a1e113f
fundingmanager: when funding new channels, set a default max htlc
In this commit, we set a default max HTLC value in ChannelUpdates
sent out for newly funded channels. As a result, we also default
to setting `MessageFlags` equal to 1 in each new ChannelUpdate, since
the max HTLC field is an optional field and MessageFlags indicates
the presence of optional fields within the ChannelUpdate.

For a default max HTLC, we choose the maximum msats worth of
HTLCs that can be pending (or in-flight) on our side of the channel.
The reason for this is because the spec specifies that the max
HTLC present in a ChannelUpdate must be less than or equal to
both total channel capacity and the maximum in-flight amount set
by the peer. Since this in-flight value will always be less than
or equal to channel capacity, it is a safe spec-compliant default.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:30 +01:00
Johan T. Halseth
4f9de9bf1d
fundingmanager_test: add TestFundingManagerRejectInvalidMaxValueInFlight 2019-01-22 08:42:30 +01:00
Johan T. Halseth
4aa52d267f
funding+lnwallet: ensure max_htlc_value_in_flight smaller than capacity
Return an error to the remote if larger.
2019-01-22 08:42:29 +01:00
Valentine Wallace
f8e588e2e3
autopilot/graph: set max htlc in test edge policies
Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:29 +01:00
Valentine Wallace
cced754659
routing/router: when applying onion failure channel update, set max htlc
In this commit, we ensure that when we update an edge
as a result of a ChannelUpdate being returned from an
onion failure, the max htlc portion of the channel update
is included in the edge update.
2019-01-22 08:42:29 +01:00
Valentine Wallace
390b974793
server: set max htlc in ChannelUpdate createChannelUpdate
This method is called to convert an EdgePolicy to a ChannelUpdate. We
make sure to carry over the max_htlc value.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:29 +01:00
Valentine Wallace
207c4f030a
discovery/gossiper: include max HTLC when rebroadcasting stale channel updates
Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:29 +01:00
Valentine Wallace
19c403711e
discovery/chan_series+utils: include max htlc when syncing with peers
In this commit, we ensure that max HTLC is included when we're
synchronizing ChannelUpdates with remote peers.
2019-01-22 08:42:28 +01:00
Valentine Wallace
513ac23479
discovery/gossiper: persist remote channel policy updates' max htlc 2019-01-22 08:42:28 +01:00
Johan T. Halseth
01e679786d
channeldb/graph_test: add TestEdgePolicyMissingMaxHtcl 2019-01-22 08:42:28 +01:00
Johan T. Halseth
6fcc76fc68
channeldb/channel: ignore ChannelEdgePolicy with missing max_htlc
If the max_htlc field is not found when fetching a ChannelEdgePolicy
from the DB, we treat this as an unknown policy.

This is done to ensure we won't propagate invalid data further. The data
will be overwritten with a valid one when we receive an update for this
channel.

It shouldn't be very common, but old data could be lingering in the DB
added before this field was validated.
2019-01-22 08:42:28 +01:00
Johan T. Halseth
8dd074ee57
channeldb/graph_test: assert MaxHTLC field correctness
Co-authored-by: Valentine Wallace <valentine.m.wallace@gmail.com>
2019-01-22 08:42:27 +01:00
Valentine Wallace
69d4bf051f
channeldb/graph: add max HTLC to ChannelEdgePolicy
Adding this field will allow us to persist an edge's
max HTLC to disk, thus preserving it between restarts.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:27 +01:00