Commit Graph

7100 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
480ec3bbca
channeldb: ensure restored channels can't be mutated 2019-01-28 20:24:48 -08:00
Olaoluwa Osuntokun
4fd1f832d7
contractcourt+rpc: use new FetchChannel method instead of scanning
In this commit, we modify areas where we need to force close a channel
to use the new FetchChannel method instead of manually scanning. This
dramatically reduces the CPU usage when doing things like closing a
large number of channels within lnd.
2019-01-28 20:24:45 -08:00
Olaoluwa Osuntokun
821de3e107
test: remove FullSync call from breacharbiter_test.go
In this commit, we remove an extra openChannel.FullSync() call from
breacharbiter_test.go. Before this collective diff, calling
SyncPending() then FullSync() didn't result in an error. However, a
prior commit now makes this an error to ensure we don't attempt to
override any existing channels. This is the only area in the codebase
that we made this mistake which in this case, was benign.
2019-01-28 20:24:43 -08:00
Olaoluwa Osuntokun
9e5723e1bc
lnwallet: modify ForceClose to disallow closing if local data loss
In this commit, we ensure that if a channel is detected to have local
data loss, then we don't allow a force close attempt, as this may not be
possible, or cause us to play an invalid state.
2019-01-28 20:24:42 -08:00
Olaoluwa Osuntokun
b2b57314fa
channeldb: add new RestoreChannelShells method
In this commit, we add a new type (ChannelShell) along with a new
method, RestoreChannelShells which allows a caller to insert a series of
channel shells into the database. These channel shells will allow a
restored node to initiate the DLP protocol and recover their set of
existing channels.

When we insert a channel shell, we re-create the original link node, and
also add the outgoing edge to the channel graph. This way we can be sure
that upon start up, we attempt to connect to the remote peers, and that
the normal graph query commands will operate as expected.
2019-01-28 20:24:41 -08:00
Olaoluwa Osuntokun
fa30af0475
channeldb: when fetching/inserting commits check for ChanStatusRestored
If the ChanStatusRestored flag is set, then we don't need to write or
read the set of commits for a channel as they won't exist. This will be
the case when we restore a channel from an SCB.
2019-01-28 20:24:40 -08:00
Olaoluwa Osuntokun
c656788b0b
channeldb: add new methods to allow adding a new edge+policy w/ existing db transaction 2019-01-28 20:24:38 -08:00
Olaoluwa Osuntokun
11c6887ffa
channeldb: refactor syncPending to expose new syncNewChannel function
The new syncNewChannel function will allow callers to insert a new
channel given the OpenChannel struct, and set of addresses for the
channel peer. This new method will also create a new LinkNode for the
peer if one doesn't already exist.
2019-01-28 20:24:37 -08:00
Olaoluwa Osuntokun
7a88f580ea
channeldb: add new FetchChannelMethod
In this commit, we add a new method, FetchChannel which is required in
order to implement the new chanbackup.LiveChannelSource interface.
2019-01-28 20:24:36 -08:00
Olaoluwa Osuntokun
aaf6456e12
channeldb: add HasChanStatus and ApplyChanState methods to OpenChannel
These methods allow callers to properly query if a channel is in a
particular flag, and also modify the channel status in a thread safe
manner.
2019-01-28 20:24:35 -08:00
Olaoluwa Osuntokun
a410262dda
channeldb: modify the String() method of ChannelStatus reflect all flags
In this commit, we modify the String() method of the ChannelStatus type
to reflect the fact that it's a flag set. With these new changes, we'll
now print the variable name of each assigned bit with a bar delimiting
them all.
2019-01-28 20:24:33 -08:00
Olaoluwa Osuntokun
f57f40e767
channeldb: add prefix naming to ChannelStatus enum variables
In this commit, we add a prefix naming scheme to the ChannelStatus enum
variables. We do this as it enables outside callers to more easily
identify each individual enum variable as a part of the greater
enum-like type.
2019-01-28 20:24:32 -08:00
Olaoluwa Osuntokun
f4054d2a66 channeldb: add new AddrsForNode method
In this commit, we add a new AddrsForNode method. This method will allow
a wrapper sturct to implement the new chanbackup.LiveChannelSource
method which is required to implement the full SCB feature set.
2019-01-28 20:24:22 -08:00
Olaoluwa Osuntokun
e51c58e722
Merge pull request #2538 from Roasbeef/trickle-timer
config: increase default trickle delay from 30s to 1m30s
2019-01-28 19:43:08 -08:00
Conner Fromknecht
d7ac4bd6f9
Merge pull request #2543 from alexbosworth/patch-9
trivial: typo fix
2019-01-28 16:56:28 -08:00
Olaoluwa Osuntokun
ea6f7ccf2e
Merge pull request #2547 from moshababo/master
[trivial] lntest: FetchNodeInfo/initLightningClient docs update
2019-01-28 12:37:15 -08:00
Roei Erez
ff172bdd3a proper cleanup neutrino database in case of error 2019-01-28 18:21:26 +02:00
Moshe Shababo
8ed4e887be lntest: FetchNodeInfo/initLightningClient docs update 2019-01-27 17:05:30 +02:00
Alex Bosworth
d432ee6f13
trivial: typo fix
Correct log name of WalletKit
2019-01-26 16:57:09 -08:00
Olaoluwa Osuntokun
b56e95587f
config: increase default trickle delay from 30s to 1m30s
In this commit, we increase the default trickle delay from 30s to 1m30s.
We do this as before we implement the new INV gossip mechanism, we want
to de-emphasise the quick propagation of updates through the network
which eats up bandwidth.
2019-01-25 14:40:28 -08:00
Johan T. Halseth
6a78141a2d
Merge pull request #2536 from cfromknecht/notify-online-log
server: fix notify-online log message
2019-01-25 13:29:03 +01:00
Olaoluwa Osuntokun
e9889cb899
Merge pull request #2370 from Roasbeef/static-chan-backups-chanbackup
chanbackup: add new package implementing static channel backups
2019-01-24 16:14:24 -08:00
Conner Fromknecht
58f95322c0
server: fix notify-online log message 2019-01-23 20:40:27 -08:00
Olaoluwa Osuntokun
3b370fa08d
chanbackup: add new sub-system for keeping the on disk channels.backup up to date
In this commit, we introduce the chanbackup.SubSwapper interface. It
takes a regular Swapper implementation (defined by the
chanbackup.SubSwapper) interface along with a chanbackup.ChannelNotifier
implementation. Given these two interfaces, we're able to be notified
when a new channel is opened or closed, and then use the Swapper to
atomically replace the on-disk channel back up. As a result, a Lightning
daemon can ensure that they alwayts have a up to date channels.backup on
disk that can safely be copied away by users and be used to restore
channel funds in the event of partial/total data loss.
2019-01-23 18:11:33 -08:00
Olaoluwa Osuntokun
60999df08f
chanbackup: add new MultiFile struct for storing+updating Multis on disk
In this commit, we add a new MultiFile struct. We'll use this struct in
store the latest multi-channel backup on disk, swap it out atomically,
and finally extract+unpack the contents of the multi-file. The format
that's written to disk is the same as a regular Packed multi. The
contents of this new file are meant to be used to safely implement an
always up to date multi file on disk as a way for users to easily rsync
or fsnotiy (when it changes) the backup state of their channels.

We implement an atomic update and swap in the UpdateAndSwap. The method
uses relies on the underlying file system supporting an atomic rename
syscall. We first make a temporary backup file, write the latest
contents to that, then swap the temp file with the main file using
rename(2). This way, we ensure that we always have a single up to date
file, if the protocol aborts before the rename, then we can detect this,
remove the temp file, and attempt another swap.
2019-01-23 18:11:31 -08:00
Olaoluwa Osuntokun
7fcab83bb4
chanbackup: add functions to allow recovery of existing channel backups
In this commit, we add a series of functions that will allow users to
recover existing channel backups. We do this using two primary
interfaces: the ChannelRestorer, and the PeerConnector. The first
interfaces allows us to abstract away the details w.r.t exactly how a
channel is restored. Instead, we simply expect that the channel backup
will be inserted as a sort of "channel shell" which contains only the
data required to initiate the data loss protection protocol. The second
interface is how we instruct the Lightning node to connect out to the
channel peer given its known addresses.
2019-01-23 18:11:29 -08:00
Olaoluwa Osuntokun
1348c6b35b
chanbackup: add functionality to allow external callers to backup channels
In this commit, we introduce a series of interfaces and methods that
will allow external callers to backup either all channels, or a specific
channel identified by its channel point. In order to abstract away the
details w.r.t _how_ we obtain the set of open channels, or their storage
mechanisms, we introduce a new LiveChannelSource interfaces. This
interfaces allows us to fetch all channels, a channel by its channel
point, and also all the known addresses for a node as we'll need this in
order to connect out to the node in the case of a recovery attempt.
2019-01-23 18:11:28 -08:00
Olaoluwa Osuntokun
71df4b0545
chanbackup: introduce Multi, a multi-channel backup
In this commit, we introduce the Multi sturct. Multi is a series of
static channel backups. This type of backup can contains ALL the channel
backup state in a single packed blob. This is suitable for storing on
your file system, cloud storage, etc. Systems will be in place within
lnd to ensure that one can easily obtain the latest version of the Multi
for the node, and also that it will be kept up to date if channel state
changes.
2019-01-23 18:11:27 -08:00
Olaoluwa Osuntokun
ee71e16593
chanbackup: add implementation of Single Channel Backups (SCBs)
In this commit, we add the initial implementation of the SCB structure.
Given an SCB, and a user's seed, it will be possible to recover the
settled balanced of a channel in the event of total or partial data
loss. The SCB contains all information required to initiate the data
loss protection protocol once we restore the channel and connect to the
remote channel peer.

The primary way outside callers will interact with this package are via
the Pack and Unpack methods. Packing means writing a
serialized+encrypted version of the SCB to an io.Writer. Unpacking does
the opposite.

The encoding format itself uses the same encoding as we do on the wire
within Lightning. Each encoded backup begins with a version so we can
easily add or modify the serialization format in the future, if new
channel types appear, or we need to add/remove fields.
2019-01-23 18:11:26 -08:00
Olaoluwa Osuntokun
d98a7e8138
chanbackup: add logger 2019-01-23 18:11:25 -08:00
Olaoluwa Osuntokun
67c6d0d331
chanbackup: implement crypto operations for serialized static channel baackups
In this commit, we implement a series of new crypto operations that will
allow us to encrypt and decrypt a set of serialized channel backups.
Their various backups may have distinct encodings when serialized, but
to the functions defined in this file, we treat them as simple opaque
blobs.

For encryption, we utilize chacha20poly1305 with a random 24 byte nonce.
We use a larger nonce size as this can be safely generated via a CSPRNG
without fear of frequency collisions between nonces generated. To
encrypt a blob, we then use this nonce as the AD (associated data) and
prepend the nonce to the front of the ciphertext package.

For key generation, in order to ensure the user only needs their
passphrase and the backup file, we utilize the existing keychain to
derive a private key. In order to ensure that at we don't force any
hardware signer to be aware of our crypto operations, we instead opt to
utilize a public key that will be hashed to derive our private key. The
assumption here is that this key will only be exposed to this software,
and never derived as a public facing address.
2019-01-23 18:11:21 -08:00
Olaoluwa Osuntokun
eff9899d14 keychain: add new KeyFamily: KeyFamilyStaticBackup for SCB encryption keys 2019-01-23 18:11:11 -08:00
Olaoluwa Osuntokun
ecd5541d55
Merge pull request #2350 from halseth/autopilot-dryrun-only
[Autopilot] Add heuristic dry-run option
2019-01-23 17:59:00 -08:00
Conner Fromknecht
e0cf99c53d
Makefile: use go-acc for coverage 2019-01-23 17:42:52 -08:00
Olaoluwa Osuntokun
6c610d977f
Merge pull request #2075 from wpaulino/settle-invoice-on-chain-sweep
contractcourt: settle invoice when claiming HTLC on-chain
2019-01-23 16:50:54 -08:00
Olaoluwa Osuntokun
7a6b4e2fc5
Merge pull request #2532 from wpaulino/btcwallet-initial-sync
build: update btcwallet dependency to new initial sync definition
2019-01-23 16:41:03 -08:00
Johan T. Halseth
5306ed5b68
lncli: add autopilot query command
To query the autopilot heuristics for scores.
2019-01-23 09:05:43 +01:00
Johan T. Halseth
8874be764e
lnrpc/autopilotrpc: implement QueryScores RPC 2019-01-23 09:05:43 +01:00
Johan T. Halseth
a654be5884
lnrpc/autopilotrpc: add QueryScores API
Adds a new lnrpc API that lets the caller get ("dry run") the scores
that would be given to a set of node by the available autopilot
heuristics.
2019-01-23 09:05:43 +01:00
Johan T. Halseth
8f54a2bd6f
autopilot/manager: add QueryHeuristics
Proxies a query request to the active agent.
2019-01-23 09:05:43 +01:00
Johan T. Halseth
6c556fd92a
autopilot/agent: add queryHeuristics
This commit adds a method queryHeuristic that will query all simple
heuristics for node scores, in addition to the current active heuristic.
2019-01-23 09:05:43 +01:00
Johan T. Halseth
cf01879d96
autopilot/agent: protect chanState with mutex
Since we want to access the current channel state outside the main
autopilot agent goroutine, we make sure it is protected by a mutex.
2019-01-23 09:05:42 +01:00
Johan T. Halseth
8ab718368d
lnd+pilot+autopilot: use config to set active autopilot heuristics 2019-01-23 09:05:42 +01:00
Johan T. Halseth
36e7694e8e
config+pilot: define autopilot.heuristic
And validate the config set by the user.
2019-01-23 09:05:42 +01:00
Johan T. Halseth
c0fed861d2
autopilot/interface: add Name() to AttachmentHeuristic interface 2019-01-23 09:05:42 +01:00
Wilmer Paulino
016add6adc
lnd_test: extend chain claim test with invoice settled check
In this commit, we extend the remote/receiver chain claim integration
test to assert that the on-disk representation of the invoice on the
receiving side (Carol) is marked as settled due to the claiming the HTLC
on-chain.
2019-01-22 20:46:14 -08:00
Wilmer Paulino
41f638c7cf
contractcourt+server: settle invoice for on-chain HTLC sweep
In this commit, we extend the htlcSuccessResolver to settle the invoice,
if any, of the corresponding on-chain HTLC sweep. This ensures that the
invoice state is consistent as when claiming the HTLC "off-chain".
2019-01-22 20:46:14 -08:00
Wilmer Paulino
974e0f2df5
cnct: make original htlc amt available for contract resolvers
Previously, contract resolvers that needed to publish a second level tx,
did not have access to the original htlc amount.

This commit reconstructs this amount from data that is already persisted
in arbitrator log.

Co-authored-by: Joost Jager <joost.jager@gmail.com>
2019-01-22 20:47:17 -08:00
Wilmer Paulino
93754f8d37
lnwallet: add HtlcPoint() resolution helper methods
Co-authored-by: Joost Jager <joost.jager@gmail.com>
2019-01-22 20:47:13 -08:00
Wilmer Paulino
865f7568d6
cnct: extract relaunching of resolvers to method
Co-authored-by: Joost Jager <joost.jager@gmail.com>
2019-01-22 20:46:14 -08:00