Commit Graph

674 Commits

Author SHA1 Message Date
Joost Jager
f47de09850
channeldb: log error for migration test failures 2019-09-28 08:23:49 +02:00
Olaoluwa Osuntokun
b399203e71
lnwallet: update channel state machine to be aware of tweakless commits
In this commit, we update the channel state machine to be aware of
tweakless commits. In several areas, we'll now check the channel's type
to see if it's `SingleFunderTweakless`. If so, then we'll opt to use the
remote party's non-delay based point directly in the script, skipping
any additional cryptographic operations. Along the way we move the
`validateCommitmentSanity` method to be defined _before_ it's used as is
cutomary within the codebase.

Notably, within the `NewUnilateralCloseSummary` method, we'll now _blank
out_ the `SingleTweak` value if the commitment is tweakless. This
indicates to callers the witness type they should map to, as the value
isn't needed at all any longer when sweeping a non-delay output.

We also update the signing+verification tests to also test that we're
able to properly generate a valid witness for the new tweakless
commitment format.
2019-09-25 18:25:43 -07:00
Olaoluwa Osuntokun
21133e1856
channeldb: define new channel type, SingleFunderTweakless
In this commit, we define a new channel type: SingleFunderTweakless.
We'll use this channel type to denote channels with commitments that
don't tweak the remote party's key in their non-delay output.
2019-09-25 18:25:40 -07:00
Johan T. Halseth
2a6ad6e634
channeldb+lnwallet: don't pass isRestoredChan to ChanSyncMsg
Since we have access to the internal state of the channel, we can
instead get it directly instead of passing it in as a parameter.
2019-09-25 14:04:44 +02:00
Johan T. Halseth
ac0e9b6016
channeldb/channel: add BroadcastedCommitment 2019-09-25 14:04:43 +02:00
Johan T. Halseth
02b2787e44
multi: make MarkCommitmentBroadcasted take closeTx 2019-09-25 14:04:43 +02:00
Johan T. Halseth
a810092e53
channeldb/channel: make putChanStatus take optional extra closures 2019-09-25 14:04:43 +02:00
Johan T. Halseth
eb1b84c0b4
channeldb+lnwallet: make ChanSyncMsg method on OpenChannel 2019-09-25 14:04:42 +02:00
Johan T. Halseth
0122dda88a
channeldb/channel: remove unused FullSync method
The exported FullSync method is only used by test code, so we remove it
and instead use SyncPending.
2019-09-25 14:04:42 +02:00
Olaoluwa Osuntokun
18f88cbd8d
Merge pull request #3440 from joostjager/buildroute
routing: add build route functionality
2019-09-24 20:24:24 -07:00
Joost Jager
339ff357d1
channeldb: invalidate channel signature cache on update 2019-09-23 13:07:04 +02:00
Joost Jager
a0e91b5d41
channeldb: add forward fee calculation function 2019-09-17 11:11:02 +02:00
Joost Jager
8c44cf4a22
channeldb: add ComputeFee function 2019-09-17 11:10:58 +02:00
Joost Jager
95502da7e8
Merge pull request #3414 from joostjager/report-accept-height
lnwire+htlcswitch: report htlc accept height
2019-09-16 13:49:29 +02:00
Joost Jager
d3e206ef95
invoices: return accept height in hodl event
This is a preparation for passing back the accept height in the
incorrect payment details failure message to the sender.
2019-09-16 10:10:16 +02:00
Joost Jager
49a20a87a2
channeldb+invoices: make htlc cancelation stricter
Previously it was possible to cancel a canceled htlc. This would
subtract the htlc amount from the invoice amount again.
2019-09-16 10:10:13 +02:00
Joost Jager
3ab4c749c6
channeldb: remove unsupported migrations
To reduce maintenance on migration code, we move to only supporting
upgrades from previous major releases.
2019-09-13 10:13:00 +02:00
Joost Jager
f1942a4c33
channeldb: require minimum db upgrade version 2019-09-13 10:12:58 +02:00
Lars Lehtonen
2f51ccd10b channeldb: Fix dropped error in migrations test
channeldb: Fix dropped error and wrap with context

channeldb: Fix empty error condition in waitingproof test

channeldb: Fix empty error condition in codec

channeldb: Wrap error in context
2019-09-12 15:49:21 +00:00
Joost Jager
93b40440c5
Merge pull request #3488 from joostjager/fix-channeldb-test
channeldb/test: make route comparison a pure function
2019-09-12 00:11:17 +02:00
Joost Jager
3d7de2ad39
multi: remove dead code 2019-09-10 17:21:59 +02:00
Joost Jager
62a9c2c3ac
channeldb/test: make route comparison a pure function
Previously the route to compare was modified in order for DeepEqual to
function properly. This created problems when tests were ran in
parallel.
2019-09-10 15:46:49 +02:00
Joost Jager
7f4af8f9ae
channeldb/test: make hop record comparison stricter 2019-09-10 15:46:48 +02:00
Olaoluwa Osuntokun
62dcc4b7d5
channeldb: ensure test for migration 9 is fully enclosed
In this commit, we ensure that the test for migration 9 uses the same
encoding/decoding functions as was present in the repo when the
migration was first added. Otherwise, the test will fail as it'll try to
use the decoding functions of master (migration 10 and onwards) rather
than the decoding function of migration 9.
2019-09-09 03:57:52 -07:00
chokoboko
ac617a0d01 channeldb: return errors in various missed places 2019-09-09 02:41:43 -07:00
Olaoluwa Osuntokun
754809406d
channeldb: fix migration bug due to interplay between migration #9 and #10
In this commit, we fix an issue that was recently introduced as a result
of migration #10. The new TLV format ended up modifying the
serialization functions called in `serializePaymentAttemptInfo`.
Migration #9, also used this `serializePaymentAttemptInfo` method to
serialize the _new_ (pre TLV, but new payment attempt structure) routes
into the database during its migration. However, migration #10 failed to
copy over the existing unmodified `serializePaymentAttemptInfo` method
into the legacy serialization for migration #9. As a result, once
migration #9 was run, the routes/payments were serialized using the
_new_ format, rather than the format used for v0.7.1. This then lead to
de-serialization either failing, or causing partial payment corruption
as migration #10 was expecting the "legacy" format (no TLV info).

We fix this issue by adding a new fully enclosed
`serializePaymentAttemptInfoMigration9`method that will be used for
migration #9. Note that our tests didn't catch this, as they test the
migration in isolation, rather than in series which is how users will
encounter the migrations.

Fixes #3463.
2019-09-09 02:16:13 -07:00
Joost Jager
d6d9ec6aa5
invoices: replay awareness
Previously the invoice registry wasn't aware of replayed htlcs. This was
dealt with by keeping the invoice accept/settle logic idempotent, so
that a replay wouldn't have an effect.

This mechanism has two limitations:

1. No accurate tracking of the total amount paid to an invoice. The total
amount couldn't just be increased with every htlc received, because it
could be a replay which would lead to counting the htlc amount multiple
times. Therefore the total amount was set to the amount of the first
htlc that was received, even though there may have been multiple htlcs
paying to the invoice.

2. Impossible to check htlc expiry consistently for hodl invoices. When
an htlc is new, its expiry needs to be checked against the invoice cltv
delta. But for a replay, that check must be skipped. The htlc was
accepted in time, the invoice was moved to the accepted state and a
replay some blocks later shouldn't lead to that htlc being cancelled.
Because the invoice registry couldn't recognize replays, it stopped
checking htlc expiry heights when the invoice reached the accepted
state. This prevents hold htlcs from being cancelled after a restart.
But unfortunately this also caused additional htlcs to be accepted on an
already accepted invoice without their expiry being checked.

In this commit, the invoice registry starts to persistently track htlcs
so that replays can be recognized. For replays, an htlc resolution
action is returned early. This fixes both limitations mentioned above.
2019-09-04 19:20:31 +02:00
Joost Jager
53eea09b63
channeldb: add now function
Needed for time control in unit tests.
2019-09-04 19:20:29 +02:00
Joost Jager
144856757d
channeldb+invoices: move invoice cancel logic into registry
This commit is a continuation of the centralization of invoice state
transition logic in the invoice registry.
2019-09-04 19:20:25 +02:00
Joost Jager
416bc8c68c
channeldb+invoices: move hold invoice settle logic into registry
This commit is a continuation of the centralization of invoice state
transition logic in the invoice registry.
2019-09-04 19:20:23 +02:00
Joost Jager
ad3522f1a6
channeldb+invoices: move invoice accept or settle logic into registry
As the logic around invoice mutations gets more complex, the friction
caused by having this logic split between invoice registry and channeldb
becomes more apparent. This commit brings a clearer separation of
concerns by centralizing the accept/settle logic in the invoice
registry.

The original AcceptOrSettle method is renamed to UpdateInvoice because
the update to perform is controlled by the callback.
2019-09-04 19:20:21 +02:00
Joost Jager
4105142c96
channeldb+invoices: add invoice htlcs
This commit adds a set of htlcs to the Invoice struct and
serializes/deserializes this set to/from disk. It is a preparation for
accurate invoice accounting across restarts of lnd.

A migration is added for the invoice htlcs.

In addition to these changes, separate final cltv delta and expiry
invoice fields are created and populated. Previously it was required
to decode this from the stored payment request. The reason to create
a combined commit is to prevent multiple migrations.
2019-09-04 19:20:17 +02:00
Joost Jager
061b34b924
channeldb: add int64 to codec 2019-09-04 19:20:15 +02:00
Joost Jager
762609a169
channeldb: fix suppressed error 2019-09-04 19:20:05 +02:00
Olaoluwa Osuntokun
441b15055b
channeldb: properly compare payment attempts in serialization test 2019-08-23 12:50:25 -07:00
Olaoluwa Osuntokun
c78e3aaa9d
channeldb: update route.Hop serialization to include new EOB related fields
We also include a migration for the existing routes stored on disk.
2019-08-22 18:53:04 -07:00
Johan T. Halseth
21baa7bf18
multi: fix linter errors 2019-08-07 10:53:10 +02:00
Olaoluwa Osuntokun
7767eecbb8 Merge pull request #3164 from joostjager/persistent-mc
routing: persistent mission control
2019-08-02 15:45:49 -07:00
Olaoluwa Osuntokun
69c9e2b732
Merge pull request #3278 from Crypt-iQ/bbolt_options_0707
channeldb: specify freelist bbolt options by default
2019-07-30 16:52:28 -07:00
Joost Jager
eb4e65e54f
channeldb: export route serialization 2019-07-29 13:42:57 +02:00
Olaoluwa Osuntokun
8c389d13f9
Merge pull request #3197 from breez/optimize_prune_zombie_channels
Optimize prune zombie channels
2019-07-18 20:56:56 -07:00
Conner Fromknecht
0223039fb4
channeldb/graph: add NumZombies method and tests 2019-07-16 16:32:27 -07:00
Roei Erez
7300f33fe2 graph: implement index for disabled channels.
This commit adds an index bucket, disabledEdgePolicyBucket, for those
ChannelEdgePolicy with disabled bit on.
The main purpose is to be able to iterate over these fast when prune is
needed without the need for iterating the whole graph.

The entry points for accessing this index are:
1. When updating ChannelEdgePolicy - insert an entry.
2. When deleting ChannelEdge - delete the associated entries.
3. When querying for disabled channels - implemented DisabledChannelIDs
function
2019-07-16 17:48:07 +03:00
nsa
61a1ab08fb routing+channeldb: findPath uses Vertex instead of LightningNode
This commit modifies the nodeWithDist struct to use a route.Vertex
instead of a *channeldb.LightningNode. This change, coupled with
the new ForEachNodeChannel function, allows the findPath Djikstra's
algorithm to cut down on database lookups since we no longer need
to call the FetchOtherNode function.
2019-07-12 06:08:01 -04:00
Olaoluwa Osuntokun
ba5fbb3c27
Merge pull request #3156 from joostjager/extended-fail
routerrpc: add more failure reasons and route hints
2019-07-08 19:12:03 -07:00
nsa
66d15c8e76 channeldb + wtdb: specify freelist bbolt options by default
This commit specifies two bbolt options when opening the underlying
channel and watchtower databases so that there is reduced heap
pressure in case the bbolt database has a lot of free pages in the
B+ tree.
2019-07-08 18:41:25 -04:00
Joost Jager
5ce04091d8
routing+routerrpc+channeldb: return route on invalid payment details 2019-07-04 09:27:16 +02:00
Joost Jager
ae46fb00cb
routing+channeldb: add more failure reasons 2019-07-04 09:27:12 +02:00
Johan T. Halseth
97dda4f8c8
channeldb/migrations: skip migration for empty outgoing payments bucket
Previously the migration would fail if the source node was not set in
the database. Since we know that the source node must have been set
before making any payments, we check whether there actually are any
payments to migrate, and return early if not.
2019-06-18 21:14:16 +02:00
Johan T. Halseth
f98f452528
channeldb: avoid modifying bucket during cursor traversal
This is not safe according to bbolt documentation.
2019-06-14 21:09:03 +02:00
Johan T. Halseth
b7a37728c0
channeldb/migrations: avoid modifying bucket during ForEach loop
bbolt documenation state that this is not safe and leads to undefined
behavior.
2019-06-14 21:09:02 +02:00
Joost Jager
18c025151a
invoices+channeldb: move invoice state check to invoiceregistry 2019-06-10 16:12:08 +02:00
Joost Jager
2644759924
channeldb: add todo for more detailed failure reasons 2019-06-05 12:41:55 +02:00
Joost Jager
f03533c67a
routerrpc: convert sendpayment to async
Modify the routerrpc SendPayment api to asynchronous. This allows
callers to pick up a payment after the rpc connection was lost or lnd
was restarted.
2019-06-05 12:41:53 +02:00
Joost Jager
eb2647e8fc
channeldb: add subscription to control tower
Allows other sub-systems to subscribe to payment success and fail
events.
2019-06-05 12:41:49 +02:00
Joost Jager
87d3207baf
channeldb+routing: move control tower interface to routing
This commit creates an empty shall for control tower in the routing
package. It is a preparation for adding event notification.
2019-06-05 12:41:47 +02:00
Johan T. Halseth
7cb25a5afb
channeldb/control_tower test: add TestPaymentControlDeleteNonInFlight
TestPaymentControlDeleteNonInFlight checks that calling DeletaPayments
only deletes payments from the database that are not in-flight.
2019-05-27 20:19:00 +02:00
Johan T. Halseth
589f0fcc5d
control_tower: make initial bucket bucket creation stricter 2019-05-27 20:18:59 +02:00
Johan T. Halseth
95b4828780
channeldb: derive PaymentStatus implicitly from stored info 2019-05-27 20:18:59 +02:00
Johan T. Halseth
d705b8a013
channeldb+multi: rename Grounded->Unknown, Completed->Succeeded 2019-05-27 20:18:59 +02:00
Johan T. Halseth
d78d3f50b9
channeldb: move deprecated code to migration_09_legacy_serialization.go
And unexport deprecated code.
2019-05-27 20:18:59 +02:00
Johan T. Halseth
1b788904f0
channeldb+router: record payment failure reason in db 2019-05-27 20:18:59 +02:00
Johan T. Halseth
677cb018c9
routing/control_tower: add FetchInFlightPayments 2019-05-27 20:18:58 +02:00
Johan T. Halseth
de1bf8a518
routing/router: persist payment state machine
This commit makes the router use the ControlTower to drive the payment
life cycle state machine, to keep track of active payments across
restarts.  This lets the router resume payments on startup, such that
their final results can be handled and stored when ready.
2019-05-27 20:18:58 +02:00
Johan T. Halseth
59c2557cc9
channeldb/control_tower test: test new payment behavior 2019-05-27 20:18:57 +02:00
Johan T. Halseth
1af1832ff7
channeldb/control_tower: add payment information during state changes
This commit gives a new responsibility to the control tower, letting it
populate the payment bucket structure as the payment goes through
its different stages.

The payment will transition states Grounded->InFlight->Success/Failed,
where the CreationInfo/AttemptInfo/Preimage must be set accordingly.

This will be the main driver for the router state machine.
2019-05-27 20:18:57 +02:00
Johan T. Halseth
6d80661bbb
channeldb/payments: add StatusFailed 2019-05-27 20:18:57 +02:00
Johan T. Halseth
bb4aadd16c
channeldb/control_tower: remove non-strict option
Since we have performed a migration, the db should be in a consistent
state, and we can remove the non-strict option.
2019-05-27 20:18:57 +02:00
Johan T. Halseth
b7189ba028
channeldb test: add TestOutgoingPaymentsMigration 2019-05-27 20:18:57 +02:00
Johan T. Halseth
693807cf6e
channeldb/control_tower: add FetchSentPayments 2019-05-27 20:18:56 +02:00
Johan T. Halseth
801521ed2d
channeldb/migration: add migration for new payment bucket structure
migrateOutgoingPayments moves the OutgoingPayments into a new bucket format
where they all reside in a top-level bucket indexed by the payment hash. In
this sub-bucket we store information relevant to this payment, such as the
payment status.

To avoid that the router resend payments that have the status InFlight (we
cannot resume these payments for pre-migration payments) we delete those
statuses, so only Completed payments remain in the new bucket structure.
2019-05-27 20:18:56 +02:00
Johan T. Halseth
178996f0d3
channeldb/payments+control_tower: split OutgoingPayments
This commit changes the format used to store payments within the
DB. Previously this was serialized as one continuous struct
OutgoingPayment, which also contained an Invoice struct we where only
using a few fields of. We now split it up into two simpler sub-structs
CreationInfo, AttemptInfo and PaymentPreimage.

We also want to associate the payments more closely with payment
statuses, so we move to this hierarchy:

There's one top-level bucket "sentPaymentsBucket" which contains a set
of sub-buckets indexed by a payment's payment hash. Each such sub-bucket
contains several fields:
paymentStatusKey -> the payment's status
paymentCreationInfoKey -> the payment's CreationInfo.
paymentAttemptInfoKey -> the payment's AttemptInfo.
paymentSettleInfoKey -> the payment's preimage (or zeroes for
non-settled payments)

The CreationInfo is information that is static during the whole payment
lifcycle. The attempt info is set each time a new payment attempt
(route+paymentID) is sent on the network. The preimage is information
only known when a payment succeeds.  It therefore makes sense to split
them.

We keep legacy serialization code for migration puproses.
2019-05-27 20:18:56 +02:00
Johan T. Halseth
f022810f8b
channeldb/codec: add privateKey serialization 2019-05-27 20:18:56 +02:00
Johan T. Halseth
2417f40532
channeldb: put payment status in new bucket
We move the payment status to a new bucket hierarchy. Old buckets and
fetch methods are kept around for migration purposes.
2019-05-27 20:18:56 +02:00
Johan T. Halseth
d027e10201
htlcswitch+channeldb: move control tower to channeldb 2019-05-27 20:18:56 +02:00
Conner Fromknecht
28bf49807e
watchtower/wtdb: add CSessionStatus field to ClientSession
This commit adds persisted status bit-field to ClientSessions, that can
be used to modify behavior of their handling in the client. Currently,
only a default CSessionActive status is defined. However, the intention
is that this could later be used to signal that a session is abandoned
without needing to perform a db migration to add the field. As we move
forward with testing, this will likely be useful if a session gets
borked and we need a simple method of the client to temporarily ignore
certain sessions.

The field may be useful in signaling other types of status changes,
though this was the primary motivation that warranted the addition.
2019-05-24 18:24:41 -07:00
Conner Fromknecht
2a904cb69f
watchtower/wtdb: add Encode/Decode methods to wtclient structs 2019-05-23 20:48:08 -07:00
Johan T. Halseth
15bed506b1
routing/route+multi: remove redundant TotalFees field
Instead get it on demand using method TotalFees().
2019-05-16 23:58:32 +02:00
Joost Jager
064e8492de
cnct+htlcswitch+invoices: move invoice parameter check out of link
This commit is the final step in making the link unaware of invoices. It
now purely offers the htlc to the invoice registry and follows
instructions from the invoice registry about how and when to respond to
the htlc.

The change also fixes a bug where upon restart, hodl htlcs were
subjected to the invoice minimum cltv delta requirement again. If the
block height has increased in the mean while, the htlc would be canceled
back.

Furthermore the invoice registry interaction is aligned between link and
contract resolvers.
2019-05-15 14:42:12 +02:00
AdamISZ
6e054886d8 Typo corrections in various comments 2019-05-10 17:15:54 +02:00
Olaoluwa Osuntokun
fb1819bf7f
Merge pull request #3064 from halseth/route-serialization
[Reliable payments] add routing.Route (de)serialization + test
2019-05-09 16:46:14 -07:00
Johan T. Halseth
21c989ffcc
channeldb/graph: don't nil curve of returned PubKey
Nilling the key would cause signature verification to crash if the
returned key was used.

A test to exercise the behavior is added.
2019-05-09 10:49:58 +02:00
Johan T. Halseth
89fd43ebcb
channeldb: add Route (de)serialization + test
We will store the routes used during payment attempts, so we need
serialization code.
2019-05-09 09:59:43 +02:00
Conner Fromknecht
678ca9feff
channeldb/codec: add NewUnknownElementType constructor 2019-04-26 17:20:23 -07:00
Wilmer Paulino
0b0a9f4172
channeldb+routing: refactor DeleteChannelEdge to use ChannelID
In this commit, we refactor DeleteChannelEdge to use ChannelIDs rather
than ChannelPoints. We do this as the only use of DeleteChannelEdge is
when we are pruning zombie channels from our graph. When running under a
light client, we are unable to obtain the ChannelPoint of each edge due
to the expensive operations required to do so. As a stop-gap, we'll
resort towards using an edge's ChannelID instead, which is already
gossiped between nodes.
2019-04-18 21:57:41 -07:00
Wilmer Paulino
292defd6ba
channeldb: add IsDisabled method to ChannelEdgePolicy 2019-04-18 21:57:32 -07:00
Conner Fromknecht
f82b7c9bac
channeldb/graph: remove MarkEdgeZombie
This commit removes the MarkEdgeZombie method from channeldb. This
method is currently not used in any live code paths in production, and
is only used in unit tests. However, incorrect usage of this method
could result in an edge being present in both the zombie and channel
indexes, which deviates from any state we would expect to see in
production. Removing the method will help mitigate the potential for
writing incorrect unit tests in the future, by forcing zombie edges to
be created via the relevant, production APIs, e.g. DeleteChannelEdge.

The existing unit tests that use this method have been modified to use
the DeleteChannelEdge instead. No regressions were discovered in the
process.
2019-04-09 22:12:02 -07:00
Conner Fromknecht
4a755435e6
channeldb/graph: skip unknown edges in FetchChanInfos
This commit modifies FetchChanInfos to skip any channels that are not in
the graph at the time of the call. Currently the entire call will fail
if the edge is not found, which stalls a gossip sync in the following
scenario:

 1. Remote peer queries for a channel range
 2. We return the set of channel ids in that range
 3. A channel from that set is removed from the graph, e.g. via close.
 4. Remote peer queries for removed edge, causing the query to fail.

To remedy this, we will now skip any edges that are not known in the
database at the time of the query. This prevents the syncer state
machines from halting, which otherwise could only be resolved by
disconnecting and reconnecting.
2019-04-09 17:35:58 -07:00
Conner Fromknecht
e91bacd1bc
channeldb/graph: filter zombie channels in FilterKnownChanIDs
This commit modifies FilterKnownChanIDs to skip edges that
we ourselves have deemed zombies. This prevents us from requesting
the updates from them, as this wastes bandwidth and cpu cycles.
2019-04-05 13:01:08 -07:00
Conner Fromknecht
5d98a94d60
channeldb: write chan updates through reject+channel cache 2019-04-01 16:34:51 -07:00
Conner Fromknecht
6d3e081f7b
channeldb: accept cache sizes in ChannelGraph 2019-04-01 16:33:36 -07:00
Conner Fromknecht
baa968b1ff
channeldb/options: add Options w/ functional modifiers 2019-04-01 16:33:13 -07:00
Conner Fromknecht
ae3a00a5da
channeldb/graph+db: integrate reject and channel caches 2019-04-01 16:32:52 -07:00
Conner Fromknecht
b20a254faa
channeldb/channel_cache: add channelCache w/ randomized eviction 2019-04-01 16:25:31 -07:00
Conner Fromknecht
af0ea3590b
channeldb/reject_cache: add rejectCache w/ randomized eviction 2019-04-01 16:25:31 -07:00
Conner Fromknecht
3c46ceec1d
channeldb/db: init one ChannelGraph per channeldb.DB 2019-04-01 16:25:30 -07:00
Conner Fromknecht
ecbb78651a
channeldb/graph: add newChannelGraph constructor 2019-04-01 16:25:30 -07:00
Conner Fromknecht
ac315fd051
channeldb/db: remove unused buffer pool 2019-04-01 16:25:30 -07:00
Conner Fromknecht
2f927493c8
Revert "channeldb: convert invoice settle/cancel calls to use Batch"
This reverts commit da76c34418a40761fdc99e11200755661699f7eb.
2019-04-01 16:25:30 -07:00
Conner Fromknecht
e35f676b4c
Revert "channeldb: convert concurrent channel state machine calls to use Batch"
This reverts commit e8da6dd0b48843a48cecdcff894f48e8020d940e.
2019-04-01 16:25:30 -07:00
Conner Fromknecht
f6d93c8e5d
Revert "channeldb: convert all Update calls to use Batch"
This reverts commit 3555d3dbd431fe3ddf2660d4dc270b5a595fe517.
2019-04-01 16:25:26 -07:00
Olaoluwa Osuntokun
b8cbe3a1f8
channeldb: in RestoreChannelShells don't exit if edge already exists
During the restore process, it may be possible that we have already
heard about our prior edge from a node on the network (or our channel
peers). As a result, we shouldn't exit if this happens, and instead
should continue with the rest of the restoration process.
2019-03-28 17:54:06 -07:00
Olaoluwa Osuntokun
64b8facf0e
channeldb: don't read/write funding transactions if a restore channel 2019-03-28 17:53:43 -07:00
Olaoluwa Osuntokun
19ef4bbcb9
channeldb: within AddrsForNode don't fail if no graph node is found
In this commit, we modify the `AddrsForNode` method to not fail if no
graph node is found. We do this as when the backup is being
restored/created, it's possible that we don't yet have a
NodeAnnouncement for that node.
2019-03-28 17:53:42 -07:00
Olaoluwa Osuntokun
b93ff26265
channeldb: set restored chan status within RestoreChannelShells
In this commit, we move the location where we restore the channel status
to within the `RestoreChannelShells` method itself. Before this commit,
we attempted to use `ApplyChanStatus` which creates a DB transaction and
relies on a fully populated channel state, which in the restoration
case, we don't yet have.
2019-03-28 17:53:41 -07:00
Olaoluwa Osuntokun
13e7244d14
channeldb: also restore channel capacity in RestoreChannelShells 2019-03-28 17:53:40 -07:00
Wilmer Paulino
c82d73a826
channeldb+routing: extend edge lookup methods with zombie index check
In this commit, we extend the graph's FetchChannelEdgesByID and
HasChannelEdge methods to also check the zombie index whenever the edge
to be looked up doesn't exist within the edge index. We do this to
signal to callers that the edge is known, but only as a zombie, and the
only information that we have about the edge are the node public keys of
the two parties involved in the edge.

In the event that an edge does exist within the zombie index, we make
an additional check on edge policies to ensure they are not within the
router's pruning window, indicating that it is a fresh update.
2019-03-27 13:06:57 -07:00
Wilmer Paulino
e98f4d6d9d
channeldb: extend DeleteChannelEdge to mark edge as zombie
We mark the edges as zombies when pruning them to ensure we don't
attempt to reprocess them later on. This also applies to channels that
have been removed from the graph due to being stale.
2019-03-27 13:06:34 -07:00
Wilmer Paulino
b780dfacdb
channeldb: add zombie edge index
In this commit, we add a zombie edge index to the database. This allows
us to quickly determine across restarts whether we're attempting to
process an edge we've previously deemed as zombie.
2019-03-27 13:06:12 -07:00
Wilmer Paulino
a26a643273
channeldb: remove unused buckets 2019-03-27 13:05:48 -07:00
Olaoluwa Osuntokun
da76c34418
channeldb: convert invoice settle/cancel calls to use Batch 2019-03-19 17:11:17 -07:00
Olaoluwa Osuntokun
e8da6dd0b4
channeldb: convert concurrent channel state machine calls to use Batch 2019-03-19 17:11:14 -07:00
Olaoluwa Osuntokun
3555d3dbd4 channeldb: convert all Update calls to use Batch
In this commit, we convert all the `Update` calls which are serial, to
use `Batch` calls which are optimistically batched together for
concurrent writers. This should increase performance slightly during the
initial graph sync, and also updates at tip as we can coalesce more of
these individual transactions into a single transaction.
2019-03-19 17:10:46 -07:00
Joost Jager
32f2b047e8
htlcswitch: hodl invoice
This commit modifies the invoice registry to handle invoices for which
the preimage is not known yet (hodl invoices). In that case, the
resolution channel passed in from links and resolvers is stored until we
either learn the preimage or want to cancel the htlc.
2019-03-15 10:09:17 +01:00
Joost Jager
19f79613df
channeldb: store hold invoice 2019-03-15 10:08:55 +01:00
Joost Jager
8392f6d28f
lnrpc/invoicesrpc: remove lnrpc type from add invoice 2019-03-15 10:08:52 +01:00
Olaoluwa Osuntokun
33ad645f8c
lnwallet: update TestChanSyncFailure to pass with new borked update restriction
In this commit, we update the `TestChanSyncFailure` method to pass given
the new behavior around updating borked channel states. In order to do
this, we add a new method to allow the test to clear an existing channel
state. This method may be of independent use in other areas in the
codebase in the future as well.
2019-03-08 19:15:10 -08:00
Olaoluwa Osuntokun
032eacb796
channeldb: prevent mutating on-disk commitment state if channel is borked 2019-03-08 19:15:04 -08:00
Olaoluwa Osuntokun
cbe0bf6a22
Merge pull request #2501 from cfromknecht/batch-preimage-writes
htlcswitch: batch preimage writes/consistency fix
2019-02-21 17:00:00 -08:00
Conner Fromknecht
0a3e1cfbe5
channeldb+witness_beacon: use sha256 lookup+delete witness 2019-02-19 17:06:42 -08:00
Conner Fromknecht
e8b7f1fca3
channeldb/witness_cache: create AddSha256Witnesses helper + test 2019-02-19 17:05:30 -08:00
Conner Fromknecht
56b6becc48
channeldb/witness_cache_test: test batch preimage insertion 2019-02-19 17:05:17 -08:00
Conner Fromknecht
30f61b7630
multi: make AddPreimage variadic, optimistically compute key
In this commit, we modify the WitnessCache's
AddPreimage method to accept a variadic number
of preimages. This enables callers to batch
preimage writes in performance critical areas
of the codebase, e.g. the htlcswitch.

Additionally, we lift the computation of the
witnesses' keys outside of the db transaction.
This saves us from having to do hashing inside
and blocking other callers, and limits extraneous
blocking at the call site.
2019-02-19 17:05:04 -08:00
Wilmer Paulino
9febc9cc04
channeldb: add gossiper message store key migration
In this commit, we introduce a migration for the message store
sub-bucket that will migrate all keys within it to a new key format.
This new key format is composed of the peer's public key, followed by
the short channel ID, followed by the message type. This migration is
needed in order to provide backwards-compatibility with messages that
were previously stored before the introduction of the new key format.
2019-02-14 18:29:39 -08:00
Johan T. Halseth
2b012b06a0
channeldb/graph test: add test for update policy for unknown edge 2019-02-14 14:21:18 +01:00
Joost Jager
1b87fbfab2
invoices+htlcswitch+lnrpc: cancel invoice 2019-02-06 07:29:18 +01:00
Joost Jager
436dd41c77
channeldb: move idempotency up the call tree
As a preparation for subscribing to single invoices, InvoiceRegistry
needs to become aware of settling a settled invoice.
2019-02-01 09:42:56 +01:00
Joost Jager
bacd92418a
invoices: use lntypes.Hash and lntypes.Preimage
Previously chainhash.Hash was used, which converts to/from string in
reversed format. Payment hashes and preimages are supposed to be
non-reversed.
2019-02-01 09:42:29 +01:00
Olaoluwa Osuntokun
9a66ac396b
Merge pull request #2493 from halseth/tor-addr-sanity-check
Validate addresses before writing them to DB
2019-01-31 17:15:22 -08:00
Olaoluwa Osuntokun
480ec3bbca
channeldb: ensure restored channels can't be mutated 2019-01-28 20:24:48 -08:00
Olaoluwa Osuntokun
b2b57314fa
channeldb: add new RestoreChannelShells method
In this commit, we add a new type (ChannelShell) along with a new
method, RestoreChannelShells which allows a caller to insert a series of
channel shells into the database. These channel shells will allow a
restored node to initiate the DLP protocol and recover their set of
existing channels.

When we insert a channel shell, we re-create the original link node, and
also add the outgoing edge to the channel graph. This way we can be sure
that upon start up, we attempt to connect to the remote peers, and that
the normal graph query commands will operate as expected.
2019-01-28 20:24:41 -08:00
Olaoluwa Osuntokun
fa30af0475
channeldb: when fetching/inserting commits check for ChanStatusRestored
If the ChanStatusRestored flag is set, then we don't need to write or
read the set of commits for a channel as they won't exist. This will be
the case when we restore a channel from an SCB.
2019-01-28 20:24:40 -08:00
Olaoluwa Osuntokun
c656788b0b
channeldb: add new methods to allow adding a new edge+policy w/ existing db transaction 2019-01-28 20:24:38 -08:00
Olaoluwa Osuntokun
11c6887ffa
channeldb: refactor syncPending to expose new syncNewChannel function
The new syncNewChannel function will allow callers to insert a new
channel given the OpenChannel struct, and set of addresses for the
channel peer. This new method will also create a new LinkNode for the
peer if one doesn't already exist.
2019-01-28 20:24:37 -08:00
Olaoluwa Osuntokun
7a88f580ea
channeldb: add new FetchChannelMethod
In this commit, we add a new method, FetchChannel which is required in
order to implement the new chanbackup.LiveChannelSource interface.
2019-01-28 20:24:36 -08:00
Olaoluwa Osuntokun
aaf6456e12
channeldb: add HasChanStatus and ApplyChanState methods to OpenChannel
These methods allow callers to properly query if a channel is in a
particular flag, and also modify the channel status in a thread safe
manner.
2019-01-28 20:24:35 -08:00
Olaoluwa Osuntokun
a410262dda
channeldb: modify the String() method of ChannelStatus reflect all flags
In this commit, we modify the String() method of the ChannelStatus type
to reflect the fact that it's a flag set. With these new changes, we'll
now print the variable name of each assigned bit with a bar delimiting
them all.
2019-01-28 20:24:33 -08:00
Olaoluwa Osuntokun
f57f40e767
channeldb: add prefix naming to ChannelStatus enum variables
In this commit, we add a prefix naming scheme to the ChannelStatus enum
variables. We do this as it enables outside callers to more easily
identify each individual enum variable as a part of the greater
enum-like type.
2019-01-28 20:24:32 -08:00
Olaoluwa Osuntokun
f4054d2a66 channeldb: add new AddrsForNode method
In this commit, we add a new AddrsForNode method. This method will allow
a wrapper sturct to implement the new chanbackup.LiveChannelSource
method which is required to implement the full SCB feature set.
2019-01-28 20:24:22 -08:00
Johan T. Halseth
7d34ce9d08
lnwire+multi: define HasMaxHtlc helper on msgFlags 2019-01-22 08:42:30 +01:00
Johan T. Halseth
01e679786d
channeldb/graph_test: add TestEdgePolicyMissingMaxHtcl 2019-01-22 08:42:28 +01:00
Johan T. Halseth
6fcc76fc68
channeldb/channel: ignore ChannelEdgePolicy with missing max_htlc
If the max_htlc field is not found when fetching a ChannelEdgePolicy
from the DB, we treat this as an unknown policy.

This is done to ensure we won't propagate invalid data further. The data
will be overwritten with a valid one when we receive an update for this
channel.

It shouldn't be very common, but old data could be lingering in the DB
added before this field was validated.
2019-01-22 08:42:28 +01:00
Johan T. Halseth
8dd074ee57
channeldb/graph_test: assert MaxHTLC field correctness
Co-authored-by: Valentine Wallace <valentine.m.wallace@gmail.com>
2019-01-22 08:42:27 +01:00
Valentine Wallace
69d4bf051f
channeldb/graph: add max HTLC to ChannelEdgePolicy
Adding this field will allow us to persist an edge's
max HTLC to disk, thus preserving it between restarts.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:27 +01:00
Johan T. Halseth
b9c5248915
channeldb/graph: extract ChannelEdgePolicy serialization 2019-01-22 08:42:27 +01:00
Johan T. Halseth
91c9e45031
channeldb/graph: check correct bucket for nilness 2019-01-22 08:42:27 +01:00
Valentine Wallace
0fd6004958
multi: partition lnwire.ChanUpdateFlag into ChannelFlags and MessageFlags
In this commit:

* we partition lnwire.ChanUpdateFlag into two (ChanUpdateChanFlags and
ChanUpdateMsgFlags), from a uint16 to a pair of uint8's

* we rename the ChannelUpdate.Flags to ChannelFlags and add an
additional MessageFlags field, which will be used to indicate the
presence of the optional field HtlcMaximumMsat within the ChannelUpdate.

* we partition ChannelEdgePolicy.Flags into message and channel flags.
This change corresponds to the partitioning of the ChannelUpdate's Flags
field into MessageFlags and ChannelFlags.

Co-authored-by: Johan T. Halseth <johanth@gmail.com>
2019-01-22 08:42:26 +01:00
Johan T. Halseth
8af2473644
channeldb/addr_test: add tests for invalid onion addresses 2019-01-17 14:16:15 +01:00
Johan T. Halseth
6a3e1423d2
channeldb/addr: sanity check onion address length before writing to db 2019-01-17 14:16:15 +01:00
Johan T. Halseth
e2e00f9dd2
channeldb/addr_test: add invalid TCP cases
These are invalid length IP addresses that earlier would not fail to
serialize.
2019-01-17 14:16:15 +01:00