This commit removes channeldb.FetchAllInvoices and changes tests such
that expectation sets are prepared in the test case instead of selected
from the DB.
This commit adds handling code for the key send custom record. If this
record is present and its hash matches the payment hash, invoice
registry will insert a new invoice into the database "just in time". The
subsequent settle flow is unchanged. The newly inserted invoice is
picked up and settled. Notifications will be broadcast as usual.
This field isn't optional. It was introduced in
5ed31b1030176374d4184fee0b80ae7679e0e355 which was part of release
0.4.0. This release contained breaking database changes, so it is safe
to assume that from there on the field is always populated.
If there would still be empty payment request fields in the wild, users
would also experience issues with ListInvoices. ListInvoices decodes the
payment request.
This commits builds on top of PR #3694 to further clarify invoice
state by defining pending invoices as the ones which are not
settled or canceled. Automatic cancellation of expired invoices
makes this possbile. While this change only directly affects
ChannelDB, users of the listinvoices RPC will receive actual
pending invoices when pending_only flag is set.
Previously we would return nil features when we didn't have a node
announcement or a given node. With this change, we can always assume the
feature vector is populated during pathfinding.
This commit changes how FetchAllInvoicesWithPaymentHash behaves
when the DB is empty and also adds a unit test to test that
case as well as normal expected behavior.
This commit adds InvoiceExpryWatcher which is a separate class that
receives new invoices (and existing ones upon restart) from InvoiceRegistry
and actively watches their expiry. When an invoice is expired
InvoiceExpiryWatcher will call into InvoiceRegistry to cancel the
invoice and by that notify all subscribers about the state change.
This commit prepares for more manipulation of custom records. A list of
tlv.Record types is more difficult to use than the more basic
map[uint64][]byte.
Furthermore fields and variables are renamed to make them more
consistent.
Before we change the Hop struct, isolate the code that is used in older
migrations to prevent breaking them.
route.go was taken from commit 6e463c1634061d595953f20813860207e5d485ce
Previously the cancel and add actions were combined in a single map.
Nil values implictly signaled cancel actions. This wasn't very obvious.
Furthermore this split prepares for processing the adds and cancels
separately, which is more efficient if there are already two maps.
This commit moves the update code into its own function as a preparation
for extending the logic further for mpp.
In order to make this change cleanly, structured result codes are
introduced. This also prepares for a future htlc notifier rpc hook that
reports htlc settle decisions to external applications.
Furthermore the awkward use of errNoUpdate as a way to signal no update
is removed.
This commit adds fields for upfront shutdown scripts set
by the local and remote peer to the OpenChannel struct.
These values are optional, so they are added with their
own keys in the chanBucket in the DB.
In this commit, we add a new bit to the existing ChannelType bitfield.
If this bit is set, then it signals that we have the funding transaction
stored on disk. A future change will enable lnd to have the funding
transaction be constructed externally, allowing for things like funding
from a hardware wallet, or a channel created as a sub-branch within an
existing channel factory.
Refresh channel memory state whenever the short channel id is refreshed.
This is to make the in-memory channel consistent with the disk data.
Fixes#3765.
This commit restructures an invoice's ContractTerms to better encompass
the restrictions placed on settling. For instance, the final ctlv delta
and invoice expiry are moved from the main invoice body (where
additional metadata is stored). Additionally, it moves the State field
outside of the terms since it is rather metadata about the invoice
instead of any terms offered to the sender in the payment request.
This commit modifies the FetchPayment method to return MPPayment structs
converted from the legacy on-disk format. This allows us to attach the
HTLCs to the events given to clients subscribing to the outcome of an
HTLC.
This commit also bubbles up to the routerrpc/router_server, by
populating HTLCAttempts in the response and extracting the legacy route
field from the HTLCAttempts.
In this commit, we convert the existing `channeldb.ChannelType` type
into a _bit field_. This doesn't require us to change the current
serialization or interpretation or the type as it is, since all the
current defined values us a distinct bit. This PR lays the ground work
for any future changes that may introduce new channel types (like anchor
outputs), and also any changes that may modify the existing invariants
around channels (if we're the initiator, we always have the funding
transaction).
This commit removes the migrations from channeldb and references those
in the migrations_01_to_11 package. This creates a one-way dependency on
the migrations. Future changes to channeldb won't be able to break
migrations anymore.
This commit is a direct copy of the complete channeldb package. It only
changes the package declaration at the top of every file. We make this
full copy so that review can be focused on the actual changes made.
Otherwise changes may drown in all the file moves.
Linting for the new package is disabled, as it contains lots of
pre-existing issues.
In this commit, we update the channel state machine to be aware of
tweakless commits. In several areas, we'll now check the channel's type
to see if it's `SingleFunderTweakless`. If so, then we'll opt to use the
remote party's non-delay based point directly in the script, skipping
any additional cryptographic operations. Along the way we move the
`validateCommitmentSanity` method to be defined _before_ it's used as is
cutomary within the codebase.
Notably, within the `NewUnilateralCloseSummary` method, we'll now _blank
out_ the `SingleTweak` value if the commitment is tweakless. This
indicates to callers the witness type they should map to, as the value
isn't needed at all any longer when sweeping a non-delay output.
We also update the signing+verification tests to also test that we're
able to properly generate a valid witness for the new tweakless
commitment format.
In this commit, we define a new channel type: SingleFunderTweakless.
We'll use this channel type to denote channels with commitments that
don't tweak the remote party's key in their non-delay output.
channeldb: Fix dropped error and wrap with context
channeldb: Fix empty error condition in waitingproof test
channeldb: Fix empty error condition in codec
channeldb: Wrap error in context
In this commit, we ensure that the test for migration 9 uses the same
encoding/decoding functions as was present in the repo when the
migration was first added. Otherwise, the test will fail as it'll try to
use the decoding functions of master (migration 10 and onwards) rather
than the decoding function of migration 9.
In this commit, we fix an issue that was recently introduced as a result
of migration #10. The new TLV format ended up modifying the
serialization functions called in `serializePaymentAttemptInfo`.
Migration #9, also used this `serializePaymentAttemptInfo` method to
serialize the _new_ (pre TLV, but new payment attempt structure) routes
into the database during its migration. However, migration #10 failed to
copy over the existing unmodified `serializePaymentAttemptInfo` method
into the legacy serialization for migration #9. As a result, once
migration #9 was run, the routes/payments were serialized using the
_new_ format, rather than the format used for v0.7.1. This then lead to
de-serialization either failing, or causing partial payment corruption
as migration #10 was expecting the "legacy" format (no TLV info).
We fix this issue by adding a new fully enclosed
`serializePaymentAttemptInfoMigration9`method that will be used for
migration #9. Note that our tests didn't catch this, as they test the
migration in isolation, rather than in series which is how users will
encounter the migrations.
Fixes#3463.
Previously the invoice registry wasn't aware of replayed htlcs. This was
dealt with by keeping the invoice accept/settle logic idempotent, so
that a replay wouldn't have an effect.
This mechanism has two limitations:
1. No accurate tracking of the total amount paid to an invoice. The total
amount couldn't just be increased with every htlc received, because it
could be a replay which would lead to counting the htlc amount multiple
times. Therefore the total amount was set to the amount of the first
htlc that was received, even though there may have been multiple htlcs
paying to the invoice.
2. Impossible to check htlc expiry consistently for hodl invoices. When
an htlc is new, its expiry needs to be checked against the invoice cltv
delta. But for a replay, that check must be skipped. The htlc was
accepted in time, the invoice was moved to the accepted state and a
replay some blocks later shouldn't lead to that htlc being cancelled.
Because the invoice registry couldn't recognize replays, it stopped
checking htlc expiry heights when the invoice reached the accepted
state. This prevents hold htlcs from being cancelled after a restart.
But unfortunately this also caused additional htlcs to be accepted on an
already accepted invoice without their expiry being checked.
In this commit, the invoice registry starts to persistently track htlcs
so that replays can be recognized. For replays, an htlc resolution
action is returned early. This fixes both limitations mentioned above.
As the logic around invoice mutations gets more complex, the friction
caused by having this logic split between invoice registry and channeldb
becomes more apparent. This commit brings a clearer separation of
concerns by centralizing the accept/settle logic in the invoice
registry.
The original AcceptOrSettle method is renamed to UpdateInvoice because
the update to perform is controlled by the callback.
This commit adds a set of htlcs to the Invoice struct and
serializes/deserializes this set to/from disk. It is a preparation for
accurate invoice accounting across restarts of lnd.
A migration is added for the invoice htlcs.
In addition to these changes, separate final cltv delta and expiry
invoice fields are created and populated. Previously it was required
to decode this from the stored payment request. The reason to create
a combined commit is to prevent multiple migrations.
This commit adds an index bucket, disabledEdgePolicyBucket, for those
ChannelEdgePolicy with disabled bit on.
The main purpose is to be able to iterate over these fast when prune is
needed without the need for iterating the whole graph.
The entry points for accessing this index are:
1. When updating ChannelEdgePolicy - insert an entry.
2. When deleting ChannelEdge - delete the associated entries.
3. When querying for disabled channels - implemented DisabledChannelIDs
function
This commit modifies the nodeWithDist struct to use a route.Vertex
instead of a *channeldb.LightningNode. This change, coupled with
the new ForEachNodeChannel function, allows the findPath Djikstra's
algorithm to cut down on database lookups since we no longer need
to call the FetchOtherNode function.
This commit specifies two bbolt options when opening the underlying
channel and watchtower databases so that there is reduced heap
pressure in case the bbolt database has a lot of free pages in the
B+ tree.
Previously the migration would fail if the source node was not set in
the database. Since we know that the source node must have been set
before making any payments, we check whether there actually are any
payments to migrate, and return early if not.
This commit makes the router use the ControlTower to drive the payment
life cycle state machine, to keep track of active payments across
restarts. This lets the router resume payments on startup, such that
their final results can be handled and stored when ready.
This commit gives a new responsibility to the control tower, letting it
populate the payment bucket structure as the payment goes through
its different stages.
The payment will transition states Grounded->InFlight->Success/Failed,
where the CreationInfo/AttemptInfo/Preimage must be set accordingly.
This will be the main driver for the router state machine.
migrateOutgoingPayments moves the OutgoingPayments into a new bucket format
where they all reside in a top-level bucket indexed by the payment hash. In
this sub-bucket we store information relevant to this payment, such as the
payment status.
To avoid that the router resend payments that have the status InFlight (we
cannot resume these payments for pre-migration payments) we delete those
statuses, so only Completed payments remain in the new bucket structure.
This commit changes the format used to store payments within the
DB. Previously this was serialized as one continuous struct
OutgoingPayment, which also contained an Invoice struct we where only
using a few fields of. We now split it up into two simpler sub-structs
CreationInfo, AttemptInfo and PaymentPreimage.
We also want to associate the payments more closely with payment
statuses, so we move to this hierarchy:
There's one top-level bucket "sentPaymentsBucket" which contains a set
of sub-buckets indexed by a payment's payment hash. Each such sub-bucket
contains several fields:
paymentStatusKey -> the payment's status
paymentCreationInfoKey -> the payment's CreationInfo.
paymentAttemptInfoKey -> the payment's AttemptInfo.
paymentSettleInfoKey -> the payment's preimage (or zeroes for
non-settled payments)
The CreationInfo is information that is static during the whole payment
lifcycle. The attempt info is set each time a new payment attempt
(route+paymentID) is sent on the network. The preimage is information
only known when a payment succeeds. It therefore makes sense to split
them.
We keep legacy serialization code for migration puproses.
This commit adds persisted status bit-field to ClientSessions, that can
be used to modify behavior of their handling in the client. Currently,
only a default CSessionActive status is defined. However, the intention
is that this could later be used to signal that a session is abandoned
without needing to perform a db migration to add the field. As we move
forward with testing, this will likely be useful if a session gets
borked and we need a simple method of the client to temporarily ignore
certain sessions.
The field may be useful in signaling other types of status changes,
though this was the primary motivation that warranted the addition.
This commit is the final step in making the link unaware of invoices. It
now purely offers the htlc to the invoice registry and follows
instructions from the invoice registry about how and when to respond to
the htlc.
The change also fixes a bug where upon restart, hodl htlcs were
subjected to the invoice minimum cltv delta requirement again. If the
block height has increased in the mean while, the htlc would be canceled
back.
Furthermore the invoice registry interaction is aligned between link and
contract resolvers.
