Prior to AMP, there could only be one HTLC set. Now that there can be
multiple, we introduce the inherent risk that we might be persisting a
settle/accept of the wrong HTLC set. To migitigate this risk, we add a
check to assert that an invoice can only be transitioned into an
Accepted or Settled state if the specified HTLC set is non-empty, i.e.
has accepted HTLCs.
To do this check, we unroll the loops in UpdateInvoice to first process
any added or canceled HTLCs. This ensures that the set of accepted HTLCs
is up-to-date when we got to transition the invoice into a new state, at
which point we can accurately check that the HTLC set is non-empty.
Previously this sort of check wasn't possible because a newly added HTLC
wouldn't be populated on the invoice when going to call
updateInvoiceState.
Once the invoice-level transition checks have completed, we complete a
final pass to move the HTLCs into their final states and recompute the
amount paid.
With this refactor, it is now possible to make stronger assertions
about the invoice and the state transitions being made by the
invoiceregistry before those changes are ultimately persisted.
This commit extends channeldb with the DeleteInvoices call which is when
passed a slice of delete references will attempt to delete the invoices
pointed to by the references and also clean up all our invoice indexes.
This commit adds channeldb.ScanInvoices to scan through all invoices in
the database. The new call will also replace the already existing
channeldb.FetchAllInvoicesWithPaymentHash call in preparation to collect
invoices we'd like to delete and watch for expiry in one scan in later
commits.
This commit moves makeTestDB to db.go and exports it so that we'll be
able to use this function in other unit tests to make them testable with
etcd if needed.
Avoids indexing the all-zeros pay addr, since it is still in use by
legacy keysend. Without this, the pay addr index will reject all but the
first keysend since they will be detected as duplicates within the set
id index.
We now use the same method of pagination for invoices and payments.
Rather than duplicate logic across calls, we add a pagnator struct
which can have query specific logic plugged into it. This commit also
addresses an existing issue where a reverse query for invoices with an
offset larger than our last offset would not return any invoices. We
update this behaviour to act more like c.Seek and just start from the
last entry. This behaviour change is covered by a unit test that
previously checked for the lack of invoices.
In our current invoice pagination logic, we would not return any
invoices if our offset index was more than 1 off our last index and we
were paginating backwards. This commit adds a test case for this
behaviour before fixing it in the next commit.
Previously it wasn't possible to store a preimage in the invoice
database and signal that a payment should not be settled right away. The
only way to hold a payment was to insert the magic UnknownPreimage value
in the invoice database. This commit introduces a distinct flag to
signal that an invoice is a hold invoice and thereby allows the preimage
to be present in the database already.
Preparation for (key send) hodl invoices for which we already know the
preimage.
This commit removes channeldb.FetchAllInvoices and changes tests such
that expectation sets are prepared in the test case instead of selected
from the DB.
This commits builds on top of PR #3694 to further clarify invoice
state by defining pending invoices as the ones which are not
settled or canceled. Automatic cancellation of expired invoices
makes this possbile. While this change only directly affects
ChannelDB, users of the listinvoices RPC will receive actual
pending invoices when pending_only flag is set.
This commit changes how FetchAllInvoicesWithPaymentHash behaves
when the DB is empty and also adds a unit test to test that
case as well as normal expected behavior.
Previously the cancel and add actions were combined in a single map.
Nil values implictly signaled cancel actions. This wasn't very obvious.
Furthermore this split prepares for processing the adds and cancels
separately, which is more efficient if there are already two maps.
This commit restructures an invoice's ContractTerms to better encompass
the restrictions placed on settling. For instance, the final ctlv delta
and invoice expiry are moved from the main invoice body (where
additional metadata is stored). Additionally, it moves the State field
outside of the terms since it is rather metadata about the invoice
instead of any terms offered to the sender in the payment request.
Previously the invoice registry wasn't aware of replayed htlcs. This was
dealt with by keeping the invoice accept/settle logic idempotent, so
that a replay wouldn't have an effect.
This mechanism has two limitations:
1. No accurate tracking of the total amount paid to an invoice. The total
amount couldn't just be increased with every htlc received, because it
could be a replay which would lead to counting the htlc amount multiple
times. Therefore the total amount was set to the amount of the first
htlc that was received, even though there may have been multiple htlcs
paying to the invoice.
2. Impossible to check htlc expiry consistently for hodl invoices. When
an htlc is new, its expiry needs to be checked against the invoice cltv
delta. But for a replay, that check must be skipped. The htlc was
accepted in time, the invoice was moved to the accepted state and a
replay some blocks later shouldn't lead to that htlc being cancelled.
Because the invoice registry couldn't recognize replays, it stopped
checking htlc expiry heights when the invoice reached the accepted
state. This prevents hold htlcs from being cancelled after a restart.
But unfortunately this also caused additional htlcs to be accepted on an
already accepted invoice without their expiry being checked.
In this commit, the invoice registry starts to persistently track htlcs
so that replays can be recognized. For replays, an htlc resolution
action is returned early. This fixes both limitations mentioned above.
As the logic around invoice mutations gets more complex, the friction
caused by having this logic split between invoice registry and channeldb
becomes more apparent. This commit brings a clearer separation of
concerns by centralizing the accept/settle logic in the invoice
registry.
The original AcceptOrSettle method is renamed to UpdateInvoice because
the update to perform is controlled by the callback.
This commit adds a set of htlcs to the Invoice struct and
serializes/deserializes this set to/from disk. It is a preparation for
accurate invoice accounting across restarts of lnd.
A migration is added for the invoice htlcs.
In addition to these changes, separate final cltv delta and expiry
invoice fields are created and populated. Previously it was required
to decode this from the stored payment request. The reason to create
a combined commit is to prevent multiple migrations.
This commit is the final step in making the link unaware of invoices. It
now purely offers the htlc to the invoice registry and follows
instructions from the invoice registry about how and when to respond to
the htlc.
The change also fixes a bug where upon restart, hodl htlcs were
subjected to the invoice minimum cltv delta requirement again. If the
block height has increased in the mean while, the htlc would be canceled
back.
Furthermore the invoice registry interaction is aligned between link and
contract resolvers.
This commit modifies the invoice registry to handle invoices for which
the preimage is not known yet (hodl invoices). In that case, the
resolution channel passed in from links and resolvers is stored until we
either learn the preimage or want to cancel the htlc.
This commit is a preparation for the addition of new invoice
states. A database migration is not needed because we keep
the same field length and values.
Previously a call to QueryInvoices with reversed=true and index_offset=1
would make the cursor point to the first available invoice (num 1) that
would be returned as part of the response. This is inconsistent with the
othre indexes, so we instead just return an empty list in this case.
A test case for this situation is also added.
In this commit, we introduce support for querying the database for invoices
that occurred within a specific add index range. The query format includes an
index to start with and a limit on the number of returned results.
Co-authored-by: Valentine Wallace <valentine.m.wallace@gmail.com>
In this commit, we add a new test to ensure that duplicate invoice
settles work as expected. At the present time, this test will fail as
the second to last assertion fails as we'll return a nil invoice the
second time around.
