This commits builds on top of PR #3694 to further clarify invoice
state by defining pending invoices as the ones which are not
settled or canceled. Automatic cancellation of expired invoices
makes this possbile. While this change only directly affects
ChannelDB, users of the listinvoices RPC will receive actual
pending invoices when pending_only flag is set.
This commit adds InvoiceExpryWatcher which is a separate class that
receives new invoices (and existing ones upon restart) from InvoiceRegistry
and actively watches their expiry. When an invoice is expired
InvoiceExpiryWatcher will call into InvoiceRegistry to cancel the
invoice and by that notify all subscribers about the state change.
Previously the cancel and add actions were combined in a single map.
Nil values implictly signaled cancel actions. This wasn't very obvious.
Furthermore this split prepares for processing the adds and cancels
separately, which is more efficient if there are already two maps.
This commit moves the update code into its own function as a preparation
for extending the logic further for mpp.
In order to make this change cleanly, structured result codes are
introduced. This also prepares for a future htlc notifier rpc hook that
reports htlc settle decisions to external applications.
Furthermore the awkward use of errNoUpdate as a way to signal no update
is removed.
This commit restructures an invoice's ContractTerms to better encompass
the restrictions placed on settling. For instance, the final ctlv delta
and invoice expiry are moved from the main invoice body (where
additional metadata is stored). Additionally, it moves the State field
outside of the terms since it is rather metadata about the invoice
instead of any terms offered to the sender in the payment request.
Previously the invoice registry wasn't aware of replayed htlcs. This was
dealt with by keeping the invoice accept/settle logic idempotent, so
that a replay wouldn't have an effect.
This mechanism has two limitations:
1. No accurate tracking of the total amount paid to an invoice. The total
amount couldn't just be increased with every htlc received, because it
could be a replay which would lead to counting the htlc amount multiple
times. Therefore the total amount was set to the amount of the first
htlc that was received, even though there may have been multiple htlcs
paying to the invoice.
2. Impossible to check htlc expiry consistently for hodl invoices. When
an htlc is new, its expiry needs to be checked against the invoice cltv
delta. But for a replay, that check must be skipped. The htlc was
accepted in time, the invoice was moved to the accepted state and a
replay some blocks later shouldn't lead to that htlc being cancelled.
Because the invoice registry couldn't recognize replays, it stopped
checking htlc expiry heights when the invoice reached the accepted
state. This prevents hold htlcs from being cancelled after a restart.
But unfortunately this also caused additional htlcs to be accepted on an
already accepted invoice without their expiry being checked.
In this commit, the invoice registry starts to persistently track htlcs
so that replays can be recognized. For replays, an htlc resolution
action is returned early. This fixes both limitations mentioned above.
As the logic around invoice mutations gets more complex, the friction
caused by having this logic split between invoice registry and channeldb
becomes more apparent. This commit brings a clearer separation of
concerns by centralizing the accept/settle logic in the invoice
registry.
The original AcceptOrSettle method is renamed to UpdateInvoice because
the update to perform is controlled by the callback.
This commit adds a set of htlcs to the Invoice struct and
serializes/deserializes this set to/from disk. It is a preparation for
accurate invoice accounting across restarts of lnd.
A migration is added for the invoice htlcs.
In addition to these changes, separate final cltv delta and expiry
invoice fields are created and populated. Previously it was required
to decode this from the stored payment request. The reason to create
a combined commit is to prevent multiple migrations.
This commit is the final step in making the link unaware of invoices. It
now purely offers the htlc to the invoice registry and follows
instructions from the invoice registry about how and when to respond to
the htlc.
The change also fixes a bug where upon restart, hodl htlcs were
subjected to the invoice minimum cltv delta requirement again. If the
block height has increased in the mean while, the htlc would be canceled
back.
Furthermore the invoice registry interaction is aligned between link and
contract resolvers.
This commit modifies the invoice registry to handle invoices for which
the preimage is not known yet (hodl invoices). In that case, the
resolution channel passed in from links and resolvers is stored until we
either learn the preimage or want to cancel the htlc.
This commit is a preparation for the addition of new invoice
states. A database migration is not needed because we keep
the same field length and values.
Previously a call to QueryInvoices with reversed=true and index_offset=1
would make the cursor point to the first available invoice (num 1) that
would be returned as part of the response. This is inconsistent with the
othre indexes, so we instead just return an empty list in this case.
A test case for this situation is also added.
In this commit, we introduce support for querying the database for invoices
that occurred within a specific add index range. The query format includes an
index to start with and a limit on the number of returned results.
Co-authored-by: Valentine Wallace <valentine.m.wallace@gmail.com>
In this commit, we fix an existing bug related to duplicate invoice
settle.s Before this commit, the second (and later) times an invoice was
settled we would return a nil pointer. This would result in the new
invoiceRegistry panicing as it would go to attempt to notify with a nil
invoice.
We fix this by returning the invoice on disk (unmodified) for each
settle after the initial one.
Fixes#1568.
In this commit, we add two new methods: InvoicesAddedSince and
InvoicesSettledSince. These methods will be used by higher level
sub-systems that implement notifications to deliver any notifications
backlog based on the last add index, and last settle index that the
client knows of.
It's important to note that care has been taken to ensure that this new
API can be used in a backwards compatible manner. If a client specifies
and index of 0 for either of the methods, then no backlog will be sent.
This is due to the fact that current users of the API don't expect any
backlog notifications to be sent. Additionally, the index actually
starts at 1, instead of 0.
In this commit, we add two new indexes to the invoice database: the add
index, and the settle index. These to indexes essentially form a time
series index on top of the existing primary index bucket. Each time an
invoice is added, we'll advance the addIndex seqno, and then create a
mapping from seqNo -> invoiceNum. Each time an invoice is settled, we'll
do the same, but within the settle index.
This change is required in order to allow callers to effectively seek
into the current invoice database in order to obtain notifications for
any invoices they may have missed out on while they were disconnected.
This will allow us to implement robust streaming invoice notifications
within lnd to ensure that clients never miss an event.
In this commit, in order to allow the caller to specify the amount that
was ultimately accepted for an invoice, the SettleInvoice method has
gained a new parameter: amtPaid. SettleInvoice will now populate the
final amount paid in the database upon db commit.
In this commit, we move to explicitly storing a bit more information
within the invoice. Currently this information is already stored in the
payment request, but by storing it at this level, callers that may not
be in the state to fully decode a payment request can obtain this data.
We avoid a database migration by appending this data to the end of an
invoice. When decoding, we'll try to read out this extra information,
and simply return what we have if it isn't found.
This commit removes all instances of the fastsha256 library and
replaces it with the sha256 library in the standard library. This
change should see a number of performance improvements as the standard
library has highly optimized assembly instructions with use vectorized
instructions as the platform supports.
Go-fmt files. Refactored code according to the guidelines.
Enhanced payment test: add error checking
and individual context for each API call.
Add Timestamp field to payment struct.
This commit modifies the composition of the boltdb pointer within the
DB struct to use embedding.
The rationale for this change is that the daemon may soon store some
semi-transient items within the database which requires us to expose
the boltdb’s transaction API. The logic for serialization of this data
will likely lie outside of the channeldb package as the items that may
be stored in the future will be specific to the current sub-systems
within the daemon and not generic channel related data.
This commit modifies the on-disk storage of invoices to stop the
optional fields (memo+receipt) on-disk as variable length byte arrays.
This change saves space as the optional fields now only take up as much
space as is strictly needed, rather than always being padded out to max
size (1KB).
This commit adds a new invoice related method: FetchAllInvoices. This
method allows callers to query the state of all invoices currently
stored within the database. The method takes a toggle bit which
determines if only pending (unsettled) invoices should be returned, or
if they al should be.
This commit moves the location of the invoice counter key which is an
auto-incrementing primary key for all invoices. Rather than storing the counter
in the same top-level invoice bucket, the counter is now stored within the
invoiceIndex bucket. With this change, the top-level bucket can now cleanly be
scanned in a sequential manner to retrieve all invoices.
This commit adds the necessary database functionality required for a
high-level payment invoice workflow. Invoices can be added dealing the
requirements for fulfillment, looked by payment hash, and the finally
also settled by payment hash. For record keeping and the possibility of
reconciling future disputes, invoices are currently never deleted from
disk. Instead when an invoice is settled a bit is toggled indicating as
much.
The current invoiceManger within the daemon will be modified to use
this persistent invoice store, only storing certain “debug” invoices in
memory as dictated by a command line flag.
