This commit adds a PendingCommitTicker to the link config, which allows
us to control how quickly we fail the link if the commitment dance
stalls. Now that the mailbox has the ability to cancel packets, when the
link fails it will reset the mailbox packets on exit, forcing a
reevaluation of the HTLCs against their mailbox expiries.
Now that packet failure is handled by the mailbox, we can now enforce
a delivery deadline and fail the packet if it the deadilne is exceeded.
This gives senders quicker feedback about tried routes, and allows them
to try alternative paths to the destination in the meantime.
This commit splits the packet courier internally into two distinct
queues, one for adds and one for settles+fails. This allows us to
prioritize HTLCs that will clear the commitment transaction and make
space for adds. Previously this responsibility was handled by the
overflow queue.
This commit delays the advancement of the pktHead until after the
message has been delivered. This is a prepatory step, as in the future
we may fail to deliver the packet due to a deadline expiring.
This commit moves the current logic for sending failures out of the link
and into the mailbox in preparation for our failing delayed htlcs. We do
so because the mailbox may need to fail packets while the link is
offline, and needs to be able to complete the task without member
methods on the link.
This commit removes the overflowQueue from the link. We do so in order
to promote better UX for senders, so that HTLCs are failed faster when
the commitment is full. This gives the sender the opportunity to try
another, more open path, rather than perceive the HTLC as being stuck.
At the same time, we remove the total number of active goroutines in lnd
by a factor of N where N is the number of active channels.
This commit is preparation for the test added in the subsequent commit.
We modify makeHoldPayment to return any failures direectly when trying
to add an HTLC to the switch. This lets us know that the HTLC was indeed
sent without failure when the method returns.
A following commits will move/modify callsites of AckPacket, Start, and
Stop, none of which use the return value and ultimately cause the linter
to complain. However, none of these in-memory operations can fail so we
just remove the returned errors altogether.
The linter complains about not checking the return value from
WipeChannel in certain places. Instead of checking we simply remove the
returned error because the in-memory modifications cannot fail.
This commit changes the switch to only log an error if update_fail_htlc
comes in and closeCircuit returns ErrUnknownCircuit. Rationale
being that only settles should hit this code path, anything else
is a result of a link flap and should be treated as an error.
This commit modifies updateCommitTx to error with ErrLinkShuttingDown
when we try to send a ContractUpdate on the htlcUpdates chan and the
link has closed the quit chan. It also changes the order of the call
to ackDownStreamPackets and contract update call for consistency since
the packets should be acknowledged before the link goes down.
This commit changes the fallback in NextLocalHtlcIndex to
RemoteCommitment since the LocalHtlcIndex field lags behind
on the LocalCommitment. Without this bug fix, open circuits
would get prematurely trimmed, resulting in more erroneous
logs. A test case is included to check that the fix works.
This commit adds link failure notifications for failures which occur
on our incoming link. These failures may be receives which we failed or
forwards which we could not parse.
Add notifications for local initiated sends settles and forwarding
failures. As with link failures, local send settles and forwarding
failures are reported directly to the router so must have their own
notification handling.
Notify link failures for our own payments. Separate handling code is
required for local payment link failures because we do not pass these
failures back through the switch (like we do for link failures for
forwards), but rather send them straight back to the router. Our own
sends have the payment ID saved in the incoming htlc ID of the packet's
incoming circuit. This change replaces that value with for the sake
of consistent notifying of sends and receives from our node.
This commit adds notifications for htlcs which are forwarded through
our node. Forwards are notified when the htlc is added on our ougoing
link, settles when we send a settle message to the downstream peer.
If a failure occurs, we check whether it occurred at our node, then
notify a link or forwarding failure accordingly.
Note that this change also adds forward event notifications for sends
which are initiated by our node because the handling code for adding
a htlc which originates from our node is the same as that for handling
forwards. Htlcs for our locally initiated sends have our internal pid
set in the incoming htlcs id field, so we extract this value and notify
with a zero htlc id to be consistent with receives (which have zero
outgoing circuits). Subsequent settles or failures are not noitfied
for local sends in this commit, and will be handled in a follow up.
This commit sets more fields on the htlcPacket created to fail adding
a htlc packet to the switch for notification purposes. This new data is
copied by value from the original packet. The packet is then failed
back to the peer that forwarded us the packet, which is handled by
handledownstream packet. The values added to the packet are not used
in the handling of a failed packet.
In this commit, a htlcNotifier interface is added to allow for easy
unit testing. Instances of the HtlcNotifier are added to the server,
switch and link.
This commit adds a HTLCNotifier to htlcswitch which HTLC events
will be piped through to provide clients with subscriptions to
HTLC level events.
The event types added are forward events (which occur for sends
from and forwards through our node), forward failues (when a
send or forward fails down the line), settles for forwards or
receives to our node and link failures which occur when a htlc
is failed at our node (which may occur for a send, receive or
foreward).
Since we want to handle the edge case where paying the HTLC fee would
take the initiator below the reserve, we move the subtraction of the
reserve into availableBalance where this calculation will be performed.
This commit enables the user to specify he is not interested in
automatically close channels with pending payments that their
corresponding htlcs have timed-out.
By requiring a configurable grace period uptime of our node
before closing such channels, we give a chance to the other node to
properly cancel the htlc and avoid unnecessary on-chain transaction.
In mobile it is very important for the user experience as otherwise
channels will be force closed more frequently.
This commit adds LinkErrors with failure details to htlcs which fail on
our incoming link. This change is made with the intention of notifying
detailed htlc failure reasons in sendHTLCError. The FailureDetail
interface is implemented on FailureResolutionResults so that they can
directly be used to enrich LinkErrors. sendHtlcError is updated to
take a LinkError in preparation for the addition of a htlcnotifier
which will notify the detail of the error.
This commit adds a linkError field to track the value of failures
which occur at our node. This field is set when local payments or
multi hop htlcs fail in the switch or on our outgoing link. This
addition is required for the addition of a htlc notifier which will
notify these failures in handleDownstreamPacket.
The passing of link error to failAddPacket removes the need for an
additional error field, because the link error's failure detail will
contain any additional metadata. In the places where the failure detail
does not cover all the metadata that was previously supplied by addr
err, the error is logged before calling failAddPacket so that this
change does not reduce the amount of information we log.
Add a FailureDetail interface which allows us have different kinds of
failures for link errors. This interface will be used to cover failures
that occur when on invoice payment, because the errors have already
been enumerated in the invoices package.
Rename FailureDetail in a separate commit so that a FailureDetail
interface can be introduced in the following commit.
OutgoingFailureOnionDecode is renamed to OutgoingFailureDecodeError
to specifically indicate that we could not decode the wire
failure that our payment experienced.
