This commit adds an integration test that runs on a Windows virtual
machine on Travis. The tests run inside of a "Git Bash" environment
which supports the same command line syntax as a proper bash but doesn't
have all the tooling installed. Some tools also behave differently on
Windows. Therefore we also have to simplify the command to upload the
logs to termbin and remove the upload to file.io on Windows because both
the find and tar command don't work as expected.
In this commit, unify the old and new values for `sync-freelist`, and
also ensure that we don't break behavior for any users that're using the
_old_ value.
To do this, we first rename what was `--db.bolt.no-sync-freelist`, to
`--db.bolt.sync-freelist`. This gets rid of the negation on the config
level, and lets us override that value if the user is specifying the
legacy config option.
In the future, we'll deprecate the old config option, in favor of the
new DB scoped option.
This value actually isn't read anywhere, since it's no longer used.
Instead, `cfg.Db.Bolt.NoSyncFreeList` is what's evaluated when we go to
open the DB.
This commit extends compatibility with the bbolt kvdb implementation,
which returns ErrIncompatibleValue in case of a bucket/value key
collision. Furthermore the commit also adds an extra precondition to the
transaction when a key doesn't exist. This is needed as we fix reads to
a snapshot revision and other writers may commit the key otherwise.
This commit includes a regression test that checks that we remember
to restore updates that we sent to the peer but they haven't sent
us a signature for yet.
This fixes a long-standing force close bug. When we receive a
revocation, store the updates that the remote should sign next under
a new database key. Previously, these were not persisted which would
lead to force closure.
Follow up labelling of external transactions with labels for the
transaction types we create within lnd. Since these labels will live
a life of string matching, a version number and rigid format is added
so that string matching is less painful. We start out with channel ID,
where available, and a transaction "type". External labels, added in a
previous PR, are not updated to this new versioned label because they
are not lnd-initiated transactions. Label matching can check this case,
then check for a version number.
This commit removes the lock set which was used to only add bucket keys
to the tx predicate while also bumping their mod version.
This was useful to reduce the size of the compare set but wasn't useful
to reduce contention as top level buckets were always in the lock set.
This commit moves makeTestDB to db.go and exports it so that we'll be
able to use this function in other unit tests to make them testable with
etcd if needed.
This commit changes the key derivation algo we use to emulate buckets
similar to bbolt. The issue with prefixing keys with either a bucket or
a value prefix is that the cursor couldn't effectively iterate trough
all keys in a bucket, as it skipped the bucket keys.
While there are multiple ways to fix that issue (eg. two pointers,
iterating value keys then bucket keys, etc), the cleanest is to instead
of prefixes in keys we use a postfix indicating whether a key is a
bucket or a value. This also simplifies all operations where we
(recursively) iterate a bucket and is equivalent with the prefixing key
derivation with the addition that bucket and value keys are now
continous.
Previously, we could sign a pending commitment for the remote party,
disconnect, and not restore these signed remote updates as having been
removed at the pending commitment height. This commit fixes that to
look up whether the update under the unsigned acked updates key is
present on the pending commitment or not and appropriately set
the remove commit heights.
This commit moves the deletion of all updates under the unsigned
acked updates key from AppendRemoteCommitChain to
AdvanceCommitChainTail. This is done because if we went down after
signing for these updates but before receiving a revocation, we would
incorrectly reject their commitment signature:
Alice Bob
-----add----->
-----sig----->
<----rev------
<----sig------
-----rev----->
<----fail-----
<----sig------
-----rev----->
-----sig----->
*reconnect*
<----rev------
<----add------
x----sig------
It is also important to note that filtering is required when we
receive a revocation to ensure that we aren't erroneously deleting
remote updates. Take the following state transitions:
Alice Bob
-----add----->
-----sig----->
<----rev------
<----sig------
-----rev----->
-----add----->
-----sig----->
<----fail-----
<----sig------
-----rev-----> (alice stores updates here)
<----rev------
In the above case, if Alice deleted all updates rather than filtering
when receiving the final revocation from Bob, then Alice would have
to force close the channel due to missing updates. Since Alice hasn't
signed for any of the unsigned acked updates, she should not filter any
of them out.
This update previously happened in 1589810 but was overwritten again by
a later PR. We need to use a version that doesn't include the broken ARM
assembly for poly1305. We might as well use the latest version of the
library.
This commit clamps all user-chosen CLTVs in LND to be at least 18, which
is the new conservative value used in the sepc. This minimum is applied
uniformly to forwarding CLTV deltas (via channel updates) as well as
final CLTV deltas for new invoices.
This commit copies over the relevant zpay32 decoding logic to ensure
that our prior migrations aren't affected by upcoming changes to the
zpay32 package, most notably changes to the default final_cltv_expiry
and expiry values.
As a preliminary step to isolating zpay32 in migrations 01-11, we'll
split out the encoding and decoding logic into separate files. Migration
11 only requires invoice decoding, so this prevents us from needing to
copy in the encoding logic that would otherwise be unused.
To free up build in Travis, we decided to run the non-flaky parts of
the CI pipeline in GitHub Workflows/Actions only. The integration tests
on the other hand are removed from GitHub because individual actions
cannot be restarted there which caused us to restart the whole workflow
if one test was flaky.
This split should give us the best of both worlds: Fast run of small
checks, linting and unit tests with an easy overview of what failed in
the PR directly. And more free build slots on Travis to do more advanced
integration tests on other architectures and/or operating systems. And
the option to restart a single flaky integration test on Travis.
