In this commit, we split the database storage into two classes: remote
and local data. If etcd isn't active, then everything is actually just
local though we use two pointers everywhere. If etcd is active, then
everything but the graph goes into the remote database.
In this commit, we modify the existing `GetBackend` method to now be
called `GetBackends`. This new method will populate a new `RemoteDB`
attribute based on if the replicated backend is active or not. As is,
the local backend is used everywhere. An upcoming commit will once again
re-enable the remote backend, in a hybrid manner.
For security reasons, browsers are limited in the header fields they can
send when opening a WebSocket connection. Specifically, the macaroon
cannot be sent in the Grpc-Metadata-Macaroon header field as that would
be possible for normal REST requests. Instead we only have the special
field "Sec-Websocket-Protocol" that can be used to transport custom
data. We allow the macaroon to be sent there and transform it into a
proper header field for the target request.
According to the recent discussion `noseedbackup` is not deprecated.
This change clarifies the message about deprecation.
Also fixes a typo.
Closes#4499
We use the event timestamp of a forwarding event as its primary storage
key. On systems with a bad clock resolution this can lead to collisions
of the events if some of the timestamps are identical. We fix this
problem by shifting the timestamps on the nanosecond level until only
unique values remain.
This commit adds an integration test that runs on a Windows virtual
machine on Travis. The tests run inside of a "Git Bash" environment
which supports the same command line syntax as a proper bash but doesn't
have all the tooling installed. Some tools also behave differently on
Windows. Therefore we also have to simplify the command to upload the
logs to termbin and remove the upload to file.io on Windows because both
the find and tar command don't work as expected.
In this commit, unify the old and new values for `sync-freelist`, and
also ensure that we don't break behavior for any users that're using the
_old_ value.
To do this, we first rename what was `--db.bolt.no-sync-freelist`, to
`--db.bolt.sync-freelist`. This gets rid of the negation on the config
level, and lets us override that value if the user is specifying the
legacy config option.
In the future, we'll deprecate the old config option, in favor of the
new DB scoped option.
This value actually isn't read anywhere, since it's no longer used.
Instead, `cfg.Db.Bolt.NoSyncFreeList` is what's evaluated when we go to
open the DB.
This commit extends compatibility with the bbolt kvdb implementation,
which returns ErrIncompatibleValue in case of a bucket/value key
collision. Furthermore the commit also adds an extra precondition to the
transaction when a key doesn't exist. This is needed as we fix reads to
a snapshot revision and other writers may commit the key otherwise.
This commit includes a regression test that checks that we remember
to restore updates that we sent to the peer but they haven't sent
us a signature for yet.
This fixes a long-standing force close bug. When we receive a
revocation, store the updates that the remote should sign next under
a new database key. Previously, these were not persisted which would
lead to force closure.
This commit removes the lock set which was used to only add bucket keys
to the tx predicate while also bumping their mod version.
This was useful to reduce the size of the compare set but wasn't useful
to reduce contention as top level buckets were always in the lock set.
This commit moves makeTestDB to db.go and exports it so that we'll be
able to use this function in other unit tests to make them testable with
etcd if needed.
This commit changes the key derivation algo we use to emulate buckets
similar to bbolt. The issue with prefixing keys with either a bucket or
a value prefix is that the cursor couldn't effectively iterate trough
all keys in a bucket, as it skipped the bucket keys.
While there are multiple ways to fix that issue (eg. two pointers,
iterating value keys then bucket keys, etc), the cleanest is to instead
of prefixes in keys we use a postfix indicating whether a key is a
bucket or a value. This also simplifies all operations where we
(recursively) iterate a bucket and is equivalent with the prefixing key
derivation with the addition that bucket and value keys are now
continous.
Previously, we could sign a pending commitment for the remote party,
disconnect, and not restore these signed remote updates as having been
removed at the pending commitment height. This commit fixes that to
look up whether the update under the unsigned acked updates key is
present on the pending commitment or not and appropriately set
the remove commit heights.
This commit moves the deletion of all updates under the unsigned
acked updates key from AppendRemoteCommitChain to
AdvanceCommitChainTail. This is done because if we went down after
signing for these updates but before receiving a revocation, we would
incorrectly reject their commitment signature:
Alice Bob
-----add----->
-----sig----->
<----rev------
<----sig------
-----rev----->
<----fail-----
<----sig------
-----rev----->
-----sig----->
*reconnect*
<----rev------
<----add------
x----sig------
It is also important to note that filtering is required when we
receive a revocation to ensure that we aren't erroneously deleting
remote updates. Take the following state transitions:
Alice Bob
-----add----->
-----sig----->
<----rev------
<----sig------
-----rev----->
-----add----->
-----sig----->
<----fail-----
<----sig------
-----rev-----> (alice stores updates here)
<----rev------
In the above case, if Alice deleted all updates rather than filtering
when receiving the final revocation from Bob, then Alice would have
to force close the channel due to missing updates. Since Alice hasn't
signed for any of the unsigned acked updates, she should not filter any
of them out.
