Commit Graph

6517 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
0734e53260
build: inline travis btcd build 2018-11-30 14:01:53 -08:00
Olaoluwa Osuntokun
24ab9970bf
build: modify make file to use go get with precise btcd hash 2018-11-30 14:01:00 -08:00
Olaoluwa Osuntokun
eafee05dd2
build: update go.mod and go.sum with latest btcd+neutrino+btcwallet 2018-11-30 13:59:39 -08:00
Olaoluwa Osuntokun
9e5ba86cef
build: force modules at the travis script level 2018-11-29 21:40:47 -08:00
Olaoluwa Osuntokun
1483af79da
build: force gomodules when executing linter 2018-11-29 21:35:51 -08:00
Olaoluwa Osuntokun
ac04306dcf
travis: collapse commands into single instance to ensure exit on fail
Before this commit, if for example the linter failed, then we would go
on to all the other tests rather than halting. We fix this by instead
chaining the relevant commands, and eliminating the LINT env variable
in the build matrix.
2018-11-29 21:27:55 -08:00
Olaoluwa Osuntokun
910757d30e
build: force go modules off when using metalinter in makefile 2018-11-29 21:26:30 -08:00
Olaoluwa Osuntokun
00001e7dad
watchtower: fix linter errors 2018-11-29 21:26:02 -08:00
Olaoluwa Osuntokun
9bdc1dc2f2
make: add env before go test for unit tests to support modules 2018-11-29 20:33:53 -08:00
Olaoluwa Osuntokun
0936aebf1d
build: remove glide usage to build btcd from Makefile
We no longer need glide to build btcd as btcd itself now uses go modules
as we do.
2018-11-29 20:33:50 -08:00
Olaoluwa Osuntokun
1fd3aac925
multi: switch from bolt packge to bbolt package for all imports 2018-11-29 20:33:49 -08:00
Olaoluwa Osuntokun
0edf1ae081
build: update to latest btcwallet, neutrino, bbolt versions 2018-11-29 20:01:45 -08:00
Conner Fromknecht
668bd4db33
Merge pull request #2094 from cfromknecht/validate-color
config: validate node color after reading config
2018-11-29 16:18:44 -08:00
Johan T. Halseth
bea31310e1
lnd_test: make sure miner have seen tx before mining
This commit makes mineBlocks take an argument to check the number of txs
expected in the mempool before mining. By using waitForTxInMempool
before mining blocks we ensure that the txs in question have actually
propagated to the miner's mempool before mining the blocks.
2018-11-29 12:33:16 +01:00
Johan T. Halseth
cdcb7105b3
lnd_test: assert existence of unconfirmed tx in mempool 2018-11-29 12:31:40 +01:00
Joost Jager
6ba1144528
routing: move failed channels map into payment session
This is a small preparatory step towards moving mission control logic
out of router and reusing the acquired routing result data.
2018-11-29 10:31:38 +01:00
Joost Jager
ac04729cff
routing: use complete route in test
Previously not all route fields were properly populated. Example: prev
and next hop maps.
2018-11-29 10:31:38 +01:00
Joost Jager
71037969b7
routing: remove pruneVertexFailure function 2018-11-29 10:31:38 +01:00
Joost Jager
aca136a91c
routing: remove unused pruneVertexFailure parameters 2018-11-29 10:31:38 +01:00
Joost Jager
dd7e2e9e04
routing: move logging into applyChannelUpdate
To remove code duplicated at all call sites to check err and log.
2018-11-29 10:31:38 +01:00
Olaoluwa Osuntokun
fc4fe07010
Merge pull request #2081 from Roasbeef/signer-service
lnrpc+lnd+rpc: add new Signer RPC service, and sub RPC server infrastructure
2018-11-28 21:22:25 -08:00
Olaoluwa Osuntokun
b0a7a57f57
lnrpc/signrpc: add ComputeInputScript implementation 2018-11-28 20:57:13 -08:00
Olaoluwa Osuntokun
6c201e435a
lnrpc/signrpc: add ComputeInputScript to the Signer sub-server
In this commit, we add the ComputeInputScript which will allow callers
to obtain witnesses for all outputs under control of the wallet. This
allows external scripting of things like coin join, etc.
2018-11-28 20:57:11 -08:00
Olaoluwa Osuntokun
a8ac3cfe7d
lnd+rpc: fix linter errors 2018-11-28 20:57:10 -08:00
Olaoluwa Osuntokun
35b4b35eae
lnrpc: add new recursive proto generation script
In this commit, we add a recursive proto generation script. This avoids
having to add a new script for each upcoming sub-server.
2018-11-28 20:57:08 -08:00
Olaoluwa Osuntokun
06d5f2db37
macaroons: update line folding to project style 2018-11-28 20:57:07 -08:00
Olaoluwa Osuntokun
ff47ade13b
lnd+rpc: modify rpcServer to fully manaage listeners and gRPC, handle sub-servers
In this commit, we modify the existing rpcServer to fully manage the
macaroons, gRPC server, and also seek out and create all sub-servers.
With this change, the RPC server gains more responsibility, as it
becomes the "root" server in the hierarchy of gRPC sub-servers.

In addition to creating each sub-server, it will also merge the set of
macaroon permissions for each sub-server, with the permissions of the
rest of the RPC infra. As a result, each sub-server is able to
independently specify what it needs w.r.t macaroon permissions and have
that taken care of by the RPC server. In order to achieve this, we need
to unify the creation of the RPC interceptors, and also fully manage the
gRPC server ourselves.

Some examples with various build configs:
```
⛰i  make build
 Building debug lnd and lncli.
go build -v -tags="dev" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd
go build -v -tags="dev" -o lncli-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd/cmd/lncli

⛰i  ./lnd-debug --debuglevel=debug --signrpc.signermacaroonpath=~/sign.macaroon
unknown flag `signrpc.signermacaroonpath'
unknown flag `signrpc.signermacaroonpath'

⛰i  make build tags=signerrpc
 Building debug lnd and lncli.
go build -v -tags="dev signerrpc" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd
go build -v -tags="dev signerrpc" -o lncli-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd/cmd/lncli

⛰i  ./lnd-debug --debuglevel=debug --signrpc.signermacaroonpath=~/sign.macaroon
2018-10-22 17:31:01.132 [INF] LTND: Version: 0.5.0-beta commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty, build=development, logging=default
2018-10-22 17:31:01.133 [INF] LTND: Active chain: Bitcoin (network=simnet)
2018-10-22 17:31:01.140 [INF] CHDB: Checking for schema update: latest_version=6, db_version=6
2018-10-22 17:31:01.236 [INF] LTND: Primary chain is set to: bitcoin
2018-10-22 17:31:02.391 [INF] LNWL: Opened wallet
2018-10-22 17:31:03.315 [INF] LNWL: The wallet has been unlocked without a time limit
2018-10-22 17:31:03.315 [INF] LTND: LightningWallet opened
2018-10-22 17:31:03.319 [INF] LNWL: Catching up block hashes to height 3060, this will take a while...
2018-10-22 17:31:03.320 [INF] HSWC: Restoring in-memory circuit state from disk
2018-10-22 17:31:03.320 [INF] LNWL: Done catching up block hashes
2018-10-22 17:31:03.320 [INF] HSWC: Payment circuits loaded: num_pending=0, num_open=0
2018-10-22 17:31:03.322 [DBG] LTND: Populating dependencies for sub RPC server: Signrpc
```

As for the config, an example is:
```
[signrpc]
signrpc.signermacaroonpath=~/signer.macaroon
```
2018-11-28 20:57:06 -08:00
Olaoluwa Osuntokun
4002caec69
config+rpc: create new generalized dynamic sub rpc server config framework
In this commit, we add the glue infrastructure to make the sub RPC
server system work properly. Our high level goal is the following: using
only the lnrpc package (with no visibility into the sub RPC servers),
the RPC server is able to find, create, run, and manage the entire set
of present and future sub RPC servers. In order to achieve this, we use
the reflect package and build tags heavily to permit a loosely coupled
configuration parsing system for the sub RPC servers.

We start with a new `subRpcServerConfigs` struct which is _always_
present.  This struct has its own group, and will house a series of
sub-configs, one for each sub RPC server. Each sub-config is actually
gated behind a build flag, and can be used to allow users on the command
line or in the config to specify arguments related to the sub-server. If
the config isn't present, then we don't attempt to parse it at all, if
it is, then that means the RPC server has been registered, and we should
parse the contents of its config.

The `subRpcServerConfigs` struct has two main methods:
`PopulateDependancies` and `FetchConfig`. The `PopulateDependancies` is
used to dynamically locate and set the config fields for each new
sub-server. As the config may not actually have any fields (if the build
flag is off), we use the reflect pacakge to determine if things are
compiled in or not, then if so, we dynamically set each of the config
parameters. The `PopulateDependancies` method implements the
`lnrpc.SubServerConfigDispatcher` interface. Our goal is to allow sub
servers to look up their actual config in this main config struct. We
achieve this by using reflect to look up the target field _as if it were
a key in a map_. If the field is found, then we check if it has any
actual attributes (it won't if the build flag is off), if it is, then we
return it as we expect it to be populated already.
2018-11-28 20:57:05 -08:00
Olaoluwa Osuntokun
16d16cf1b6
log: register logger for new Signer RPC Service, namespace SGNR 2018-11-28 20:57:04 -08:00
Olaoluwa Osuntokun
8971931aa3
lnrpc/signrpc: add lnrpc.SubServerDriver for signrpc
In this commit, we create a lnrpc.SubServerDriver for signrpc. Note that
this file will only have its init() method executed if the proper build
flag is on. As a result, only if the build flag is set, will the RPC
server be registered, and visible at the packge lnrpc level for the root
server to manipulate.
2018-11-28 20:57:03 -08:00
Olaoluwa Osuntokun
b7757683b2
lnrpc/signrpc: implement new SignerServer sub RPC server
In this commit, we add a full implementation of the new SignerServer sub
RPC service within the main root RPC service. This service is able to
fully manage its macaroons, and service any connected clients. Atm, this
service only has a single method: SignOutputRaw which mimics the
existing lnwallet.Signer interface within lnd itself. As the API's are
so similar, it will be possible for a client to directly use the
lnwallet.Signer interface, and have a proxy that sends the request over
RPC, and translates the proto layer on both sides. To the client, it
doesn't know that it's using a remote, or local RPC.
2018-11-28 20:57:01 -08:00
Olaoluwa Osuntokun
184f160fb9
lnrpc: add new sub RPC server registration system
In this commit, we add the scafolding for the future sub-server RPC
system. The idea is that each sub server will implement this particular
interface. From there on, a "root" RPC server is able to query this
registry, and dynamically create each  sub-sever instance without
knowing the details of each sub-server.

In the init() method of the pacakge of a sub-server, the sub-server is
to call: RegisterSubServer to claim its namespace. Afterwards, the root
RPC server can use the RegisteredSubServers() method to obtain a slice
of ALL regsitered sub-servers. Once this list is obtained, it can use
the New() method of the SubServerDriver struct to create a new
sub-server instance.

Each sub-server needs to be able to locate it's primary config using the
SubServerConfigDispatcher interface. This can be a map of maps, or a
regular config structr. The main requirement is that the sub-server be
able to find a config under the same name that it registered with. This
string of abstractions will allow the main RPC server to find, create,
and run each sub-server without knowing the details of its configuration
or its role.
2018-11-28 20:57:00 -08:00
Olaoluwa Osuntokun
9f24049bbc
lnrpc/signrpc: add new gen_protos.sh to generate minimal protos
In this commit, we add a new proto generation script to match the one in
the main lnrpc package. This script differs, as we don't need to
generate the REST proxy stuff (for now).
2018-11-28 20:56:59 -08:00
Olaoluwa Osuntokun
bbbdd7f6e9
lnrpc/signrpc: add and generate new set of protos for Signer service 2018-11-28 20:56:58 -08:00
Olaoluwa Osuntokun
a432f9a4c8 lnrpc/signrpc: create new signrpc package with build-flag guarded config
In this commit, we introduce a new sub-package within the greater RPC
package.  This new sub-package will house a new set of sub-RPC servers
to expose experimental features behind build flags for upstream
consumers. In this commit, we add the first config for the service,
which will simply expose the lnwallet.Signer interface over RPC.

In the default file, we have what the config will be if the build tag
(signerrpc) is off. In this case, the config parser won't detect any
times, and if specified will error out. In the active file, we have the
true config that the server will use. With this new set up, we'll
exploit these build flags heavily in order to create a generalized
framework for adding additional sub RPC servers.
2018-11-28 20:56:38 -08:00
Olaoluwa Osuntokun
b5dd1863f6
Merge pull request #2239 from Roasbeef/go-modules
build: switch from dep to go modules
2018-11-28 20:53:03 -08:00
Olaoluwa Osuntokun
eda45adb5b
docs: update docs to reflect new go modules usage and building 2018-11-28 20:13:56 -08:00
Olaoluwa Osuntokun
f65b1a4d6e
travis: force go modules support in $GOPATH, cache modules directory 2018-11-28 20:13:54 -08:00
Olaoluwa Osuntokun
e2e73379c7
build: update makefile to be aware of go modules, remove dep support
In this commit, we update the makefile to be aware of go modules. Along
the way, we remove all references to dep as we no longer use it within
this project. Note that in order to allow usage of go modules within the
$GOPATH directory, we set the `GO111MODULE=on` environment variable.
2018-11-28 20:13:52 -08:00
Olaoluwa Osuntokun
e5f4b7ca66
travis: remove dep directive from travis, as we use go modules now 2018-11-28 20:13:51 -08:00
Olaoluwa Osuntokun
e6c22d658d
docs: update contribution guidelines to reference latest version of Go
with '#' will be ignored, and an empty message aborts the commit.
2018-11-28 20:13:50 -08:00
Olaoluwa Osuntokun
f09a4daeb1
docs/MAKEFILE: remove obsolete travis section
In this commit, we remove the travis section as the directive has been
removed from the makefile.
2018-11-28 20:13:49 -08:00
Olaoluwa Osuntokun
ca28c0b5a1
docs/MAKEFILE: remove dep section
In this commit, we remove the dep section as we no longer use it, and
instead use go modues.
2018-11-28 20:13:48 -08:00
Olaoluwa Osuntokun
ebff57a1f7
build: remove old stale dep files 2018-11-28 20:13:46 -08:00
Olaoluwa Osuntokun
4a29f56a3f build: add new go.mod and go.sum files for go modules usage 2018-11-28 17:19:10 -08:00
Olaoluwa Osuntokun
4e83ce5c00
Merge pull request #1892 from halseth/travis-caching
travis cache
2018-11-28 17:07:40 -08:00
Olaoluwa Osuntokun
fd82200a15
Merge pull request #2140 from cfromknecht/wtserver-move
[watchtower/wtserver] rename server package, add godocs, general code health
2018-11-28 15:07:01 -08:00
Johan T. Halseth
b07499f227
Merge pull request #2224 from halseth/itest-timeouts
lnd_test: increase test timeouts
2018-11-28 08:22:46 +01:00
Johan T. Halseth
7e48b012e2
Merge pull request #2230 from halseth/itests-tryconnect-peer
lntest/harness: retry ConnectPeer of chain backend still syncing
2018-11-28 08:21:08 +01:00
Johan T. Halseth
0011bcdad1
Merge pull request #2229 from halseth/global-defaultcsv
lnd_test: make tests use global defaultCSV
2018-11-28 08:19:58 +01:00