Commit Graph

5721 Commits

Author SHA1 Message Date
Conner Fromknecht
a4442608e6
queue/queue: wait group inner go routines, add idempotency 2018-12-03 13:58:25 -08:00
Olaoluwa Osuntokun
5107feb1ce
Merge pull request #2261 from cfromknecht/makefile-unused-declarations
Makefile: remove unused HAVE_ declarations + cleanup
2018-12-03 13:34:43 -08:00
Brian Sipple
c9a5593a64 build: update make btcd command with GO111MODULE=on 2018-12-02 15:43:26 -08:00
Conner Fromknecht
7557bded50
Makefile: remove unused HAVE_ declarations + cleanup 2018-12-02 15:24:14 -08:00
Olaoluwa Osuntokun
f4b6e0b775
Merge pull request #2253 from monaco-ex/pr-fix-build-failure-on-docker-btcd
Fix the build error.
2018-12-01 13:58:23 -08:00
Cryptcoin Junkey
126787074b Fix the build error. 2018-12-01 03:06:38 +00:00
Olaoluwa Osuntokun
ec01bebb81
Merge pull request #1902 from cfromknecht/move-chan-series-to-discovery
Move chan series to discovery
2018-11-30 18:20:09 -08:00
Olaoluwa Osuntokun
7a39131a85
Merge pull request #2249 from cfromknecht/makefile-cleanup
Makefile: use travis targets, remove unnecessary symbols
2018-11-30 16:26:45 -08:00
Conner Fromknecht
ae46d2e716
Merge pull request #2211 from Roasbeef/notifier-reverse-scan
chainntnfs: modify all historical rescans to scan backwards
2018-11-30 16:26:24 -08:00
Conner Fromknecht
da21f33a2a
chan_series: remove chan_series from main 2018-11-30 15:33:34 -08:00
Conner Fromknecht
4e48a56d5e
server: init gossiper w/ discovery.ChanSeries 2018-11-30 15:33:34 -08:00
Conner Fromknecht
5fc0a192ad
discovery/syncer: move CGTimeSerires to chan_series 2018-11-30 15:33:34 -08:00
Conner Fromknecht
1c81a8fad6
discovery/chan_series: copy chan_series to discovery 2018-11-30 15:33:34 -08:00
Conner Fromknecht
157d181a66
Merge pull request #2194 from xsb/num-inactive-chan-reorder
lnrpc: reorder num_inactive_channels
2018-11-30 15:24:22 -08:00
Conner Fromknecht
0d9b5ecc3c
Makefile: use period for print for consistency 2018-11-30 15:20:50 -08:00
Conner Fromknecht
5039c106f6
make/testing_flags: remove unused PROD_TAGS 2018-11-30 15:20:49 -08:00
Conner Fromknecht
0ad3302e16
Makefile: remove unnecessary PROD_TAGS 2018-11-30 15:20:47 -08:00
Conner Fromknecht
2c716095d5
travis: use make targets instead of chaining 2018-11-30 15:20:46 -08:00
Conner Fromknecht
3e0ca05740
Makefile: add travis-race and travis-cover targets 2018-11-30 15:20:45 -08:00
Olaoluwa Osuntokun
8d624fa1fd
Merge pull request #2245 from Roasbeef/modules-fix
build: switch to btcd version that uses go modules, switch from bolt to bbolt
2018-11-30 14:59:53 -08:00
Olaoluwa Osuntokun
41fde4c796
build: update to latest btcd, neutrino, btcwallet 2018-11-30 14:26:28 -08:00
Olaoluwa Osuntokun
0734e53260
build: inline travis btcd build 2018-11-30 14:01:53 -08:00
Olaoluwa Osuntokun
24ab9970bf
build: modify make file to use go get with precise btcd hash 2018-11-30 14:01:00 -08:00
Olaoluwa Osuntokun
eafee05dd2
build: update go.mod and go.sum with latest btcd+neutrino+btcwallet 2018-11-30 13:59:39 -08:00
Olaoluwa Osuntokun
9e5ba86cef
build: force modules at the travis script level 2018-11-29 21:40:47 -08:00
Olaoluwa Osuntokun
1483af79da
build: force gomodules when executing linter 2018-11-29 21:35:51 -08:00
Olaoluwa Osuntokun
ac04306dcf
travis: collapse commands into single instance to ensure exit on fail
Before this commit, if for example the linter failed, then we would go
on to all the other tests rather than halting. We fix this by instead
chaining the relevant commands, and eliminating the LINT env variable
in the build matrix.
2018-11-29 21:27:55 -08:00
Olaoluwa Osuntokun
910757d30e
build: force go modules off when using metalinter in makefile 2018-11-29 21:26:30 -08:00
Olaoluwa Osuntokun
00001e7dad
watchtower: fix linter errors 2018-11-29 21:26:02 -08:00
Olaoluwa Osuntokun
9bdc1dc2f2
make: add env before go test for unit tests to support modules 2018-11-29 20:33:53 -08:00
Olaoluwa Osuntokun
0936aebf1d
build: remove glide usage to build btcd from Makefile
We no longer need glide to build btcd as btcd itself now uses go modules
as we do.
2018-11-29 20:33:50 -08:00
Olaoluwa Osuntokun
1fd3aac925
multi: switch from bolt packge to bbolt package for all imports 2018-11-29 20:33:49 -08:00
Olaoluwa Osuntokun
0edf1ae081
build: update to latest btcwallet, neutrino, bbolt versions 2018-11-29 20:01:45 -08:00
Conner Fromknecht
668bd4db33
Merge pull request #2094 from cfromknecht/validate-color
config: validate node color after reading config
2018-11-29 16:18:44 -08:00
Olaoluwa Osuntokun
fc4fe07010
Merge pull request #2081 from Roasbeef/signer-service
lnrpc+lnd+rpc: add new Signer RPC service, and sub RPC server infrastructure
2018-11-28 21:22:25 -08:00
Olaoluwa Osuntokun
b0a7a57f57
lnrpc/signrpc: add ComputeInputScript implementation 2018-11-28 20:57:13 -08:00
Olaoluwa Osuntokun
6c201e435a
lnrpc/signrpc: add ComputeInputScript to the Signer sub-server
In this commit, we add the ComputeInputScript which will allow callers
to obtain witnesses for all outputs under control of the wallet. This
allows external scripting of things like coin join, etc.
2018-11-28 20:57:11 -08:00
Olaoluwa Osuntokun
a8ac3cfe7d
lnd+rpc: fix linter errors 2018-11-28 20:57:10 -08:00
Olaoluwa Osuntokun
35b4b35eae
lnrpc: add new recursive proto generation script
In this commit, we add a recursive proto generation script. This avoids
having to add a new script for each upcoming sub-server.
2018-11-28 20:57:08 -08:00
Olaoluwa Osuntokun
06d5f2db37
macaroons: update line folding to project style 2018-11-28 20:57:07 -08:00
Olaoluwa Osuntokun
ff47ade13b
lnd+rpc: modify rpcServer to fully manaage listeners and gRPC, handle sub-servers
In this commit, we modify the existing rpcServer to fully manage the
macaroons, gRPC server, and also seek out and create all sub-servers.
With this change, the RPC server gains more responsibility, as it
becomes the "root" server in the hierarchy of gRPC sub-servers.

In addition to creating each sub-server, it will also merge the set of
macaroon permissions for each sub-server, with the permissions of the
rest of the RPC infra. As a result, each sub-server is able to
independently specify what it needs w.r.t macaroon permissions and have
that taken care of by the RPC server. In order to achieve this, we need
to unify the creation of the RPC interceptors, and also fully manage the
gRPC server ourselves.

Some examples with various build configs:
```
⛰i  make build
 Building debug lnd and lncli.
go build -v -tags="dev" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd
go build -v -tags="dev" -o lncli-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd/cmd/lncli

⛰i  ./lnd-debug --debuglevel=debug --signrpc.signermacaroonpath=~/sign.macaroon
unknown flag `signrpc.signermacaroonpath'
unknown flag `signrpc.signermacaroonpath'

⛰i  make build tags=signerrpc
 Building debug lnd and lncli.
go build -v -tags="dev signerrpc" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd
go build -v -tags="dev signerrpc" -o lncli-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd/cmd/lncli

⛰i  ./lnd-debug --debuglevel=debug --signrpc.signermacaroonpath=~/sign.macaroon
2018-10-22 17:31:01.132 [INF] LTND: Version: 0.5.0-beta commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty, build=development, logging=default
2018-10-22 17:31:01.133 [INF] LTND: Active chain: Bitcoin (network=simnet)
2018-10-22 17:31:01.140 [INF] CHDB: Checking for schema update: latest_version=6, db_version=6
2018-10-22 17:31:01.236 [INF] LTND: Primary chain is set to: bitcoin
2018-10-22 17:31:02.391 [INF] LNWL: Opened wallet
2018-10-22 17:31:03.315 [INF] LNWL: The wallet has been unlocked without a time limit
2018-10-22 17:31:03.315 [INF] LTND: LightningWallet opened
2018-10-22 17:31:03.319 [INF] LNWL: Catching up block hashes to height 3060, this will take a while...
2018-10-22 17:31:03.320 [INF] HSWC: Restoring in-memory circuit state from disk
2018-10-22 17:31:03.320 [INF] LNWL: Done catching up block hashes
2018-10-22 17:31:03.320 [INF] HSWC: Payment circuits loaded: num_pending=0, num_open=0
2018-10-22 17:31:03.322 [DBG] LTND: Populating dependencies for sub RPC server: Signrpc
```

As for the config, an example is:
```
[signrpc]
signrpc.signermacaroonpath=~/signer.macaroon
```
2018-11-28 20:57:06 -08:00
Olaoluwa Osuntokun
4002caec69
config+rpc: create new generalized dynamic sub rpc server config framework
In this commit, we add the glue infrastructure to make the sub RPC
server system work properly. Our high level goal is the following: using
only the lnrpc package (with no visibility into the sub RPC servers),
the RPC server is able to find, create, run, and manage the entire set
of present and future sub RPC servers. In order to achieve this, we use
the reflect package and build tags heavily to permit a loosely coupled
configuration parsing system for the sub RPC servers.

We start with a new `subRpcServerConfigs` struct which is _always_
present.  This struct has its own group, and will house a series of
sub-configs, one for each sub RPC server. Each sub-config is actually
gated behind a build flag, and can be used to allow users on the command
line or in the config to specify arguments related to the sub-server. If
the config isn't present, then we don't attempt to parse it at all, if
it is, then that means the RPC server has been registered, and we should
parse the contents of its config.

The `subRpcServerConfigs` struct has two main methods:
`PopulateDependancies` and `FetchConfig`. The `PopulateDependancies` is
used to dynamically locate and set the config fields for each new
sub-server. As the config may not actually have any fields (if the build
flag is off), we use the reflect pacakge to determine if things are
compiled in or not, then if so, we dynamically set each of the config
parameters. The `PopulateDependancies` method implements the
`lnrpc.SubServerConfigDispatcher` interface. Our goal is to allow sub
servers to look up their actual config in this main config struct. We
achieve this by using reflect to look up the target field _as if it were
a key in a map_. If the field is found, then we check if it has any
actual attributes (it won't if the build flag is off), if it is, then we
return it as we expect it to be populated already.
2018-11-28 20:57:05 -08:00
Olaoluwa Osuntokun
16d16cf1b6
log: register logger for new Signer RPC Service, namespace SGNR 2018-11-28 20:57:04 -08:00
Olaoluwa Osuntokun
8971931aa3
lnrpc/signrpc: add lnrpc.SubServerDriver for signrpc
In this commit, we create a lnrpc.SubServerDriver for signrpc. Note that
this file will only have its init() method executed if the proper build
flag is on. As a result, only if the build flag is set, will the RPC
server be registered, and visible at the packge lnrpc level for the root
server to manipulate.
2018-11-28 20:57:03 -08:00
Olaoluwa Osuntokun
b7757683b2
lnrpc/signrpc: implement new SignerServer sub RPC server
In this commit, we add a full implementation of the new SignerServer sub
RPC service within the main root RPC service. This service is able to
fully manage its macaroons, and service any connected clients. Atm, this
service only has a single method: SignOutputRaw which mimics the
existing lnwallet.Signer interface within lnd itself. As the API's are
so similar, it will be possible for a client to directly use the
lnwallet.Signer interface, and have a proxy that sends the request over
RPC, and translates the proto layer on both sides. To the client, it
doesn't know that it's using a remote, or local RPC.
2018-11-28 20:57:01 -08:00
Olaoluwa Osuntokun
184f160fb9
lnrpc: add new sub RPC server registration system
In this commit, we add the scafolding for the future sub-server RPC
system. The idea is that each sub server will implement this particular
interface. From there on, a "root" RPC server is able to query this
registry, and dynamically create each  sub-sever instance without
knowing the details of each sub-server.

In the init() method of the pacakge of a sub-server, the sub-server is
to call: RegisterSubServer to claim its namespace. Afterwards, the root
RPC server can use the RegisteredSubServers() method to obtain a slice
of ALL regsitered sub-servers. Once this list is obtained, it can use
the New() method of the SubServerDriver struct to create a new
sub-server instance.

Each sub-server needs to be able to locate it's primary config using the
SubServerConfigDispatcher interface. This can be a map of maps, or a
regular config structr. The main requirement is that the sub-server be
able to find a config under the same name that it registered with. This
string of abstractions will allow the main RPC server to find, create,
and run each sub-server without knowing the details of its configuration
or its role.
2018-11-28 20:57:00 -08:00
Olaoluwa Osuntokun
9f24049bbc
lnrpc/signrpc: add new gen_protos.sh to generate minimal protos
In this commit, we add a new proto generation script to match the one in
the main lnrpc package. This script differs, as we don't need to
generate the REST proxy stuff (for now).
2018-11-28 20:56:59 -08:00
Olaoluwa Osuntokun
bbbdd7f6e9
lnrpc/signrpc: add and generate new set of protos for Signer service 2018-11-28 20:56:58 -08:00
Olaoluwa Osuntokun
a432f9a4c8 lnrpc/signrpc: create new signrpc package with build-flag guarded config
In this commit, we introduce a new sub-package within the greater RPC
package.  This new sub-package will house a new set of sub-RPC servers
to expose experimental features behind build flags for upstream
consumers. In this commit, we add the first config for the service,
which will simply expose the lnwallet.Signer interface over RPC.

In the default file, we have what the config will be if the build tag
(signerrpc) is off. In this case, the config parser won't detect any
times, and if specified will error out. In the active file, we have the
true config that the server will use. With this new set up, we'll
exploit these build flags heavily in order to create a generalized
framework for adding additional sub RPC servers.
2018-11-28 20:56:38 -08:00
Olaoluwa Osuntokun
b5dd1863f6
Merge pull request #2239 from Roasbeef/go-modules
build: switch from dep to go modules
2018-11-28 20:53:03 -08:00