Commit Graph

6203 Commits

Author SHA1 Message Date
Johan T. Halseth
cdcb7105b3
lnd_test: assert existence of unconfirmed tx in mempool 2018-11-29 12:31:40 +01:00
Joost Jager
6ba1144528
routing: move failed channels map into payment session
This is a small preparatory step towards moving mission control logic
out of router and reusing the acquired routing result data.
2018-11-29 10:31:38 +01:00
Joost Jager
ac04729cff
routing: use complete route in test
Previously not all route fields were properly populated. Example: prev
and next hop maps.
2018-11-29 10:31:38 +01:00
Joost Jager
71037969b7
routing: remove pruneVertexFailure function 2018-11-29 10:31:38 +01:00
Joost Jager
aca136a91c
routing: remove unused pruneVertexFailure parameters 2018-11-29 10:31:38 +01:00
Joost Jager
dd7e2e9e04
routing: move logging into applyChannelUpdate
To remove code duplicated at all call sites to check err and log.
2018-11-29 10:31:38 +01:00
Olaoluwa Osuntokun
fc4fe07010
Merge pull request #2081 from Roasbeef/signer-service
lnrpc+lnd+rpc: add new Signer RPC service, and sub RPC server infrastructure
2018-11-28 21:22:25 -08:00
Olaoluwa Osuntokun
b0a7a57f57
lnrpc/signrpc: add ComputeInputScript implementation 2018-11-28 20:57:13 -08:00
Olaoluwa Osuntokun
6c201e435a
lnrpc/signrpc: add ComputeInputScript to the Signer sub-server
In this commit, we add the ComputeInputScript which will allow callers
to obtain witnesses for all outputs under control of the wallet. This
allows external scripting of things like coin join, etc.
2018-11-28 20:57:11 -08:00
Olaoluwa Osuntokun
a8ac3cfe7d
lnd+rpc: fix linter errors 2018-11-28 20:57:10 -08:00
Olaoluwa Osuntokun
35b4b35eae
lnrpc: add new recursive proto generation script
In this commit, we add a recursive proto generation script. This avoids
having to add a new script for each upcoming sub-server.
2018-11-28 20:57:08 -08:00
Olaoluwa Osuntokun
06d5f2db37
macaroons: update line folding to project style 2018-11-28 20:57:07 -08:00
Olaoluwa Osuntokun
ff47ade13b
lnd+rpc: modify rpcServer to fully manaage listeners and gRPC, handle sub-servers
In this commit, we modify the existing rpcServer to fully manage the
macaroons, gRPC server, and also seek out and create all sub-servers.
With this change, the RPC server gains more responsibility, as it
becomes the "root" server in the hierarchy of gRPC sub-servers.

In addition to creating each sub-server, it will also merge the set of
macaroon permissions for each sub-server, with the permissions of the
rest of the RPC infra. As a result, each sub-server is able to
independently specify what it needs w.r.t macaroon permissions and have
that taken care of by the RPC server. In order to achieve this, we need
to unify the creation of the RPC interceptors, and also fully manage the
gRPC server ourselves.

Some examples with various build configs:
```
⛰i  make build
 Building debug lnd and lncli.
go build -v -tags="dev" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd
go build -v -tags="dev" -o lncli-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd/cmd/lncli

⛰i  ./lnd-debug --debuglevel=debug --signrpc.signermacaroonpath=~/sign.macaroon
unknown flag `signrpc.signermacaroonpath'
unknown flag `signrpc.signermacaroonpath'

⛰i  make build tags=signerrpc
 Building debug lnd and lncli.
go build -v -tags="dev signerrpc" -o lnd-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd
go build -v -tags="dev signerrpc" -o lncli-debug -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty" github.com/lightningnetwork/lnd/cmd/lncli

⛰i  ./lnd-debug --debuglevel=debug --signrpc.signermacaroonpath=~/sign.macaroon
2018-10-22 17:31:01.132 [INF] LTND: Version: 0.5.0-beta commit=v0.5-beta-143-gb2069914c4b76109b7c59320dc48f8a5f30deb75-dirty, build=development, logging=default
2018-10-22 17:31:01.133 [INF] LTND: Active chain: Bitcoin (network=simnet)
2018-10-22 17:31:01.140 [INF] CHDB: Checking for schema update: latest_version=6, db_version=6
2018-10-22 17:31:01.236 [INF] LTND: Primary chain is set to: bitcoin
2018-10-22 17:31:02.391 [INF] LNWL: Opened wallet
2018-10-22 17:31:03.315 [INF] LNWL: The wallet has been unlocked without a time limit
2018-10-22 17:31:03.315 [INF] LTND: LightningWallet opened
2018-10-22 17:31:03.319 [INF] LNWL: Catching up block hashes to height 3060, this will take a while...
2018-10-22 17:31:03.320 [INF] HSWC: Restoring in-memory circuit state from disk
2018-10-22 17:31:03.320 [INF] LNWL: Done catching up block hashes
2018-10-22 17:31:03.320 [INF] HSWC: Payment circuits loaded: num_pending=0, num_open=0
2018-10-22 17:31:03.322 [DBG] LTND: Populating dependencies for sub RPC server: Signrpc
```

As for the config, an example is:
```
[signrpc]
signrpc.signermacaroonpath=~/signer.macaroon
```
2018-11-28 20:57:06 -08:00
Olaoluwa Osuntokun
4002caec69
config+rpc: create new generalized dynamic sub rpc server config framework
In this commit, we add the glue infrastructure to make the sub RPC
server system work properly. Our high level goal is the following: using
only the lnrpc package (with no visibility into the sub RPC servers),
the RPC server is able to find, create, run, and manage the entire set
of present and future sub RPC servers. In order to achieve this, we use
the reflect package and build tags heavily to permit a loosely coupled
configuration parsing system for the sub RPC servers.

We start with a new `subRpcServerConfigs` struct which is _always_
present.  This struct has its own group, and will house a series of
sub-configs, one for each sub RPC server. Each sub-config is actually
gated behind a build flag, and can be used to allow users on the command
line or in the config to specify arguments related to the sub-server. If
the config isn't present, then we don't attempt to parse it at all, if
it is, then that means the RPC server has been registered, and we should
parse the contents of its config.

The `subRpcServerConfigs` struct has two main methods:
`PopulateDependancies` and `FetchConfig`. The `PopulateDependancies` is
used to dynamically locate and set the config fields for each new
sub-server. As the config may not actually have any fields (if the build
flag is off), we use the reflect pacakge to determine if things are
compiled in or not, then if so, we dynamically set each of the config
parameters. The `PopulateDependancies` method implements the
`lnrpc.SubServerConfigDispatcher` interface. Our goal is to allow sub
servers to look up their actual config in this main config struct. We
achieve this by using reflect to look up the target field _as if it were
a key in a map_. If the field is found, then we check if it has any
actual attributes (it won't if the build flag is off), if it is, then we
return it as we expect it to be populated already.
2018-11-28 20:57:05 -08:00
Olaoluwa Osuntokun
16d16cf1b6
log: register logger for new Signer RPC Service, namespace SGNR 2018-11-28 20:57:04 -08:00
Olaoluwa Osuntokun
8971931aa3
lnrpc/signrpc: add lnrpc.SubServerDriver for signrpc
In this commit, we create a lnrpc.SubServerDriver for signrpc. Note that
this file will only have its init() method executed if the proper build
flag is on. As a result, only if the build flag is set, will the RPC
server be registered, and visible at the packge lnrpc level for the root
server to manipulate.
2018-11-28 20:57:03 -08:00
Olaoluwa Osuntokun
b7757683b2
lnrpc/signrpc: implement new SignerServer sub RPC server
In this commit, we add a full implementation of the new SignerServer sub
RPC service within the main root RPC service. This service is able to
fully manage its macaroons, and service any connected clients. Atm, this
service only has a single method: SignOutputRaw which mimics the
existing lnwallet.Signer interface within lnd itself. As the API's are
so similar, it will be possible for a client to directly use the
lnwallet.Signer interface, and have a proxy that sends the request over
RPC, and translates the proto layer on both sides. To the client, it
doesn't know that it's using a remote, or local RPC.
2018-11-28 20:57:01 -08:00
Olaoluwa Osuntokun
184f160fb9
lnrpc: add new sub RPC server registration system
In this commit, we add the scafolding for the future sub-server RPC
system. The idea is that each sub server will implement this particular
interface. From there on, a "root" RPC server is able to query this
registry, and dynamically create each  sub-sever instance without
knowing the details of each sub-server.

In the init() method of the pacakge of a sub-server, the sub-server is
to call: RegisterSubServer to claim its namespace. Afterwards, the root
RPC server can use the RegisteredSubServers() method to obtain a slice
of ALL regsitered sub-servers. Once this list is obtained, it can use
the New() method of the SubServerDriver struct to create a new
sub-server instance.

Each sub-server needs to be able to locate it's primary config using the
SubServerConfigDispatcher interface. This can be a map of maps, or a
regular config structr. The main requirement is that the sub-server be
able to find a config under the same name that it registered with. This
string of abstractions will allow the main RPC server to find, create,
and run each sub-server without knowing the details of its configuration
or its role.
2018-11-28 20:57:00 -08:00
Olaoluwa Osuntokun
9f24049bbc
lnrpc/signrpc: add new gen_protos.sh to generate minimal protos
In this commit, we add a new proto generation script to match the one in
the main lnrpc package. This script differs, as we don't need to
generate the REST proxy stuff (for now).
2018-11-28 20:56:59 -08:00
Olaoluwa Osuntokun
bbbdd7f6e9
lnrpc/signrpc: add and generate new set of protos for Signer service 2018-11-28 20:56:58 -08:00
Olaoluwa Osuntokun
a432f9a4c8 lnrpc/signrpc: create new signrpc package with build-flag guarded config
In this commit, we introduce a new sub-package within the greater RPC
package.  This new sub-package will house a new set of sub-RPC servers
to expose experimental features behind build flags for upstream
consumers. In this commit, we add the first config for the service,
which will simply expose the lnwallet.Signer interface over RPC.

In the default file, we have what the config will be if the build tag
(signerrpc) is off. In this case, the config parser won't detect any
times, and if specified will error out. In the active file, we have the
true config that the server will use. With this new set up, we'll
exploit these build flags heavily in order to create a generalized
framework for adding additional sub RPC servers.
2018-11-28 20:56:38 -08:00
Olaoluwa Osuntokun
b5dd1863f6
Merge pull request #2239 from Roasbeef/go-modules
build: switch from dep to go modules
2018-11-28 20:53:03 -08:00
Olaoluwa Osuntokun
eda45adb5b
docs: update docs to reflect new go modules usage and building 2018-11-28 20:13:56 -08:00
Olaoluwa Osuntokun
f65b1a4d6e
travis: force go modules support in $GOPATH, cache modules directory 2018-11-28 20:13:54 -08:00
Olaoluwa Osuntokun
e2e73379c7
build: update makefile to be aware of go modules, remove dep support
In this commit, we update the makefile to be aware of go modules. Along
the way, we remove all references to dep as we no longer use it within
this project. Note that in order to allow usage of go modules within the
$GOPATH directory, we set the `GO111MODULE=on` environment variable.
2018-11-28 20:13:52 -08:00
Olaoluwa Osuntokun
e5f4b7ca66
travis: remove dep directive from travis, as we use go modules now 2018-11-28 20:13:51 -08:00
Olaoluwa Osuntokun
e6c22d658d
docs: update contribution guidelines to reference latest version of Go
with '#' will be ignored, and an empty message aborts the commit.
2018-11-28 20:13:50 -08:00
Olaoluwa Osuntokun
f09a4daeb1
docs/MAKEFILE: remove obsolete travis section
In this commit, we remove the travis section as the directive has been
removed from the makefile.
2018-11-28 20:13:49 -08:00
Olaoluwa Osuntokun
ca28c0b5a1
docs/MAKEFILE: remove dep section
In this commit, we remove the dep section as we no longer use it, and
instead use go modues.
2018-11-28 20:13:48 -08:00
Olaoluwa Osuntokun
ebff57a1f7
build: remove old stale dep files 2018-11-28 20:13:46 -08:00
Olaoluwa Osuntokun
4a29f56a3f build: add new go.mod and go.sum files for go modules usage 2018-11-28 17:19:10 -08:00
Olaoluwa Osuntokun
4e83ce5c00
Merge pull request #1892 from halseth/travis-caching
travis cache
2018-11-28 17:07:40 -08:00
Olaoluwa Osuntokun
fd82200a15
Merge pull request #2140 from cfromknecht/wtserver-move
[watchtower/wtserver] rename server package, add godocs, general code health
2018-11-28 15:07:01 -08:00
Johan T. Halseth
b07499f227
Merge pull request #2224 from halseth/itest-timeouts
lnd_test: increase test timeouts
2018-11-28 08:22:46 +01:00
Johan T. Halseth
7e48b012e2
Merge pull request #2230 from halseth/itests-tryconnect-peer
lntest/harness: retry ConnectPeer of chain backend still syncing
2018-11-28 08:21:08 +01:00
Johan T. Halseth
0011bcdad1
Merge pull request #2229 from halseth/global-defaultcsv
lnd_test: make tests use global defaultCSV
2018-11-28 08:19:58 +01:00
Olaoluwa Osuntokun
6d4a769203
Merge pull request #2231 from karliatto/change-contribution-guide-link-#2214
Docs: code_contribution_guidelines change link
2018-11-27 15:48:27 -08:00
Johan T. Halseth
8854db5191
gitignore: ignore temp backend logs folder 2018-11-27 10:21:19 +01:00
Johan T. Halseth
4d05e6bbd8
lnd_test+lntest: keep btcd debug logs from test run
Useful for debugging 🤷
2018-11-27 10:21:19 +01:00
Johan T. Halseth
3a2fd51f2f
lnd_test+lntest: move copyFile() -> lntest.CopyFile() 2018-11-27 10:21:17 +01:00
Carlos Garcia Ortiz
0f03120a46 Docs: code_contribution_guidelines change link 2018-11-27 10:20:06 +01:00
Johan T. Halseth
b389d5f6b0
lntest/harness: retry ConnectPeer of chain backend still syncing 2018-11-27 10:15:00 +01:00
Johan T. Halseth
ec76a25530
lnd_test: make test use globale defaultCSV 2018-11-27 10:11:32 +01:00
Johan T. Halseth
62e37ec840
lnd_test: use global defaultTimeout instead of test specific timeouts 2018-11-27 10:01:02 +01:00
Johan T. Halseth
f13e5472d5
lnd_test: use global minerMempoolTimeout when waiting for miner to see TXs 2018-11-27 09:59:46 +01:00
Johan T. Halseth
ea32a13fdb
lnd_test: use global channelCloseTimeout when closing channels 2018-11-27 09:59:31 +01:00
Johan T. Halseth
864d77d82e
lnd_test: use global channelOpenTimeout when opening channels 2018-11-27 09:59:29 +01:00
Olaoluwa Osuntokun
0348db760e
Merge pull request #2221 from Roasbeef/open-channel-sync-fix
rpc: properly set output index in OpenChannelSync resp
2018-11-26 16:36:13 -08:00
ErikEk
6884fe40db Set private setting for invoice 2018-11-26 09:05:37 +01:00
Olaoluwa Osuntokun
a1c39af313
rpc: properly set output index in OpenChannelSync resp
Fixes #2219
2018-11-25 21:13:00 -06:00