1# gRPC C++ Debugging Example 2 3This example demonstrates a handful of ways you can debug your gRPC C++ applications. 4 5## Enabling Trace Logs 6 7gRPC allows you to configure more detailed log output for various aspects of gRPC behavior. The tracing log generation might have a large overhead and result in significantly larger log file sizes, especially when you try to trace transport or timer_check. But it is a powerful tool in your debugging toolkit. 8 9### The Most Verbose Logging 10 11Specify environment variables, then run your application: 12 13``` 14GRPC_VERBOSITY=debug 15GRPC_TRACE=all 16``` 17 18For more granularity, please see 19[environment_variables](https://github.com/grpc/grpc/blob/master/doc/environment_variables.md). 20 21### Debug Transport Protocol 22 23``` 24GRPC_VERBOSITY=debug 25GRPC_TRACE=tcp,http,secure_endpoint,transport_security 26``` 27 28### Debug Connection Behavior 29 30``` 31GRPC_VERBOSITY=debug 32GRPC_TRACE=call_error,connectivity_state,pick_first,round_robin,glb 33``` 34 35## GDB and other debuggers 36 37`gdb` (and the like) are tools that lets you inspect your application while it is running, view stack traces on exceptions, pause and step through code at specified points or under certain conditions, etc. See https://www.sourceware.org/gdb/ 38 39### Inspecting errors 40 41``` 42bazel build --config=dbg examples/cpp/debugging:crashing_greeter_client 43gdb -ex run \ 44 --args ./bazel-bin/examples/cpp/debugging/crashing_greeter_client \ 45 --crash_on_errors=true \ 46 --target=localhork:50051 47``` 48 49Once the exception is thrown, you can use `bt` to see the stack trace and examine the crash, `info threads` to get the set of threads, etc. See the [GDB documentation](https://sourceware.org/gdb/current/onlinedocs/gdb.html/) for a more complete list of available features and commands. 50 51### Breaking inside a function 52 53After building the application above, this will break inside gRPC generated stub code: 54 55``` 56gdb -ex 'b helloworld::Greeter::Stub::SayHello' \ 57 -ex run \ 58 --args ./bazel-bin/examples/cpp/debugging/crashing_greeter_client \ 59 --crash_on_errors=true \ 60 --target=localhork:50051 61``` 62 63## gRPC Admin Interface: Live Channel Tracing 64 65The [gRPC Admin Service](https://github.com/grpc/proposal/blob/master/A38-admin-interface-api.md) 66provides a convenient API in each gRPC language to improve the usability of 67creating a gRPC server with admin services to expose states in the gRPC library. 68This includes channelz, which is a channel tracing feature; it tracks statistics 69like how many messages have been sent, how many of them failed, what are the 70connected sockets. See the [Channelz design doc](https://github.com/grpc/proposal/blob/master/A14-channelz.md). 71 72### Integrating the gRPC Admin Service Into Your Server 73 74As seen in the `greeter_callback_admin_server` target, you canenable admin services by using the `AddAdminServices` method. 75 76``` 77grpc::ServerBuilder builder; 78grpc::AddAdminServices(&builder); 79builder.AddListeningPort(":50051", grpc::ServerCredentials(...)); 80std::unique_ptr<grpc::Server> server(builder.BuildAndStart()); 81``` 82 83### Using grpcdebug 84 85grpcdebug is a tool created to access the metrics from channelz and health services. 86 87#### Installing the grpcdebug tool 88 89The source code is located in a github project 90[grpc-ecosystem/grpcdebug](https://github.com/grpc-ecosystem/grpcdebug). You 91can either download [the latest built version] 92(https://github.com/grpc-ecosystem/grpcdebug/releases/latest) (recommended) or 93follow the README.md to build it yourself. 94 95#### Running the grpcdebug tool 96##### Usage 97`grpcdebug <target address> [flags] channelz <command> [argument]` 98 99 100| Command | Argument | Description | 101| :--------- | :------------------: | :------------------------------------------------ | 102| channel | \<channel id or URL> | Display channel states in a human readable way. | 103| channels | | Lists client channels for the target application. | 104| server | \<server ID> | Displays server state in a human readable way. | 105| servers | | Lists servers in a human readable way. | 106| socket | \<socket ID> | Displays socket states in a human readable way. | 107| subchannel | \<id> | Display subchannel states in human readable way. | 108 109Generally, you will start with either `servers` or `channels` and then work down 110to the details 111 112##### Getting overall server info 113 114To begin with, build and run the server binary in the background 115 116``` 117bazel build --config=dbg examples/cpp/debugging:all 118./bazel-bin/examples/cpp/debugging/greeter_callback_server_admin& 119``` 120 121You can then inspect the server 122```bash 123grpcdebug localhost:50051 channelz servers 124``` 125 126This will show you the server ids with their activity 127```text 128Server ID Listen Addresses Calls(Started/Succeeded/Failed) Last Call Started 1291 [[::]:50051] 38/34/3 now 130``` 131 132For more information about `grpcdebug` features, please see [the grpcdebug documentation](https://github.com/grpc-ecosystem/grpcdebug) 133