1*387f9dfdSAndroid Build Coastguard WorkerDemonstrations of tplist. 2*387f9dfdSAndroid Build Coastguard Worker 3*387f9dfdSAndroid Build Coastguard Worker 4*387f9dfdSAndroid Build Coastguard Workertplist displays kernel tracepoints and USDT probes, including their 5*387f9dfdSAndroid Build Coastguard Workerformat. It can be used to discover probe points for use with the trace 6*387f9dfdSAndroid Build Coastguard Workerand argdist tools. Kernel tracepoints are scattered around the kernel 7*387f9dfdSAndroid Build Coastguard Workerand provide valuable static tracing on block and network I/O, scheduling, 8*387f9dfdSAndroid Build Coastguard Workerpower events, and many other subjects. USDT probes are placed in libraries 9*387f9dfdSAndroid Build Coastguard Worker(such as libc) and executables (such as node) and provide static tracing 10*387f9dfdSAndroid Build Coastguard Workerinformation that can (optionally) be turned on and off at runtime. 11*387f9dfdSAndroid Build Coastguard Worker 12*387f9dfdSAndroid Build Coastguard WorkerFor example, suppose you want to discover which USDT probes a particular 13*387f9dfdSAndroid Build Coastguard Workerexecutable contains. Just run tplist on that executable (or library): 14*387f9dfdSAndroid Build Coastguard Worker 15*387f9dfdSAndroid Build Coastguard Worker$ tplist -l basic_usdt 16*387f9dfdSAndroid Build Coastguard Worker/home/vagrant/basic_usdt basic_usdt:start_main 17*387f9dfdSAndroid Build Coastguard Worker/home/vagrant/basic_usdt basic_usdt:loop_iter 18*387f9dfdSAndroid Build Coastguard Worker/home/vagrant/basic_usdt basic_usdt:end_main 19*387f9dfdSAndroid Build Coastguard Worker 20*387f9dfdSAndroid Build Coastguard WorkerThe loop_iter probe sounds interesting. How many arguments are available? 21*387f9dfdSAndroid Build Coastguard Worker 22*387f9dfdSAndroid Build Coastguard Worker$ tplist '*loop_iter' -l basic_usdt -v 23*387f9dfdSAndroid Build Coastguard Workerbasic_usdt:loop_iter [sema 0x601036] 24*387f9dfdSAndroid Build Coastguard Worker 2 location(s) 25*387f9dfdSAndroid Build Coastguard Worker 2 argument(s) 26*387f9dfdSAndroid Build Coastguard Worker 27*387f9dfdSAndroid Build Coastguard WorkerThis output indicates that the loop_iter probe is used in two locations 28*387f9dfdSAndroid Build Coastguard Workerin the basic_usdt executable, and that it has two arguments. Fortunately, 29*387f9dfdSAndroid Build Coastguard Workerthe argdist and trace tools understand the probe format and can print out 30*387f9dfdSAndroid Build Coastguard Workerthe arguments automatically -- you can refer to them as arg1, arg2, and 31*387f9dfdSAndroid Build Coastguard Workerso on. 32*387f9dfdSAndroid Build Coastguard Worker 33*387f9dfdSAndroid Build Coastguard WorkerTry to explore with some common libraries on your system and see if they 34*387f9dfdSAndroid Build Coastguard Workercontain UDST probes. Here are two examples you might find interesting: 35*387f9dfdSAndroid Build Coastguard Worker 36*387f9dfdSAndroid Build Coastguard Worker$ tplist -l pthread # list probes in libpthread 37*387f9dfdSAndroid Build Coastguard Worker/lib64/libpthread.so.0 libpthread:pthread_start 38*387f9dfdSAndroid Build Coastguard Worker/lib64/libpthread.so.0 libpthread:pthread_create 39*387f9dfdSAndroid Build Coastguard Worker/lib64/libpthread.so.0 libpthread:pthread_join 40*387f9dfdSAndroid Build Coastguard Worker/lib64/libpthread.so.0 libpthread:pthread_join_ret 41*387f9dfdSAndroid Build Coastguard Worker/lib64/libpthread.so.0 libpthread:mutex_init 42*387f9dfdSAndroid Build Coastguard Worker... more output truncated 43*387f9dfdSAndroid Build Coastguard Worker 44*387f9dfdSAndroid Build Coastguard Worker$ tplist -l c # list probes in libc 45*387f9dfdSAndroid Build Coastguard Worker/lib64/libc.so.6 libc:setjmp 46*387f9dfdSAndroid Build Coastguard Worker/lib64/libc.so.6 libc:longjmp 47*387f9dfdSAndroid Build Coastguard Worker/lib64/libc.so.6 libc:longjmp_target 48*387f9dfdSAndroid Build Coastguard Worker/lib64/libc.so.6 libc:memory_arena_reuse_free_list 49*387f9dfdSAndroid Build Coastguard Worker/lib64/libc.so.6 libc:memory_heap_new 50*387f9dfdSAndroid Build Coastguard Worker... more output truncated 51*387f9dfdSAndroid Build Coastguard Worker 52*387f9dfdSAndroid Build Coastguard Workertplist also understands kernel tracepoints, and can list their format 53*387f9dfdSAndroid Build Coastguard Workeras well. For example, let's look for all block I/O-related tracepoints: 54*387f9dfdSAndroid Build Coastguard Worker 55*387f9dfdSAndroid Build Coastguard Worker# tplist 'block*' 56*387f9dfdSAndroid Build Coastguard Workerblock:block_touch_buffer 57*387f9dfdSAndroid Build Coastguard Workerblock:block_dirty_buffer 58*387f9dfdSAndroid Build Coastguard Workerblock:block_rq_abort 59*387f9dfdSAndroid Build Coastguard Workerblock:block_rq_requeue 60*387f9dfdSAndroid Build Coastguard Workerblock:block_rq_complete 61*387f9dfdSAndroid Build Coastguard Workerblock:block_rq_insert 62*387f9dfdSAndroid Build Coastguard Workerblock:block_rq_issue 63*387f9dfdSAndroid Build Coastguard Workerblock:block_bio_bounce 64*387f9dfdSAndroid Build Coastguard Workerblock:block_bio_complete 65*387f9dfdSAndroid Build Coastguard Workerblock:block_bio_backmerge 66*387f9dfdSAndroid Build Coastguard Workerblock:block_bio_frontmerge 67*387f9dfdSAndroid Build Coastguard Workerblock:block_bio_queue 68*387f9dfdSAndroid Build Coastguard Workerblock:block_getrq 69*387f9dfdSAndroid Build Coastguard Workerblock:block_sleeprq 70*387f9dfdSAndroid Build Coastguard Workerblock:block_plug 71*387f9dfdSAndroid Build Coastguard Workerblock:block_unplug 72*387f9dfdSAndroid Build Coastguard Workerblock:block_split 73*387f9dfdSAndroid Build Coastguard Workerblock:block_bio_remap 74*387f9dfdSAndroid Build Coastguard Workerblock:block_rq_remap 75*387f9dfdSAndroid Build Coastguard Worker 76*387f9dfdSAndroid Build Coastguard WorkerThe block:block_rq_complete tracepoints sounds interesting. Let's print 77*387f9dfdSAndroid Build Coastguard Workerits format to see what we can trace with argdist and trace: 78*387f9dfdSAndroid Build Coastguard Worker 79*387f9dfdSAndroid Build Coastguard Worker$ tplist -v block:block_rq_complete 80*387f9dfdSAndroid Build Coastguard Workerblock:block_rq_complete 81*387f9dfdSAndroid Build Coastguard Worker dev_t dev; 82*387f9dfdSAndroid Build Coastguard Worker sector_t sector; 83*387f9dfdSAndroid Build Coastguard Worker unsigned int nr_sector; 84*387f9dfdSAndroid Build Coastguard Worker int errors; 85*387f9dfdSAndroid Build Coastguard Worker char rwbs[8]; 86*387f9dfdSAndroid Build Coastguard Worker 87*387f9dfdSAndroid Build Coastguard WorkerThe dev, sector, nr_sector, etc. variables can now all be used in probes 88*387f9dfdSAndroid Build Coastguard Workeryou specify with argdist or trace. 89*387f9dfdSAndroid Build Coastguard Worker 90*387f9dfdSAndroid Build Coastguard Worker 91*387f9dfdSAndroid Build Coastguard WorkerFor debugging USDT probes, it is sometimes useful to see the exact locations 92*387f9dfdSAndroid Build Coastguard Workerand arguments of the probes, including the registers or global variables from 93*387f9dfdSAndroid Build Coastguard Workerwhich their values are coming from. In super-verbose mode, tplist will print 94*387f9dfdSAndroid Build Coastguard Workerthis information (note the -vv): 95*387f9dfdSAndroid Build Coastguard Worker 96*387f9dfdSAndroid Build Coastguard Worker$ tplist -vv -l c *alloc* 97*387f9dfdSAndroid Build Coastguard Workerlibc:memory_malloc_retry [sema 0x0] 98*387f9dfdSAndroid Build Coastguard Worker location #0 /lib64/libc.so.6 0x835c0 99*387f9dfdSAndroid Build Coastguard Worker argument #0 8 unsigned bytes @ bp 100*387f9dfdSAndroid Build Coastguard Worker location #1 /lib64/libc.so.6 0x83778 101*387f9dfdSAndroid Build Coastguard Worker argument #0 8 unsigned bytes @ bp 102*387f9dfdSAndroid Build Coastguard Worker location #2 /lib64/libc.so.6 0x85a50 103*387f9dfdSAndroid Build Coastguard Worker argument #0 8 unsigned bytes @ bp 104*387f9dfdSAndroid Build Coastguard Workerlibc:memory_realloc_retry [sema 0x0] 105*387f9dfdSAndroid Build Coastguard Worker location #0 /lib64/libc.so.6 0x84b90 106*387f9dfdSAndroid Build Coastguard Worker argument #0 8 unsigned bytes @ r13 107*387f9dfdSAndroid Build Coastguard Worker argument #1 8 unsigned bytes @ bp 108*387f9dfdSAndroid Build Coastguard Worker location #1 /lib64/libc.so.6 0x85cf0 109*387f9dfdSAndroid Build Coastguard Worker argument #0 8 unsigned bytes @ r13 110*387f9dfdSAndroid Build Coastguard Worker argument #1 8 unsigned bytes @ bp 111*387f9dfdSAndroid Build Coastguard Workerlibc:memory_calloc_retry [sema 0x0] 112*387f9dfdSAndroid Build Coastguard Worker location #0 /lib64/libc.so.6 0x850f0 113*387f9dfdSAndroid Build Coastguard Worker argument #0 8 unsigned bytes @ bp 114*387f9dfdSAndroid Build Coastguard Worker 115*387f9dfdSAndroid Build Coastguard Worker 116*387f9dfdSAndroid Build Coastguard WorkerUSAGE message: 117*387f9dfdSAndroid Build Coastguard Worker 118*387f9dfdSAndroid Build Coastguard Worker$ tplist -h 119*387f9dfdSAndroid Build Coastguard Workerusage: tplist.py [-h] [-p PID] [-l LIB] [-v] [filter] 120*387f9dfdSAndroid Build Coastguard Worker 121*387f9dfdSAndroid Build Coastguard WorkerDisplay kernel tracepoints or USDT probes and their formats. 122*387f9dfdSAndroid Build Coastguard Worker 123*387f9dfdSAndroid Build Coastguard Workerpositional arguments: 124*387f9dfdSAndroid Build Coastguard Worker filter A filter that specifies which probes/tracepoints to print 125*387f9dfdSAndroid Build Coastguard Worker 126*387f9dfdSAndroid Build Coastguard Workeroptional arguments: 127*387f9dfdSAndroid Build Coastguard Worker -h, --help show this help message and exit 128*387f9dfdSAndroid Build Coastguard Worker -p PID, --pid PID List USDT probes in the specified process 129*387f9dfdSAndroid Build Coastguard Worker -l LIB, --lib LIB List USDT probes in the specified library or executable 130*387f9dfdSAndroid Build Coastguard Worker -v Increase verbosity level (print variables, arguments, etc.) 131*387f9dfdSAndroid Build Coastguard Worker 132