1*387f9dfdSAndroid Build Coastguard WorkerDemonstrations of ppchcalls, the Linux/eBPF version. 2*387f9dfdSAndroid Build Coastguard Worker 3*387f9dfdSAndroid Build Coastguard Worker 4*387f9dfdSAndroid Build Coastguard Workerppchcalls summarizes hcall counts across the system or a specific process, 5*387f9dfdSAndroid Build Coastguard Workerwith optional latency information. It is very useful for general workload 6*387f9dfdSAndroid Build Coastguard Workercharacterization, for example: 7*387f9dfdSAndroid Build Coastguard Worker 8*387f9dfdSAndroid Build Coastguard Worker# ./ppchcalls.py 9*387f9dfdSAndroid Build Coastguard WorkerTracing ppc hcalls, printing top 10... Ctrl+C to quit. 10*387f9dfdSAndroid Build Coastguard Worker^C[04:59:47] 11*387f9dfdSAndroid Build Coastguard WorkerPPC HCALL COUNT 12*387f9dfdSAndroid Build Coastguard WorkerH_IPI 26 13*387f9dfdSAndroid Build Coastguard WorkerH_EOI 22 14*387f9dfdSAndroid Build Coastguard WorkerH_XIRR 22 15*387f9dfdSAndroid Build Coastguard WorkerH_VIO_SIGNAL 4 16*387f9dfdSAndroid Build Coastguard WorkerH_REMOVE 3 17*387f9dfdSAndroid Build Coastguard WorkerH_PUT_TCE 2 18*387f9dfdSAndroid Build Coastguard WorkerH_SEND_CRQ 2 19*387f9dfdSAndroid Build Coastguard WorkerH_STUFF_TCE 2 20*387f9dfdSAndroid Build Coastguard WorkerH_ENTER 1 21*387f9dfdSAndroid Build Coastguard WorkerH_PROTECT 1 22*387f9dfdSAndroid Build Coastguard Worker 23*387f9dfdSAndroid Build Coastguard WorkerDetaching... 24*387f9dfdSAndroid Build Coastguard Worker# 25*387f9dfdSAndroid Build Coastguard Worker 26*387f9dfdSAndroid Build Coastguard WorkerThese are the top 10 entries; you can get more by using the -T switch. Here, 27*387f9dfdSAndroid Build Coastguard Workerthe output indicates that the H_IPI, H_EOI and H_XIRR hcalls were very common, 28*387f9dfdSAndroid Build Coastguard Workerfollowed immediately by H_VIO_SIGNAL, H_REMOVE and so on. By default, ppchcalls 29*387f9dfdSAndroid Build Coastguard Workercounts across the entire system, but we can point it to a specific process of 30*387f9dfdSAndroid Build Coastguard Workerinterest: 31*387f9dfdSAndroid Build Coastguard Worker 32*387f9dfdSAndroid Build Coastguard Worker# ./ppchcalls.py -p $(pidof vim) 33*387f9dfdSAndroid Build Coastguard WorkerTracing ppc hcalls, printing top 10... Ctrl+C to quit. 34*387f9dfdSAndroid Build Coastguard Worker^C[06:23:12] 35*387f9dfdSAndroid Build Coastguard WorkerPPC HCALL COUNT 36*387f9dfdSAndroid Build Coastguard WorkerH_PUT_TERM_CHAR 62 37*387f9dfdSAndroid Build Coastguard WorkerH_ENTER 2 38*387f9dfdSAndroid Build Coastguard Worker 39*387f9dfdSAndroid Build Coastguard WorkerDetaching... 40*387f9dfdSAndroid Build Coastguard Worker# 41*387f9dfdSAndroid Build Coastguard Worker 42*387f9dfdSAndroid Build Coastguard Worker 43*387f9dfdSAndroid Build Coastguard WorkerOccasionally, the count of hcalls is not enough, and you'd also want to know 44*387f9dfdSAndroid Build Coastguard Workerthe minimum, maximum and aggregate latency for each of the hcalls: 45*387f9dfdSAndroid Build Coastguard Worker 46*387f9dfdSAndroid Build Coastguard Worker# ./ppchcalls.py -L 47*387f9dfdSAndroid Build Coastguard WorkerTracing ppc hcalls, printing top 10... Ctrl+C to quit. 48*387f9dfdSAndroid Build Coastguard Worker[00:53:59] 49*387f9dfdSAndroid Build Coastguard WorkerPPC HCALL COUNT MIN (us) MAX (us) AVG (us) 50*387f9dfdSAndroid Build Coastguard WorkerH_IPI 32 0.808 7.730 2.329 51*387f9dfdSAndroid Build Coastguard WorkerH_EOI 25 0.697 1.984 1.081 52*387f9dfdSAndroid Build Coastguard WorkerH_PUT_TERM_CHAR 25 10.315 47.184 14.667 53*387f9dfdSAndroid Build Coastguard WorkerH_XIRR 25 0.868 6.223 2.397 54*387f9dfdSAndroid Build Coastguard WorkerH_VIO_SIGNAL 6 1.418 22.053 7.507 55*387f9dfdSAndroid Build Coastguard WorkerH_STUFF_TCE 3 0.865 2.349 1.384 56*387f9dfdSAndroid Build Coastguard WorkerH_SEND_CRQ 3 18.015 21.137 19.673 57*387f9dfdSAndroid Build Coastguard WorkerH_REMOVE 3 1.838 7.407 3.735 58*387f9dfdSAndroid Build Coastguard WorkerH_PUT_TCE 3 1.473 4.808 2.698 59*387f9dfdSAndroid Build Coastguard WorkerH_GET_TERM_CHAR 2 8.379 26.729 17.554 60*387f9dfdSAndroid Build Coastguard Worker 61*387f9dfdSAndroid Build Coastguard WorkerDetaching... 62*387f9dfdSAndroid Build Coastguard Worker# 63*387f9dfdSAndroid Build Coastguard Worker 64*387f9dfdSAndroid Build Coastguard WorkerAnother direction would be to understand which processes are making a lot of 65*387f9dfdSAndroid Build Coastguard Workerhcalls, thus responsible for a lot of activity. This is what the -P switch 66*387f9dfdSAndroid Build Coastguard Workerdoes: 67*387f9dfdSAndroid Build Coastguard Worker 68*387f9dfdSAndroid Build Coastguard Worker# ./ppchcalls.py -P 69*387f9dfdSAndroid Build Coastguard WorkerTracing ppc hcalls, printing top 10... Ctrl+C to quit. 70*387f9dfdSAndroid Build Coastguard Worker^C[04:07:39] 71*387f9dfdSAndroid Build Coastguard WorkerPID COMM COUNT 72*387f9dfdSAndroid Build Coastguard Worker14118 top 1073 73*387f9dfdSAndroid Build Coastguard Worker0 [unknown] 286 74*387f9dfdSAndroid Build Coastguard Worker1679 bash 67 75*387f9dfdSAndroid Build Coastguard Worker14111 kworker/12:0-events_freezable_power_ 12 76*387f9dfdSAndroid Build Coastguard Worker2 kthreadd 4 77*387f9dfdSAndroid Build Coastguard Worker11753 kworker/0:0-events 4 78*387f9dfdSAndroid Build Coastguard Worker141 kworker/21:0H-xfs-log/dm-0 3 79*387f9dfdSAndroid Build Coastguard Worker847 systemd-udevd 3 80*387f9dfdSAndroid Build Coastguard Worker14116 ppchcalls.py 3 81*387f9dfdSAndroid Build Coastguard Worker13368 kworker/u64:1-events_unbound 3 82*387f9dfdSAndroid Build Coastguard Worker 83*387f9dfdSAndroid Build Coastguard WorkerDetaching... 84*387f9dfdSAndroid Build Coastguard Worker# 85*387f9dfdSAndroid Build Coastguard Worker 86*387f9dfdSAndroid Build Coastguard WorkerSometimes, you'd want both, the process making the most hcalls and respective 87*387f9dfdSAndroid Build Coastguard Workerprocess-wide latencies. All you need to do is combine both options: 88*387f9dfdSAndroid Build Coastguard Worker 89*387f9dfdSAndroid Build Coastguard Worker# ./ppchcalls.py -P -L 90*387f9dfdSAndroid Build Coastguard WorkerTracing ppc hcalls, printing top 10... Ctrl+C to quit. 91*387f9dfdSAndroid Build Coastguard Worker^C[04:35:27] 92*387f9dfdSAndroid Build Coastguard WorkerPID COMM COUNT MIN (us) MAX (us) AVG (us) 93*387f9dfdSAndroid Build Coastguard Worker0 [unknown] 69 0.666 13.059 2.834 94*387f9dfdSAndroid Build Coastguard Worker14151 kworker/12:1-events_freezable_power_ 8 6.489 84.470 34.354 95*387f9dfdSAndroid Build Coastguard Worker11753 kworker/0:0-events 4 1.415 2.059 1.784 96*387f9dfdSAndroid Build Coastguard Worker14152 kworker/u64:0-events_unbound 2 2.402 2.935 2.668 97*387f9dfdSAndroid Build Coastguard Worker14154 ppchcalls.py 2 3.139 11.934 7.537 98*387f9dfdSAndroid Build Coastguard Worker1751 sshd 1 7.227 7.227 7.227 99*387f9dfdSAndroid Build Coastguard Worker3413 kworker/6:2-mm_percpu_wq 1 6.775 6.775 6.775 100*387f9dfdSAndroid Build Coastguard Worker 101*387f9dfdSAndroid Build Coastguard WorkerDetaching... 102*387f9dfdSAndroid Build Coastguard Worker# 103*387f9dfdSAndroid Build Coastguard Worker 104*387f9dfdSAndroid Build Coastguard WorkerSometimes, you'd only care about a single hcall rather than all hcalls. 105*387f9dfdSAndroid Build Coastguard WorkerUse the --hcall option for this; the following example also demonstrates 106*387f9dfdSAndroid Build Coastguard Workerthe --hcall option, for printing at predefined intervals: 107*387f9dfdSAndroid Build Coastguard Worker 108*387f9dfdSAndroid Build Coastguard Worker# ./ppchcalls.py --hcall H_VIO_SIGNAL -i 5 109*387f9dfdSAndroid Build Coastguard Workerhcall H_VIO_SIGNAL , hcall_nr =260 110*387f9dfdSAndroid Build Coastguard WorkerTracing ppc hcall 'H_VIO_SIGNAL'... Ctrl+C to quit. 111*387f9dfdSAndroid Build Coastguard Worker[04:29:56] 112*387f9dfdSAndroid Build Coastguard WorkerPPC HCALL COUNT 113*387f9dfdSAndroid Build Coastguard WorkerH_VIO_SIGNAL 6 114*387f9dfdSAndroid Build Coastguard Worker 115*387f9dfdSAndroid Build Coastguard Worker[04:30:01] 116*387f9dfdSAndroid Build Coastguard WorkerPPC HCALL COUNT 117*387f9dfdSAndroid Build Coastguard WorkerH_VIO_SIGNAL 4 118*387f9dfdSAndroid Build Coastguard Worker 119*387f9dfdSAndroid Build Coastguard Worker[04:30:06] 120*387f9dfdSAndroid Build Coastguard WorkerPPC HCALL COUNT 121*387f9dfdSAndroid Build Coastguard WorkerH_VIO_SIGNAL 6 122*387f9dfdSAndroid Build Coastguard Worker 123*387f9dfdSAndroid Build Coastguard Worker[04:30:07] 124*387f9dfdSAndroid Build Coastguard WorkerPPC HCALL COUNT 125*387f9dfdSAndroid Build Coastguard Worker 126*387f9dfdSAndroid Build Coastguard WorkerDetaching... 127*387f9dfdSAndroid Build Coastguard Worker# 128*387f9dfdSAndroid Build Coastguard Worker 129*387f9dfdSAndroid Build Coastguard WorkerUSAGE: 130*387f9dfdSAndroid Build Coastguard Worker# ./ppchcalls.py -h 131*387f9dfdSAndroid Build Coastguard Workerusage: ppchcalls.py [-h] [-p PID] [-t TID] [-i INTERVAL] [-d DURATION] 132*387f9dfdSAndroid Build Coastguard Worker [-T TOP] [-x] [-e ERRNO] [-L] [-m] [-P] [-l] 133*387f9dfdSAndroid Build Coastguard Worker [--hcall HCALL] 134*387f9dfdSAndroid Build Coastguard Worker 135*387f9dfdSAndroid Build Coastguard WorkerSummarize ppc hcall counts and latencies. 136*387f9dfdSAndroid Build Coastguard Worker 137*387f9dfdSAndroid Build Coastguard Workeroptional arguments: 138*387f9dfdSAndroid Build Coastguard Worker -h, --help show this help message and exit 139*387f9dfdSAndroid Build Coastguard Worker -p PID, --pid PID trace only this pid 140*387f9dfdSAndroid Build Coastguard Worker -t TID, --tid TID trace only this tid 141*387f9dfdSAndroid Build Coastguard Worker -i INTERVAL, --interval INTERVAL 142*387f9dfdSAndroid Build Coastguard Worker print summary at this interval (seconds) 143*387f9dfdSAndroid Build Coastguard Worker -d DURATION, --duration DURATION 144*387f9dfdSAndroid Build Coastguard Worker total duration of trace, in seconds 145*387f9dfdSAndroid Build Coastguard Worker -T TOP, --top TOP print only the top hcalls by count or latency 146*387f9dfdSAndroid Build Coastguard Worker -x, --failures trace only failed hcalls (return < 0) 147*387f9dfdSAndroid Build Coastguard Worker -e ERRNO, --errno ERRNO 148*387f9dfdSAndroid Build Coastguard Worker trace only hcalls that return this error (numeric or 149*387f9dfdSAndroid Build Coastguard Worker EPERM, etc.) 150*387f9dfdSAndroid Build Coastguard Worker -L, --latency collect hcall latency 151*387f9dfdSAndroid Build Coastguard Worker -m, --milliseconds display latency in milliseconds (default: 152*387f9dfdSAndroid Build Coastguard Worker microseconds) 153*387f9dfdSAndroid Build Coastguard Worker -P, --process count by process and not by hcall 154*387f9dfdSAndroid Build Coastguard Worker -l, --list print list of recognized hcalls and exit 155*387f9dfdSAndroid Build Coastguard Worker --hcall HCALL trace this hcall only (use option -l to get all 156*387f9dfdSAndroid Build Coastguard Worker recognized hcalls) 157*387f9dfdSAndroid Build Coastguard Worker# 158*387f9dfdSAndroid Build Coastguard Worker 159*387f9dfdSAndroid Build Coastguard WorkerRef: https://docs.kernel.org/powerpc/papr_hcalls.html 160