1*387f9dfdSAndroid Build Coastguard WorkerDemonstrations of mdflush, the Linux eBPF/bcc version. 2*387f9dfdSAndroid Build Coastguard Worker 3*387f9dfdSAndroid Build Coastguard Worker 4*387f9dfdSAndroid Build Coastguard WorkerThe mdflush tool traces flushes at the md driver level, and prints details 5*387f9dfdSAndroid Build Coastguard Workerincluding the time of the flush: 6*387f9dfdSAndroid Build Coastguard Worker 7*387f9dfdSAndroid Build Coastguard Worker# ./mdflush 8*387f9dfdSAndroid Build Coastguard WorkerTracing md flush requests... Hit Ctrl-C to end. 9*387f9dfdSAndroid Build Coastguard WorkerTIME PID COMM DEVICE 10*387f9dfdSAndroid Build Coastguard Worker03:13:49 16770 sync md0 11*387f9dfdSAndroid Build Coastguard Worker03:14:08 16864 sync md0 12*387f9dfdSAndroid Build Coastguard Worker03:14:49 496 kworker/1:0H md0 13*387f9dfdSAndroid Build Coastguard Worker03:14:49 488 xfsaild/md0 md0 14*387f9dfdSAndroid Build Coastguard Worker03:14:54 488 xfsaild/md0 md0 15*387f9dfdSAndroid Build Coastguard Worker03:15:00 488 xfsaild/md0 md0 16*387f9dfdSAndroid Build Coastguard Worker03:15:02 85 kswapd0 md0 17*387f9dfdSAndroid Build Coastguard Worker03:15:02 488 xfsaild/md0 md0 18*387f9dfdSAndroid Build Coastguard Worker03:15:05 488 xfsaild/md0 md0 19*387f9dfdSAndroid Build Coastguard Worker03:15:08 488 xfsaild/md0 md0 20*387f9dfdSAndroid Build Coastguard Worker03:15:10 488 xfsaild/md0 md0 21*387f9dfdSAndroid Build Coastguard Worker03:15:11 488 xfsaild/md0 md0 22*387f9dfdSAndroid Build Coastguard Worker03:15:11 488 xfsaild/md0 md0 23*387f9dfdSAndroid Build Coastguard Worker03:15:11 488 xfsaild/md0 md0 24*387f9dfdSAndroid Build Coastguard Worker03:15:11 488 xfsaild/md0 md0 25*387f9dfdSAndroid Build Coastguard Worker03:15:11 488 xfsaild/md0 md0 26*387f9dfdSAndroid Build Coastguard Worker03:15:12 488 xfsaild/md0 md0 27*387f9dfdSAndroid Build Coastguard Worker03:15:13 488 xfsaild/md0 md0 28*387f9dfdSAndroid Build Coastguard Worker03:15:15 488 xfsaild/md0 md0 29*387f9dfdSAndroid Build Coastguard Worker03:15:19 496 kworker/1:0H md0 30*387f9dfdSAndroid Build Coastguard Worker03:15:49 496 kworker/1:0H md0 31*387f9dfdSAndroid Build Coastguard Worker03:15:55 18840 sync md0 32*387f9dfdSAndroid Build Coastguard Worker03:16:49 496 kworker/1:0H md0 33*387f9dfdSAndroid Build Coastguard Worker03:17:19 496 kworker/1:0H md0 34*387f9dfdSAndroid Build Coastguard Worker03:20:19 496 kworker/1:0H md0 35*387f9dfdSAndroid Build Coastguard Worker03:21:19 496 kworker/1:0H md0 36*387f9dfdSAndroid Build Coastguard Worker03:21:49 496 kworker/1:0H md0 37*387f9dfdSAndroid Build Coastguard Worker03:25:19 496 kworker/1:0H md0 38*387f9dfdSAndroid Build Coastguard Worker[...] 39*387f9dfdSAndroid Build Coastguard Worker 40*387f9dfdSAndroid Build Coastguard WorkerThis can be useful for correlation with latency outliers or spikes in disk 41*387f9dfdSAndroid Build Coastguard Workerlatency, as measured using another tool (eg, system monitoring). If spikes in 42*387f9dfdSAndroid Build Coastguard Workerdisk latency often coincide with md flush events, then it would make flushing 43*387f9dfdSAndroid Build Coastguard Workera target for tuning. 44*387f9dfdSAndroid Build Coastguard Worker 45*387f9dfdSAndroid Build Coastguard WorkerNote that the flush events are likely to originate from higher in the I/O 46*387f9dfdSAndroid Build Coastguard Workerstack, such as from file systems. This traces md processing them, and the 47*387f9dfdSAndroid Build Coastguard Workertimestamp corresponds with when md began to issue the flush to disks. 48