1*387f9dfdSAndroid Build Coastguard WorkerDemonstrations of dbslower, the Linux eBPF/bcc version. 2*387f9dfdSAndroid Build Coastguard Worker 3*387f9dfdSAndroid Build Coastguard Worker 4*387f9dfdSAndroid Build Coastguard Workerdbslower traces queries served by a MySQL or PostgreSQL server, and prints 5*387f9dfdSAndroid Build Coastguard Workerthose that exceed a latency (query time) threshold. By default a threshold of 6*387f9dfdSAndroid Build Coastguard Worker1 ms is used. For example: 7*387f9dfdSAndroid Build Coastguard Worker 8*387f9dfdSAndroid Build Coastguard Worker# dbslower mysql 9*387f9dfdSAndroid Build Coastguard WorkerTracing database queries for pids 25776 slower than 1 ms... 10*387f9dfdSAndroid Build Coastguard WorkerTIME(s) PID MS QUERY 11*387f9dfdSAndroid Build Coastguard Worker1.315800 25776 2000.999 call getproduct(97) 12*387f9dfdSAndroid Build Coastguard Worker3.360380 25776 3.226 call getproduct(6) 13*387f9dfdSAndroid Build Coastguard Worker^C 14*387f9dfdSAndroid Build Coastguard Worker 15*387f9dfdSAndroid Build Coastguard WorkerThis traced two queries slower than 1ms, one of which is very slow: over 2 16*387f9dfdSAndroid Build Coastguard Workerseconds. We can filter out the shorter ones and keep only the really slow ones: 17*387f9dfdSAndroid Build Coastguard Worker 18*387f9dfdSAndroid Build Coastguard Worker# dbslower mysql -m 1000 19*387f9dfdSAndroid Build Coastguard WorkerTracing database queries for pids 25776 slower than 1000 ms... 20*387f9dfdSAndroid Build Coastguard WorkerTIME(s) PID MS QUERY 21*387f9dfdSAndroid Build Coastguard Worker1.421264 25776 2002.183 call getproduct(97) 22*387f9dfdSAndroid Build Coastguard Worker3.572617 25776 2001.381 call getproduct(97) 23*387f9dfdSAndroid Build Coastguard Worker5.661411 25776 2001.867 call getproduct(97) 24*387f9dfdSAndroid Build Coastguard Worker7.748296 25776 2001.329 call getproduct(97) 25*387f9dfdSAndroid Build Coastguard Worker^C 26*387f9dfdSAndroid Build Coastguard Worker 27*387f9dfdSAndroid Build Coastguard WorkerThis looks like a pattern -- we keep making this slow query every 2 seconds 28*387f9dfdSAndroid Build Coastguard Workeror so, and it takes approximately 2 seconds to run. 29*387f9dfdSAndroid Build Coastguard Worker 30*387f9dfdSAndroid Build Coastguard WorkerBy default, dbslower will try to detect mysqld and postgres processes, but if 31*387f9dfdSAndroid Build Coastguard Workernecessary, you can specify the process ids with the -p switch: 32*387f9dfdSAndroid Build Coastguard Worker 33*387f9dfdSAndroid Build Coastguard Worker# dbslower mysql -p $(pidof mysql) 34*387f9dfdSAndroid Build Coastguard WorkerTracing database queries for pids 25776 slower than 1 ms... 35*387f9dfdSAndroid Build Coastguard WorkerTIME(s) PID MS QUERY 36*387f9dfdSAndroid Build Coastguard Worker2.002125 25776 3.340 call getproduct(7) 37*387f9dfdSAndroid Build Coastguard Worker2.045006 25776 2001.558 call getproduct(97) 38*387f9dfdSAndroid Build Coastguard Worker4.131863 25776 2002.275 call getproduct(97) 39*387f9dfdSAndroid Build Coastguard Worker6.190513 25776 3.248 call getproduct(33) 40*387f9dfdSAndroid Build Coastguard Worker^C 41*387f9dfdSAndroid Build Coastguard Worker 42*387f9dfdSAndroid Build Coastguard WorkerSpecifying 0 as the threshold will print all the queries: 43*387f9dfdSAndroid Build Coastguard Worker 44*387f9dfdSAndroid Build Coastguard Worker# dbslower mysql -m 0 45*387f9dfdSAndroid Build Coastguard WorkerTracing database queries for pids 25776 slower than 0 ms... 46*387f9dfdSAndroid Build Coastguard WorkerTIME(s) PID MS QUERY 47*387f9dfdSAndroid Build Coastguard Worker6.003720 25776 2.363 /* mysql-connector-java-5.1.40 ( Revision: 402933ef52cad9aa82624e80acbea46e3a701ce6 ) */SELECT @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_conn 48*387f9dfdSAndroid Build Coastguard Worker6.599219 25776 0.068 SET NAMES latin1 49*387f9dfdSAndroid Build Coastguard Worker6.613944 25776 0.057 SET character_set_results = NULL 50*387f9dfdSAndroid Build Coastguard Worker6.645228 25776 0.059 SET autocommit=1 51*387f9dfdSAndroid Build Coastguard Worker6.653798 25776 0.059 SET sql_mode='NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES' 52*387f9dfdSAndroid Build Coastguard Worker6.682184 25776 2.526 select * from users where id = 0 53*387f9dfdSAndroid Build Coastguard Worker6.767888 25776 0.288 select id from products where userid = 0 54*387f9dfdSAndroid Build Coastguard Worker6.790642 25776 2.255 call getproduct(0) 55*387f9dfdSAndroid Build Coastguard Worker6.809865 25776 0.218 call getproduct(1) 56*387f9dfdSAndroid Build Coastguard Worker6.846878 25776 0.248 select * from users where id = 1 57*387f9dfdSAndroid Build Coastguard Worker6.847623 25776 0.166 select id from products where userid = 1 58*387f9dfdSAndroid Build Coastguard Worker6.867363 25776 0.244 call getproduct(2) 59*387f9dfdSAndroid Build Coastguard Worker6.868162 25776 0.107 call getproduct(3) 60*387f9dfdSAndroid Build Coastguard Worker6.874726 25776 0.208 select * from users where id = 2 61*387f9dfdSAndroid Build Coastguard Worker6.881722 25776 0.260 select id from products where userid = 2 62*387f9dfdSAndroid Build Coastguard Worker^C 63*387f9dfdSAndroid Build Coastguard Worker 64*387f9dfdSAndroid Build Coastguard WorkerHere we can see the MySQL connector initialization and connection establishment, 65*387f9dfdSAndroid Build Coastguard Workerbefore the actual queries start coming in. 66*387f9dfdSAndroid Build Coastguard Worker 67*387f9dfdSAndroid Build Coastguard Worker 68*387f9dfdSAndroid Build Coastguard WorkerUSAGE: 69*387f9dfdSAndroid Build Coastguard Worker# dbslower -h 70*387f9dfdSAndroid Build Coastguard Workerusage: dbslower.py [-h] [-v] [-p [PIDS [PIDS ...]]] [-x PATH] [-m THRESHOLD] 71*387f9dfdSAndroid Build Coastguard Worker {mysql,postgres} 72*387f9dfdSAndroid Build Coastguard Worker 73*387f9dfdSAndroid Build Coastguard Workerpositional arguments: 74*387f9dfdSAndroid Build Coastguard Worker {mysql,postgres} the database engine to use 75*387f9dfdSAndroid Build Coastguard Worker 76*387f9dfdSAndroid Build Coastguard Workeroptional arguments: 77*387f9dfdSAndroid Build Coastguard Worker -h, --help show this help message and exit 78*387f9dfdSAndroid Build Coastguard Worker -v, --verbose print the BPF program 79*387f9dfdSAndroid Build Coastguard Worker -p [PID [PID ...]], --pid [PID [PID ...]] 80*387f9dfdSAndroid Build Coastguard Worker the pid(s) to trace 81*387f9dfdSAndroid Build Coastguard Worker -x PATH, --exe PATH path to binary 82*387f9dfdSAndroid Build Coastguard Worker -m THRESHOLD, --threshold THRESHOLD 83*387f9dfdSAndroid Build Coastguard Worker trace queries slower than this threshold (ms) 84*387f9dfdSAndroid Build Coastguard Worker 85*387f9dfdSAndroid Build Coastguard Workerexamples: 86*387f9dfdSAndroid Build Coastguard Worker dbslower postgres # trace PostgreSQL queries slower than 1ms 87*387f9dfdSAndroid Build Coastguard Worker dbslower postgres -p 188 322 # trace specific PostgreSQL processes 88*387f9dfdSAndroid Build Coastguard Worker dbslower mysql -p 480 -m 30 # trace MySQL queries slower than 30ms 89*387f9dfdSAndroid Build Coastguard Worker dbslower mysql -p 480 -v # trace MySQL queries and print the BPF program 90*387f9dfdSAndroid Build Coastguard Worker dbslower mysql -x $(which mysqld) # trace MySQL queries with uprobes 91