1*de1e4e89SAndroid Build Coastguard Worker 2*de1e4e89SAndroid Build Coastguard WorkerThis documented is slightly dated but should give you idea of how things 3*de1e4e89SAndroid Build Coastguard Workerwork. 4*de1e4e89SAndroid Build Coastguard Worker 5*de1e4e89SAndroid Build Coastguard WorkerWhat is it? 6*de1e4e89SAndroid Build Coastguard Worker----------- 7*de1e4e89SAndroid Build Coastguard Worker 8*de1e4e89SAndroid Build Coastguard WorkerAn extension to the filtering/classification architecture of Linux Traffic 9*de1e4e89SAndroid Build Coastguard WorkerControl. 10*de1e4e89SAndroid Build Coastguard WorkerUp to 2.6.8 the only action that could be "attached" to a filter was policing. 11*de1e4e89SAndroid Build Coastguard Workeri.e you could say something like: 12*de1e4e89SAndroid Build Coastguard Worker 13*de1e4e89SAndroid Build Coastguard Worker----- 14*de1e4e89SAndroid Build Coastguard Workertc filter add dev lo parent ffff: protocol ip prio 10 u32 match ip src \ 15*de1e4e89SAndroid Build Coastguard Worker127.0.0.1/32 flowid 1:1 police mtu 4000 rate 1500kbit burst 90k 16*de1e4e89SAndroid Build Coastguard Worker----- 17*de1e4e89SAndroid Build Coastguard Worker 18*de1e4e89SAndroid Build Coastguard Workerwhich implies "if a packet is seen on the ingress of the lo device with 19*de1e4e89SAndroid Build Coastguard Workera source IP address of 127.0.0.1/32 we give it a classification id of 1:1 and 20*de1e4e89SAndroid Build Coastguard Workerwe execute a policing action which rate limits its bandwidth utilization 21*de1e4e89SAndroid Build Coastguard Workerto 1.5Mbps". 22*de1e4e89SAndroid Build Coastguard Worker 23*de1e4e89SAndroid Build Coastguard WorkerThe new extensions allow for more than just policing actions to be added. 24*de1e4e89SAndroid Build Coastguard WorkerThey are also fully backward compatible. If you have a kernel that doesnt 25*de1e4e89SAndroid Build Coastguard Workerunderstand them, then the effect is null i.e if you have a newer tc 26*de1e4e89SAndroid Build Coastguard Workerbut older kernel, the actions are not installed. Likewise if you 27*de1e4e89SAndroid Build Coastguard Workerhave a newer kernel but older tc, obviously the tc will use current 28*de1e4e89SAndroid Build Coastguard Workersyntax which will work fine. Of course to get the required effect you need 29*de1e4e89SAndroid Build Coastguard Workerboth newer tc and kernel. If you are reading this you have the 30*de1e4e89SAndroid Build Coastguard Workerright tc ;-> 31*de1e4e89SAndroid Build Coastguard Worker 32*de1e4e89SAndroid Build Coastguard WorkerA side effect is that we can now get stateless firewalling to work with tc. 33*de1e4e89SAndroid Build Coastguard WorkerEssentially this is now an alternative to iptables. 34*de1e4e89SAndroid Build Coastguard WorkerI wont go into details of my dislike for iptables at times, but 35*de1e4e89SAndroid Build Coastguard Workerscalability is one of the main issues; however, if you need stateful 36*de1e4e89SAndroid Build Coastguard Workerclassification - use netfilter (for now). 37*de1e4e89SAndroid Build Coastguard Worker 38*de1e4e89SAndroid Build Coastguard WorkerThis stuff works on both ingress and egress qdiscs. 39*de1e4e89SAndroid Build Coastguard Worker 40*de1e4e89SAndroid Build Coastguard WorkerFeatures 41*de1e4e89SAndroid Build Coastguard Worker-------- 42*de1e4e89SAndroid Build Coastguard Worker 43*de1e4e89SAndroid Build Coastguard Worker1) new additional syntax and actions enabled. Note old syntax is still valid. 44*de1e4e89SAndroid Build Coastguard Worker 45*de1e4e89SAndroid Build Coastguard WorkerEssentially this is still the same syntax as tc with a new construct 46*de1e4e89SAndroid Build Coastguard Worker"action". The syntax is of the form: 47*de1e4e89SAndroid Build Coastguard Workertc filter add <DEVICE> parent 1:0 protocol ip prio 10 <Filter description> 48*de1e4e89SAndroid Build Coastguard Workerflowid 1:1 action <ACTION description>* 49*de1e4e89SAndroid Build Coastguard Worker 50*de1e4e89SAndroid Build Coastguard WorkerYou can have as many actions as you want (within sensible reasoning). 51*de1e4e89SAndroid Build Coastguard Worker 52*de1e4e89SAndroid Build Coastguard WorkerIn the past the only real action was the policer; i.e you could do something 53*de1e4e89SAndroid Build Coastguard Workeralong the lines of: 54*de1e4e89SAndroid Build Coastguard Workertc filter add dev lo parent ffff: protocol ip prio 10 u32 \ 55*de1e4e89SAndroid Build Coastguard Workermatch ip src 127.0.0.1/32 flowid 1:1 \ 56*de1e4e89SAndroid Build Coastguard Workerpolice mtu 4000 rate 1500kbit burst 90k 57*de1e4e89SAndroid Build Coastguard Worker 58*de1e4e89SAndroid Build Coastguard WorkerAlthough you can still use the same syntax, now you can say: 59*de1e4e89SAndroid Build Coastguard Worker 60*de1e4e89SAndroid Build Coastguard Workertc filter add dev lo parent 1:0 protocol ip prio 10 u32 \ 61*de1e4e89SAndroid Build Coastguard Workermatch ip src 127.0.0.1/32 flowid 1:1 \ 62*de1e4e89SAndroid Build Coastguard Workeraction police mtu 4000 rate 1500kbit burst 90k 63*de1e4e89SAndroid Build Coastguard Worker 64*de1e4e89SAndroid Build Coastguard Worker" generic Actions" (gact) at the moment are: 65*de1e4e89SAndroid Build Coastguard Worker{ drop, pass, reclassify, continue} 66*de1e4e89SAndroid Build Coastguard Worker(If you have others, no listed here give me a reason and we will add them) 67*de1e4e89SAndroid Build Coastguard Worker+drop says to drop the packet 68*de1e4e89SAndroid Build Coastguard Worker+pass and ok (are equivalent) says to accept it 69*de1e4e89SAndroid Build Coastguard Worker+reclassify requests for reclassification of the packet 70*de1e4e89SAndroid Build Coastguard Worker+continue requests for next lookup to match 71*de1e4e89SAndroid Build Coastguard Worker 72*de1e4e89SAndroid Build Coastguard Worker2)In order to take advantage of some of the targets written by the 73*de1e4e89SAndroid Build Coastguard Workeriptables people, a classifier can have a packet being massaged by an 74*de1e4e89SAndroid Build Coastguard Workeriptable target. I have only tested with mangler targets up to now. 75*de1e4e89SAndroid Build Coastguard Worker(infact anything that is not in the mangling table is disabled right now) 76*de1e4e89SAndroid Build Coastguard Worker 77*de1e4e89SAndroid Build Coastguard WorkerIn terms of hooks: 78*de1e4e89SAndroid Build Coastguard Worker*ingress is mapped to pre-routing hook 79*de1e4e89SAndroid Build Coastguard Worker*egress is mapped to post-routing hook 80*de1e4e89SAndroid Build Coastguard WorkerI dont see much value in the other hooks, if you see it and email me good 81*de1e4e89SAndroid Build Coastguard Workerreasons, the addition is trivial. 82*de1e4e89SAndroid Build Coastguard Worker 83*de1e4e89SAndroid Build Coastguard WorkerExample syntax for iptables targets usage becomes: 84*de1e4e89SAndroid Build Coastguard Workertc filter add ..... u32 <u32 syntax> action ipt -j <iptables target syntax> 85*de1e4e89SAndroid Build Coastguard Worker 86*de1e4e89SAndroid Build Coastguard Workerexample: 87*de1e4e89SAndroid Build Coastguard Workertc filter add dev lo parent ffff: protocol ip prio 8 u32 \ 88*de1e4e89SAndroid Build Coastguard Workermatch ip dst 127.0.0.8/32 flowid 1:12 \ 89*de1e4e89SAndroid Build Coastguard Workeraction ipt -j mark --set-mark 2 90*de1e4e89SAndroid Build Coastguard Worker 91*de1e4e89SAndroid Build Coastguard WorkerNOTE: flowid 1:12 is parsed flowid 0x1:0x12. Make sure if you want flowid 92*de1e4e89SAndroid Build Coastguard Workerdecimal 12, then use flowid 1:c. 93*de1e4e89SAndroid Build Coastguard Worker 94*de1e4e89SAndroid Build Coastguard Worker3) A feature i call pipe 95*de1e4e89SAndroid Build Coastguard WorkerThe motivation is derived from Unix pipe mechanism but applied to packets. 96*de1e4e89SAndroid Build Coastguard WorkerEssentially take a matching packet and pass it through 97*de1e4e89SAndroid Build Coastguard Workeraction1 | action2 | action3 etc. 98*de1e4e89SAndroid Build Coastguard WorkerYou could do something similar to this with the tc policer and the "continue" 99*de1e4e89SAndroid Build Coastguard Workeroperator but this rather restricts it to just the policer and requires 100*de1e4e89SAndroid Build Coastguard Workermultiple rules (and lookups, hence quiet inefficient); 101*de1e4e89SAndroid Build Coastguard Worker 102*de1e4e89SAndroid Build Coastguard Workeras an example -- and please note that this is just an example _not_ The 103*de1e4e89SAndroid Build Coastguard WorkerWord Youve Been Waiting For (yes i have had problems giving examples 104*de1e4e89SAndroid Build Coastguard Workerwhich ended becoming dogma in documents and people modifying them a little 105*de1e4e89SAndroid Build Coastguard Workerto look clever); 106*de1e4e89SAndroid Build Coastguard Worker 107*de1e4e89SAndroid Build Coastguard Workeri selected the metering rates to be small so that i can show better how 108*de1e4e89SAndroid Build Coastguard Workerthings work. 109*de1e4e89SAndroid Build Coastguard Worker 110*de1e4e89SAndroid Build Coastguard WorkerThe script below does the following: 111*de1e4e89SAndroid Build Coastguard Worker- an incoming packet from 10.0.0.21 is first given a firewall mark of 1. 112*de1e4e89SAndroid Build Coastguard Worker 113*de1e4e89SAndroid Build Coastguard Worker- It is then metered to make sure it does not exceed its allocated rate of 114*de1e4e89SAndroid Build Coastguard Worker1Kbps. If it doesnt exceed rate, this is where we terminate action execution. 115*de1e4e89SAndroid Build Coastguard Worker 116*de1e4e89SAndroid Build Coastguard Worker- If it does exceed its rate, its "color" changes to a mark of 2 and it is 117*de1e4e89SAndroid Build Coastguard Workerthen passed through a second meter. 118*de1e4e89SAndroid Build Coastguard Worker 119*de1e4e89SAndroid Build Coastguard Worker-The second meter is shared across all flows on that device [i am suprised 120*de1e4e89SAndroid Build Coastguard Workerthat this seems to be not a well know feature of the policer; Bert was telling 121*de1e4e89SAndroid Build Coastguard Workerme that someone was writing a qdisc just to do sharing across multiple devices; 122*de1e4e89SAndroid Build Coastguard Workerit must be the summer heat again; weve had someone doing that every year around 123*de1e4e89SAndroid Build Coastguard Workersummer -- the key to sharing is to use a operator "index" in your policer 124*de1e4e89SAndroid Build Coastguard Workerrules (example "index 20"). All your rules have to use the same index to 125*de1e4e89SAndroid Build Coastguard Workershare.] 126*de1e4e89SAndroid Build Coastguard Worker 127*de1e4e89SAndroid Build Coastguard Worker-If the second meter is exceeded the color of the flow changes further to 3. 128*de1e4e89SAndroid Build Coastguard Worker 129*de1e4e89SAndroid Build Coastguard Worker-We then pass the packet to another meter which is shared across all devices 130*de1e4e89SAndroid Build Coastguard Workerin the system. If this meter is exceeded we drop the packet. 131*de1e4e89SAndroid Build Coastguard Worker 132*de1e4e89SAndroid Build Coastguard WorkerNote the mark can be used further up the system to do things like policy 133*de1e4e89SAndroid Build Coastguard Workeror more interesting things on the egress. 134*de1e4e89SAndroid Build Coastguard Worker 135*de1e4e89SAndroid Build Coastguard Worker------------------ cut here ------------------------------- 136*de1e4e89SAndroid Build Coastguard Worker# 137*de1e4e89SAndroid Build Coastguard Worker# Add an ingress qdisc on eth0 138*de1e4e89SAndroid Build Coastguard Workertc qdisc add dev eth0 ingress 139*de1e4e89SAndroid Build Coastguard Worker# 140*de1e4e89SAndroid Build Coastguard Worker#if you see an incoming packet from 10.0.0.21 141*de1e4e89SAndroid Build Coastguard Workertc filter add dev eth0 parent ffff: protocol ip prio 1 \ 142*de1e4e89SAndroid Build Coastguard Workeru32 match ip src 10.0.0.21/32 flowid 1:15 \ 143*de1e4e89SAndroid Build Coastguard Worker# 144*de1e4e89SAndroid Build Coastguard Worker# first give it a mark of 1 145*de1e4e89SAndroid Build Coastguard Workeraction ipt -j mark --set-mark 1 index 2 \ 146*de1e4e89SAndroid Build Coastguard Worker# 147*de1e4e89SAndroid Build Coastguard Worker# then pass it through a policer which allows 1kbps; if the flow 148*de1e4e89SAndroid Build Coastguard Worker# doesnt exceed that rate, this is where we stop, if it exceeds we 149*de1e4e89SAndroid Build Coastguard Worker# pipe the packet to the next action 150*de1e4e89SAndroid Build Coastguard Workeraction police rate 1kbit burst 9k pipe \ 151*de1e4e89SAndroid Build Coastguard Worker# 152*de1e4e89SAndroid Build Coastguard Worker# which marks the packet fwmark as 2 and pipes 153*de1e4e89SAndroid Build Coastguard Workeraction ipt -j mark --set-mark 2 \ 154*de1e4e89SAndroid Build Coastguard Worker# 155*de1e4e89SAndroid Build Coastguard Worker# next attempt to borrow b/width from a meter 156*de1e4e89SAndroid Build Coastguard Worker# used across all flows incoming on eth0("index 30") 157*de1e4e89SAndroid Build Coastguard Worker# and if that is exceeded we pipe to the next action 158*de1e4e89SAndroid Build Coastguard Workeraction police index 30 mtu 5000 rate 1kbit burst 10k pipe \ 159*de1e4e89SAndroid Build Coastguard Worker# mark it as fwmark 3 if exceeded 160*de1e4e89SAndroid Build Coastguard Workeraction ipt -j mark --set-mark 3 \ 161*de1e4e89SAndroid Build Coastguard Worker# and then attempt to borrow from a meter used by all devices in the 162*de1e4e89SAndroid Build Coastguard Worker# system. Should this be exceeded, drop the packet on the floor. 163*de1e4e89SAndroid Build Coastguard Workeraction police index 20 mtu 5000 rate 1kbit burst 90k drop 164*de1e4e89SAndroid Build Coastguard Worker--------------------------------- 165*de1e4e89SAndroid Build Coastguard Worker 166*de1e4e89SAndroid Build Coastguard WorkerNow lets see the actions installed with 167*de1e4e89SAndroid Build Coastguard Worker"tc filter show parent ffff: dev eth0" 168*de1e4e89SAndroid Build Coastguard Worker 169*de1e4e89SAndroid Build Coastguard Worker-------- output ----------- 170*de1e4e89SAndroid Build Coastguard Workerjroot# tc filter show parent ffff: dev eth0 171*de1e4e89SAndroid Build Coastguard Workerfilter protocol ip pref 1 u32 172*de1e4e89SAndroid Build Coastguard Workerfilter protocol ip pref 1 u32 fh 800: ht divisor 1 173*de1e4e89SAndroid Build Coastguard Workerfilter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:15 174*de1e4e89SAndroid Build Coastguard Worker 175*de1e4e89SAndroid Build Coastguard Worker action order 1: tablename: mangle hook: NF_IP_PRE_ROUTING 176*de1e4e89SAndroid Build Coastguard Worker target MARK set 0x1 index 2 177*de1e4e89SAndroid Build Coastguard Worker 178*de1e4e89SAndroid Build Coastguard Worker action order 2: police 1 action pipe rate 1Kbit burst 9Kb mtu 2Kb 179*de1e4e89SAndroid Build Coastguard Worker 180*de1e4e89SAndroid Build Coastguard Worker action order 3: tablename: mangle hook: NF_IP_PRE_ROUTING 181*de1e4e89SAndroid Build Coastguard Worker target MARK set 0x2 index 1 182*de1e4e89SAndroid Build Coastguard Worker 183*de1e4e89SAndroid Build Coastguard Worker action order 4: police 30 action pipe rate 1Kbit burst 10Kb mtu 5000b 184*de1e4e89SAndroid Build Coastguard Worker 185*de1e4e89SAndroid Build Coastguard Worker action order 5: tablename: mangle hook: NF_IP_PRE_ROUTING 186*de1e4e89SAndroid Build Coastguard Worker target MARK set 0x3 index 3 187*de1e4e89SAndroid Build Coastguard Worker 188*de1e4e89SAndroid Build Coastguard Worker action order 6: police 20 action drop rate 1Kbit burst 90Kb mtu 5000b 189*de1e4e89SAndroid Build Coastguard Worker 190*de1e4e89SAndroid Build Coastguard Worker match 0a000015/ffffffff at 12 191*de1e4e89SAndroid Build Coastguard Worker------------------------------- 192*de1e4e89SAndroid Build Coastguard Worker 193*de1e4e89SAndroid Build Coastguard WorkerNote the ordering of the actions is based on the order in which we entered 194*de1e4e89SAndroid Build Coastguard Workerthem. In the future i will add explicit priorities. 195*de1e4e89SAndroid Build Coastguard Worker 196*de1e4e89SAndroid Build Coastguard WorkerNow lets run a ping -f from 10.0.0.21 to this host; stop the ping after 197*de1e4e89SAndroid Build Coastguard Workeryou see a few lines of dots 198*de1e4e89SAndroid Build Coastguard Worker 199*de1e4e89SAndroid Build Coastguard Worker---- 200*de1e4e89SAndroid Build Coastguard Worker[root@jzny hadi]# ping -f 10.0.0.22 201*de1e4e89SAndroid Build Coastguard WorkerPING 10.0.0.22 (10.0.0.22): 56 data bytes 202*de1e4e89SAndroid Build Coastguard Worker.................................................................................................................................................................................................................................................................................................................................................................................................................................................... 203*de1e4e89SAndroid Build Coastguard Worker--- 10.0.0.22 ping statistics --- 204*de1e4e89SAndroid Build Coastguard Worker2248 packets transmitted, 1811 packets received, 19% packet loss 205*de1e4e89SAndroid Build Coastguard Workerround-trip min/avg/max = 0.7/9.3/20.1 ms 206*de1e4e89SAndroid Build Coastguard Worker----------------------------- 207*de1e4e89SAndroid Build Coastguard Worker 208*de1e4e89SAndroid Build Coastguard WorkerNow lets take a look at the stats with "tc -s filter show parent ffff: dev eth0" 209*de1e4e89SAndroid Build Coastguard Worker 210*de1e4e89SAndroid Build Coastguard Worker-------------- 211*de1e4e89SAndroid Build Coastguard Workerjroot# tc -s filter show parent ffff: dev eth0 212*de1e4e89SAndroid Build Coastguard Workerfilter protocol ip pref 1 u32 213*de1e4e89SAndroid Build Coastguard Workerfilter protocol ip pref 1 u32 fh 800: ht divisor 1 214*de1e4e89SAndroid Build Coastguard Workerfilter protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:1 215*de1e4e89SAndroid Build Coastguard Worker5 216*de1e4e89SAndroid Build Coastguard Worker 217*de1e4e89SAndroid Build Coastguard Worker action order 1: tablename: mangle hook: NF_IP_PRE_ROUTING 218*de1e4e89SAndroid Build Coastguard Worker target MARK set 0x1 index 2 219*de1e4e89SAndroid Build Coastguard Worker Sent 188832 bytes 2248 pkts (dropped 0, overlimits 0) 220*de1e4e89SAndroid Build Coastguard Worker 221*de1e4e89SAndroid Build Coastguard Worker action order 2: police 1 action pipe rate 1Kbit burst 9Kb mtu 2Kb 222*de1e4e89SAndroid Build Coastguard Worker Sent 188832 bytes 2248 pkts (dropped 0, overlimits 2122) 223*de1e4e89SAndroid Build Coastguard Worker 224*de1e4e89SAndroid Build Coastguard Worker action order 3: tablename: mangle hook: NF_IP_PRE_ROUTING 225*de1e4e89SAndroid Build Coastguard Worker target MARK set 0x2 index 1 226*de1e4e89SAndroid Build Coastguard Worker Sent 178248 bytes 2122 pkts (dropped 0, overlimits 0) 227*de1e4e89SAndroid Build Coastguard Worker 228*de1e4e89SAndroid Build Coastguard Worker action order 4: police 30 action pipe rate 1Kbit burst 10Kb mtu 5000b 229*de1e4e89SAndroid Build Coastguard Worker Sent 178248 bytes 2122 pkts (dropped 0, overlimits 1945) 230*de1e4e89SAndroid Build Coastguard Worker 231*de1e4e89SAndroid Build Coastguard Worker action order 5: tablename: mangle hook: NF_IP_PRE_ROUTING 232*de1e4e89SAndroid Build Coastguard Worker target MARK set 0x3 index 3 233*de1e4e89SAndroid Build Coastguard Worker Sent 163380 bytes 1945 pkts (dropped 0, overlimits 0) 234*de1e4e89SAndroid Build Coastguard Worker 235*de1e4e89SAndroid Build Coastguard Worker action order 6: police 20 action drop rate 1Kbit burst 90Kb mtu 5000b 236*de1e4e89SAndroid Build Coastguard Worker Sent 163380 bytes 1945 pkts (dropped 0, overlimits 437) 237*de1e4e89SAndroid Build Coastguard Worker 238*de1e4e89SAndroid Build Coastguard Worker match 0a000015/ffffffff at 12 239*de1e4e89SAndroid Build Coastguard Worker------------------------------- 240*de1e4e89SAndroid Build Coastguard Worker 241*de1e4e89SAndroid Build Coastguard WorkerNeat, eh? 242*de1e4e89SAndroid Build Coastguard Worker 243*de1e4e89SAndroid Build Coastguard Worker 244*de1e4e89SAndroid Build Coastguard WorkerWanna write an action module? 245*de1e4e89SAndroid Build Coastguard Worker------------------------------ 246*de1e4e89SAndroid Build Coastguard WorkerIts easy. Either look at the code or send me email. I will document at 247*de1e4e89SAndroid Build Coastguard Workersome point; will also accept documentation. 248*de1e4e89SAndroid Build Coastguard Worker 249*de1e4e89SAndroid Build Coastguard WorkerTODO 250*de1e4e89SAndroid Build Coastguard Worker---- 251*de1e4e89SAndroid Build Coastguard Worker 252*de1e4e89SAndroid Build Coastguard WorkerLotsa goodies/features coming. Requests also being accepted. 253*de1e4e89SAndroid Build Coastguard WorkerAt the moment the focus has been on getting the architecture in place. 254*de1e4e89SAndroid Build Coastguard WorkerExpect new things in the spurious time i have to work on this 255*de1e4e89SAndroid Build Coastguard Worker(particularly around end of year when i have typically get time off 256*de1e4e89SAndroid Build Coastguard Workerfrom work). 257*de1e4e89SAndroid Build Coastguard Worker 258