gmazza
Joined 10 years ago
Joined 10 years ago
Big network gear vendors that ship to ISPs have adopted early forms of AQM (both RED [3] and proprietary algorithms) quite a while ago - they had to:
(a) backbone routers have much smaller buffer-to-bandwidth ratio (compared to a PC or even home router), so endless buffering is not an option;
(b) buffer tail drops (i.e. what drop keyword does when added to tc filters/disciplines) interact really poorly with TCP bandwidth control algorithms, and rather badly with RTP streams, too (so drop is not an option either - it ruins user's connections; and
(c) ISPs and carriers would typically run links (on average) much closer to saturation than your typical home router, so situation where router has to make buffer/drop decision is much, much more frequent.
[1] My point above was that you don't really want to drop packets on the receiver side, after that packet already traversed expensive part of the network.
Yep. TC's default is to policy outgoing traffic, which in OP's example is a bunch of TCP ACKs essentially. Instead, they should be using ingress keyword, something like described here:
http://blog.stevedoria.net/20050906/ingress-policing-with-li...
Caveat emptor: ingress rate-limiting is hard. Long story short, it all boils down to what you do with non-confirming packets: There are two alternatives, and both are rather sub-optimal. You can either buffer/delay packets in kernel space (default, which leads to bufferbload and memory waste), or drop (which author linked above opted for, which leads to excessive retransmits and bandwidth waste).
# htb
tc qdisc add dev eth0 handle 1: root htb default 11
tc class add dev eth0 parent 1: classid 1:1 htb rate 1kbps
tc class add dev eth0 parent 1:1 classid 1:11 htb rate 1kbps
vs. # tbf
tc qdisc add dev eth0 root tbf rate 1kbit latency 42ms burst 2k
"man htb" and "man tbf" are quite usable too.[1] i.e. stuff like "I would like to have tcp/80 limited to 10 mbit/s, tcp/443 limited to 15 mbit/s, while sum of above should never exceed 20mbit/s, and tcp/80 should get priority when competing for that shared 20mbit/s"
EDIT: changed to "burst 2k". Having burst lower than interface MTU will delay large packets essentially forever.
From example code attached to the original article (Linux only):
int outstanding;
ioctl(fd, SIOCOUTQ, &outstanding);If both you and your correspondents do use 3rd party IM client ([1], [2], etc), then just run OTR2 or OMEMO on top of the protocol, and let google store whatever it pleases - it's not going to be much use for them.
[2] http://www.theverge.com/2016/1/16/10780876/microsoft-windows...
Caddy is written in Go, and make use of native fibers/green threads/goroutines.