Skip to content 
I show how Bandwidth Shaping on servers and traffic control in Linux to control packet flows in such a way that latency, jitter and outages are noticeably reduced. I use prioritized queues, limits and QoS rules to protect business-critical flows such as VoIP, APIs or store requests from background loads and backups.
Key points
The following key statements help me to control bandwidths and traffic on Linux servers in a targeted manner and to make them permanently plannable.
- Prioritization time-critical flows reduces latency and jitter.
- Rate limits and throttling avoid bursts and buffer jams.
- HTB/SFQ distribute bandwidth fairly and keep classes constant.
- QoS filter control by IP, port, protocol or markers.
- Monitoring via P95 and alerts detects bottlenecks at an early stage.
I build up these points step by step, measure effects continuously and adapt classes and rates to real usage.
What bandwidth shaping actually means
When shaping, I regulate the Parcel flow actively instead of just reactively throttling. I keep rates constant, prioritize real-time and interactive traffic and smooth out irregular data bursts. To do this, I use rate limiting for outgoing traffic and throttling for incoming data streams. This separation creates clear responsibilities per direction and prevents full buffers. For hosting environments, I set defined upper limits for each customer or application so that a load peak does not slow down the entire system.
Traffic control in Linux: tools and concepts
Under Linux I control traffic with the tool tc and the kernel queuing disciplines (qdisc). Typical building blocks are root qdisc, classes and filters that define the assignment of packets to priorities and limits. I often start with HTB as a hierarchical controller for guaranteed and maximum rates. I also use SFQ for fair distribution within a class. I limit bandwidth to 90-95 percent of the physically possible rate in order to retain burst headroom and avoid latency peaks.
Ingress shaping (Ingress): IFB instead of Policer
For incoming traffic, I do not form directly on the physical interface, but use a IFB-device (Intermediate Functional Block). I mirror ingress packets to the IFB and treat them there like outgoing traffic - including HTB hierarchy, fairness and limits. This is finer than a pure policer, which only discards hard and often increases jitter.
modprobe ifb numifbs=1
ip link add ifb0 type ifb
ip link set dev ifb0 up
# Activate ingress on PHY and redirect to IFB
tc qdisc add dev eth0 handle ffff: ingress
tc filter add dev eth0 parent ffff: protocol ip u32 match u32 0 0 \
action mirred egress redirect dev ifb0
# Shaping on the IFB as with egress
tc qdisc add dev ifb0 root handle 2: htb default 20
tc class add dev ifb0 parent 2: classid 2:10 htb rate 40mbit ceil 60mbit
tc class add dev ifb0 parent 2: classid 2:20 htb rate 20mbit ceil 40mbit
tc qdisc add dev ifb0 parent 2:10 handle 210: fq_codel
tc qdisc add dev ifb0 parent 2:20 handle 220: sfq
With IFB I gain control over download peaks, for example when backup or mirror jobs tie up incoming bandwidth. In practice, I use IFB on interfaces with highly asymmetrical rates (e.g. 1G/200M) or where incoming bursts jeopardize interactivity.
HTB, TBF and SFQ in comparison
For the right choice of qdisc I look at application goals: Guarantees, burst behavior and fairness. HTB offers hierarchical classes with fixed and maximum rates, TBF strictly limits by token bucket, SFQ distributes opportunities via flows. In combination, they form a resilient framework in practice: HTB caps and guarantees, SFQ prevents dominance of individual connections, TBF tames stubborn bursts. The following table summarizes core features for typical server scenarios. This allows me to decide more quickly which discipline makes sense at which point.
| qdisc/Feature | Purpose | Strengths | Typical use |
|---|---|---|---|
| HTB | Hierarchy and rate control | Guaranteed rate, upper limit, inheritance | Clients, service classes, QoS profiles |
| TBF | Strict Lids | Simple, very deterministic | Uplink cap, hard app limits |
| SFQ | Fairness per flow | Low overhead, good distribution | Downloads, P2P, many sessions |
| FQ_CoDel | AQM against bufferbloat | Low latency, queue defusing | Edge routers, latency-critical links |
For accesses with significantly fluctuating RTTs, I use FQ_CoDel or - depending on the kernel - CAKE as an all-rounder. In server practice, however, I stick with HTB+SFQ/FQ_CoDel because I can combine guarantees, borrowing and fair distribution cleanly in a hierarchy.
Practice: tc rules for typical servers
I start with a simple HTB-structure and then assign using a filter. A root qdisc with default class catches unclassified packets, prioritized classes receive guaranteed rates. I then refine the filters: HTTP/S to class A, database replication to class B, backups to class C. This keeps store requests fast, while backups use the leftovers. For basics and vocabulary, I refer you to this introduction to Bandwidth management, which makes the procedure tangible.
tc qdisc add dev eth0 root handle 1: htb default 20
tc class add dev eth0 parent 1: classid 1:10 htb rate 50mbit ceil 70mbit
tc class add dev eth0 parent 1: classid 1:20 htb rate 20mbit ceil 50mbit
tc class add dev eth0 parent 1: classid 1:30 htb rate 10mbit ceil 30mbit
tc qdisc add dev eth0 parent 1:10 handle 110: sfq
tc qdisc add dev eth0 parent 1:20 handle 120: sfq
tc qdisc add dev eth0 parent 1:30 handle 130: sfq
tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip dport 443 0xffff flowid 1:10
tc filter add dev eth0 protocol ip parent 1:0 prio 2 u32 match ip dport 3306 0xffff flowid 1:20
tc filter add dev eth0 protocol ip parent 1:0 prio 3 u32 match ip sport 22 0xffff flowid 1:30
Classification with DSCP and Marks (IPv4/IPv6-capable)
To ensure that filters work regardless of IP version and NAT, I mark packets early and then map them via fwmark in classes. This saves me complex u32 matches and keeps rules lean. I also use DSCP for end-to-end semantics, e.g. for VoIP or interactivity.
# Example with nftables: Prioritize TLS, throttle backups
nft add table inet mangle
nft add chain inet mangle prerouting { type filter hook prerouting priority -150; }
nft add rule inet mangle prerouting tcp dport 443 meta mark set 10
nft add rule inet mangle prerouting tcp dport 22 meta mark set 30
nft add rule inet mangle prerouting tcp dport 873 meta mark set 40 # rsync/Backups
# Mapping in HTB classes (works for IPv4/IPv6 equally)
tc filter add dev eth0 parent 1:0 prio 1 handle 10 fw flowid 1:10
tc filter add dev eth0 parent 1:0 prio 2 handle 30 fw flowid 1:30
tc filter add dev eth0 parent 1:0 prio 3 handle 40 fw flowid 1:20
Important: I set DSCP/markers as far as possible. at the edge (input of the machine or in front of it) so that later decisions can be made quickly and tc has less work to do under load.
QoS strategies for hosting: fairness and limits
In multi-tenant setups I secure Fairness via fixed guarantees per customer and caps per application. I mark packages via DSCP or according to ports and assign them to suitable classes. Downloads and backups are allowed to fill capacity, while interactive sessions are prioritized. In this way, SLA-relevant services remain prioritized without excluding other tenants. I summarize practical scenarios and prioritization logic in this overview Traffic prioritization which fits well with tc rules.
Persistence and automation
My QoS policies survive reboots and interface restarts. I store tc commands as an idempotent script and integrate it into systemd or netplan/networkd. For devices with dynamic names (e.g. veth/tap) I use udev rules or systemd templates.
# /usr/local/sbin/tc-setup.sh (build idempotent)
#!/bin/sh
tc qdisc replace dev eth0 root handle 1: htb default 20
# ... more classes/filters here
# systemd unit: /etc/systemd/system/tc-setup.service
[Unit]
Description=Traffic Control Setup
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/tc-setup.sh
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
I roll out changes in a controlled manner: first on staging, then for a limited time in the production system (tc replace instead of add), accompanied by metrics and a rollback button.
Monitoring, P95 and troubleshooting without frustration
I measure effects continuously instead of focusing on Gut feeling to leave. P95 latencies, queue lengths and packet losses show whether rules are effective or too strict. Tools such as iftop, vnStat and Netdata make load peaks visible, logs from tc and iptables show the allocation. If jitter occurs, I reduce ceil values slightly and check CoDel/FQ_CoDel as an AQM measure. In the event of bottlenecks, I adjust class weights step by step and maintain measurement windows after each change.
Test methodology: load simulation and verification
I simulate realistic scenarios: a continuous flow (iperf3), in parallel short interactions (small HTTP requests) and periodic bursts (backup/rsync). I then check whether interactive flows maintain consistently low latency and whether bursts are smoothed cleanly.
# Test opposite direction (download/ingress)
iperf3 -c -R -t 60
# Read shaping statistics
tc -s qdisc show dev eth0
tc -s class show dev eth0
# Check jitter/RTT distribution
ping -i 0.2 -c 100 | awk '/time=/{print $7}'If classes need borrows permanently, I increase guaranteed rates slightly. If drops accumulate in AQM queues, I check buffer sizes and whether limits are set too aggressively.
Performance tuning: 90-95 % cover, burst smoothing, MTU
I limit the output rate to 90-95% of the link speed to avoid buffer bloat and allow AQM to take effect. I smooth bursts with token bucket parameters (rate, burst/latency) so that short-term peaks do not fill entire queues. I check the MTU to reduce fragmentation and avoid path MTU problems. For highly fluctuating workloads, I set generous ceil values but conservative guaranteed rates. This setup keeps priority traffic fast while background processes continue to run neutrally.
Hardware offloads, multiqueue and IRQ tuning
For precise shaping on fast links, I know the interaction with NIC offloads. TSO/GSO/GRO accelerate, but at very low target rates they can worsen the fine grain. For sensitive links, I switch off TSO/GSO as a test and measure latency/CPU gain against it. On multiqueue NICs I use a mq-I distribute the CPU load with RPS/XPS and IRQ pinning so that QoS work does not starve on a CPU.
# Selectively test offloads (cautious in production)
ethtool -K eth0 tso off gso off gro off
# Multiqueue layout
tc qdisc add dev eth0 root handle 1: mq
tc qdisc add dev eth0 parent 1:1 handle 10: htb default 20
tc qdisc add dev eth0 parent 1:2 handle 20: htb default 20
# ... continue per queue and set classes/filters as usual
With txqueuelen, ring buffer sizes and IRQ affinity, I continue to trim the latency. The aim is to achieve a stable, short queue path that doesn't tip over under load.
Security and segmentation: shaping with firewall and VLAN
I combine QoS with Segmentation, so that critical networks retain their own capacities. I set separate queues for each VLAN or interface, firewalls mark packets as soon as they enter the server. This means that tc has to make fewer decisions under load because packets are classified early on. Backups from the storage VLAN remain on their path, while frontend HTTP does not starve. The separation also shortens error analyses because flows can be assigned more clearly.
Virtualization and containers: Where I form
In KVM/virtio setups I prefer to form Edge: on the bridge interface (br0), on the physical uplink (eth0) or specifically per guest on its vnet interface. I like to implement per-tenant guarantees on vnetX, global caps on the uplink. In Kubernetes, tc is practicable on veth peers or node uplinks; I assign markers early via CNI/iptables/nftables so that the allocation remains deterministic. For SR-IOV or DPDK, I make sure that the data paths still pass through tc at all - otherwise I form beforehand or use NIC's own rate limiters.
Costs and benefits: Efficiency instead of hardware upgrades
With clean Shaping I make better use of existing lines and often save on expensive upgrades. Less packet loss and lower latency directly improve the user experience. In hosting environments, this pays off twice over because fair limits prevent escalations between clients. In practice, I see that stable rates increase throughput because retransmits are reduced. These effects are ultimately reflected in clearer SLAs and fewer support cases.
Frequent pitfalls and quick checks
- Inappropriate units: I check whether
mbitandkbitare written correctly and burst/latency match the MTU. - Priority reversal: Too many high-priority classes without a real limit lead to starvation of the defaults. I strictly adhere to upper limits.
- NAT/IPv6 overlooked: Port filters do not work as intended after NAT/IPv6. I mark early (fwmark) and then map into classes.
- Ceil less than rate: A common typo that blocks borrowing. I validate each class with
tc -s class. - Ingress only polarized: Hard dropping creates jitter. With IFB I form finer and fairer.
- Offloads distort measurements: I document the offload status for every test and compare apples with apples.
Future outlook: AI-supported reservation and adaptive policies
I use trends like Predictions based on historical load to dynamically adjust classes shortly before peak times. Learning models reserve bandwidth for VoIP or checkout phases before queues grow. In hybrid networks between cloud and on-prem, I use temporary caps and burst budgets that change policies on a schedule. This reduces operational interventions and keeps services predictable even during special events. I bundle more in-depth background knowledge on scaling and limits here: Traffic management and hosting limits.
Summary & checklist
I first set a clear Hierarchy with HTB, issue guarantees and keep Ceil slightly below link speed. I then classify according to services, protocols or DSCP and check latency, jitter and P95 values. I use SFQ or FQ_CoDel to ensure fair distribution and short queues. Monitoring accompanies every change so that I can decide on effects instead of guessing. In this way, bandwidth shaping is not a one-off act, but a lean control loop that keeps server traffic secure and predictable.
