Skip to content 
Receive-Side Scaling distributes network traffic across multiple cores via 10- and 25-Gbit/s links, enabling Linux servers to handle high throughput with low latency. I’ll demonstrate in a practical way how I enable RSS, which Cues on Core's map to avoid bottlenecks in interrupts and cache hits.
Key points
I'll briefly summarize the most important points so you can quickly plan your next steps.
- Load distribution: Packages are distributed across multiple cores via several queues.
- Cache Locality: A flow remains consistently on the same queue.
- Hashing: 4-tuple hashing distributes flows evenly across queues.
- affinity: Targeted IRQ mapping reduces latency.
- Scaling: Starting at 10/25 Gbit/s, RSS ensures high throughput.
These points are interrelated and support the Performance based on real-world workloads. I prioritize the correct number of queues first, then the CPU-Affinity. Next, I check the hash parameters and fine-tune the settings.
What Receive Side Scaling Does
RSS breaks down packet reception into several Receive Queues, which I assign to specific CPU cores so that no single core becomes a bottleneck. This reduces hard interrupt spikes and smooths out processing via soft IRQs, which reduces latency spikes and boosts throughput. Each queue triggers its own interrupts, which I bind firmly to specific cores to keep data paths consistent. This consistency promotes the Cache-Locality, because a flow always encounters the same core. It is precisely this interaction that directly contributes to measurable efficiency at high PPS rates.
How RSS Works Technically
The NIC uses the source/destination IP addresses and source/destination ports to form a Hash and uses it as an index for the indirection table, which points to queues. In this way, packets from a single flow always end up in the same queue and thus remain bound to the same kernel. Different flows are distributed evenly, provided that the hash keys and protocol fields are configured appropriately. The workload is thus distributed close to the Hardware, which reduces the load on the kernel and lowers overhead. That's exactly what I want to do to keep packet processing per core low at 10G/25G.
Why RSS Matters at 10 and 25 Gbit/s
At 1 Gbit/s, a single Core the packet load, but once it reaches 10 Gbit/s, the balance shifts rapidly. Small packets cause the PPS count to rise, causing a core to quickly reach 100 percent load and resulting in packet drops. That’s exactly when RSS acts as a multiplier for usable bandwidth. I distribute the load across several Cores, reduce context switches, and keep latency curves more stable. The result: actual throughput only approaches the link rate when RSS is clean.
Setting Up RSS on a Linux Server
On Linux, I mainly manage RSS through ethtool, driver options, and sysfs, so that the NIC's capabilities are fully utilized. First, I read the maximum number of RX channels, then I set the number of queues to match the CPU. Next, I check the RSS hash for TCP/UDP and, optionally, for VLAN or tunneling, to ensure that load profiles remain properly distributed. To help with the background noise of interrupt distribution, I use IRQ balancing, even though I prefer to pin critical queues manually. That's how I link Cues closely follow the host's topology and prevent disruptive migrations.
Indirection Table, RSS Key, and Hash Fine-Tuning: Specific Commands
First, I check the current configuration and the NIC key:
ethtool -x eth0 # Display the Indirection Table (RX Queues) and RSS Key
ethtool -n eth0 rx-flow-hash tcp4
ethtool -n eth0 rx-flow-hash udp4
To ensure a clean, even distribution, I set the Indirection Table to the desired number of queues. With 16 queues, I choose an even mapping:
ethtool -X eth0 equal 16 # distribution evenly across 16 queues
If necessary, I'll adjust the hash fields. For TCP4 with 4-tuples (s=src-ip, d=dst-ip, f=src-port, n=dst-port):
ethtool -N eth0 rx-flow-hash tcp4 sdfn
ethtool -N eth0 rx-flow-hash udp4 sdfn
ethtool -N eth0 rx-flow-hash tcp6 sdfn
ethtool -N eth0 rx-flow-hash udp6 sdfn
Some drivers also allow you to set your own RSS key (e.g., for better distribution in special cases):
ethtool -X eth0 hkey # only if the driver/NIC supports it
Set CPU Affinity and NUMA Correctly
I map each RX queue using IRQ affinity Assign them to dedicated cores, taking NUMA into account so that data travels only a short distance through the memory controller. If the NIC is on Node 0, I also bind the main queues to cores on Node 0 and place workloads nearby. This proximity reduces remote accesses and significantly lowers memory latencies. It’s helpful to have a profile for production queues as well as separate cores for management and offload tasks. If you want to dive deeper, you’ll find guidance on fine-tuning at IRQ affinity, which affects planning per core simplified.
IRQ Affinity Playbook: From IRQs to Stable Core Binding
First, I determine which IRQs are associated with the RX queues, and then I pin them:
grep -E "eth0.*Rx" /proc/interrupts
cat /sys/class/net/eth0/device/numa_node
I set the mapping using smp_affinity_list, so I don't have to calculate hex masks. Example: RX queues 0–7 on cores 2–9:
# Example: Assign IRQs to cores 2–9 (one line per IRQ)
echo 2 > /proc/irq//smp_affinity_list
echo 3 > /proc/irq//smp_affinity_list
echo 4 > /proc/irq//smp_affinity_list
...
echo 9 > /proc/irq//smp_affinity_list
Important: MSI-X must be enabled so that each queue has its own interrupts. If I use manual pinning, I disable irqbalance for these IRQs (e.g., via a blacklist) or disable the service specifically on hosts with a static layout. I also check NUMA using lscpu and the PCIe mapping, so that I don't create any cross-node paths.
Hash Configuration and Protocols
I define the hash fields so that real Traffic-Distribute traffic evenly rather than concentrating it on just a few queues. For TCP/UDP, I use the 4-tuple; for IPv6, I use a similar approach, while for VXLAN or GRE, I take additional encapsulation fields into account. Some NICs offer configurable hash keys, which I adapt to the dominant workload. As soon as I see load clusters on individual queues, I adjust the hash selection accordingly. This step takes very little time but prevents a Imbalance when the number of connections is high.
Interrupt Coalescing and PPS
I combine RSS with moderate Interrupt coalescing, to bundle PPS-intensive traffic into manageable batches. This reduces interrupt overhead, but must not compromise the latency of sensitive services. That’s why I measure round-trip times and adjust coalescing values incrementally. If you’re handling storage or backup workloads, you can coalesce more aggressively than with L7 APIs or VoIP. Overall, I balance Latency against throughput until both are consistent.
Coalescing in Practice: Profiles and Measurement Points
I'll start with moderate default settings and work my way toward the optimal settings for each workload. Three proven starting profiles:
- API/Low Latency:
rx-usecs 2–6,rx-frames 16–32, adaptive off - All-around:
rx-usecs 8–16,rx-frames 32–64, adaptive to - Bulk/Storage:
24–48 rx-usecs,rx-frames 128–256, adaptive to
ethtool -c eth0
ethtool -C eth0 rx-usecs 12 rx-frames 64 adaptive-rx on
To do this, I measure p95/p99 latencies, PPS, CPU load per core, and retransmissions. As soon as I see increasing variance in API/VoIP, I proceed with rx-usecs back down. For storage, I tend to scale up using frames to save on interrupts.
RSS in 10-Gbit Environments
On 10G NICs, I usually work with 8 to 16 Cues per port, provided the CPU has enough cores available. Web servers, storage gateways, and virtualization hosts scale cleanly across many parallel connections. I map main queues to available cores and then measure PPS, latency, and retransmissions. If drops occur, I check coalescing, hashing, and utilization per queue. Afterward, I fine-tune the affinity, until the load appears to be even.
RSS in 25-Gbit and Multi-25G Setups
At 25 Gbit/s, the PPS and bus load increase, which is why I NUMA-Pay closer attention to awareness, sufficient queues, and offloads. Large Receive Offload (LRO) or RSC can reduce packet pressure on the stack, provided applications can tolerate it. I also check PCIe lanes to rule out bottlenecks outside the network. On hosts with multiple 25G links, I strictly separate queues and affinity based on tasks and nodes. This is how I use Bandwidth and cores efficiently, without resorting to cross-node traffic.
Hardware/Driver Details: What I Look For
Not every NIC behaves the same way. Intel generations (e.g., ixgbe, i40e, ice) offer features such as Flow Director/ATR, which specifically bind flows to queues—which is helpful when I want to smooth out hotspots. Mellanox mlx5 can support aRFS in hardware, which reduces CPU load when the stack serves many sockets. I decide on a case-by-case basis whether to enable these features and measure whether they improve load balancing. On routing/NAT systems, I often disable LRO and rely on GRO to maintain header consistency; on pure server workloads, LRO/GRO can help mitigate PPS pressure. It’s also important to have sufficient MSI-X vectors per queue and up-to-date firmware versions.
RSS in Virtualization and Containers
In the hypervisor, I combine physical RSS-Queues with multi-queue-capable vNICs, such as virtio-net, so that guests do not experience artificial bottlenecks. I ensure CPU pinning for the VMs and configure their vCPU NUMA proximity to the physical NIC. This keeps data local and reduces the host’s memory access overhead. For containers, I bind critical pods to appropriate cores and keep host queues free of interference. This organization increases the Efficiency in microservices, where many small flows are created.
Using SR-IOV and VF-RSS Correctly
With SR-IOV, I assign VMs their own VFs, which in turn can provide multiple queues and RSSs. I plan for enough VFs per port, pay attention to MSI-X capacity, and pin the VF IRQs in the VM to match their vCPUs. In Linux guests, I explicitly enable multi-queue; otherwise, the vNIC often remains single-queue:
# in the guest (virtio-net example)
ethtool -l eth0
ethtool -L eth0 combined 4
I strictly distribute hosts with multiple VFs by NUMA node and workload so that VMs do not interfere with each other on the same physical RX paths.
Monitoring and troubleshooting
I monitor the utilization per Queue, individual kernels, packet loss, and retransmissions, in order to detect imbalances early on. If one kernel takes off while others remain idle, the affinity or the number of queues is often incorrect. In such cases, I check hash tables, IRQ masks, and coalescing values one after another. In addition, I look at the SoftIRQ Load, because it provides indications of crowding-out effects. Only when these signals appear stable do I scale up traffic or expand Cues continue.
RPS, RFS, and XPS: Software Extensions to RSS
If a NIC has only a few queues or if I'm using bonding/tunneling, I supplement RSS with RPS (Receive Packet Steering) and RFS (Receive Flow Steering). RPS distributes SoftIRQs across cores, while RFS binds flows to the core on which the corresponding socket is active. I enable both specifically:
# Increase the global number of flow entries (RFS)
echo 32768 > /proc/sys/net/core/rps_sock_flow_entries
# Set CPUs for RPS per RX queue (example mask—adjust as needed!)
for f in /sys/class/net/eth0/queues/rx-*/rps_cpus; do echo ffff > "$f"; done
Set the flow table per RX queue for RFS to #
for f in /sys/class/net/eth0/queues/rx-*/rps_flow_cnt; do echo 4096 > "$f"; done
On the TX side, I use XPS (Transmit Packet Steering), so that outgoing packets are sent by the core that generated them:
for f in /sys/class/net/eth0/queues/tx-*/xps_cpus; do echo ffff > "$f"; done
RPS/RFS/XPS use up some CPU resources, but they're helpful when I don't have enough queues on the hardware side or when I want to strictly maintain socket locality.
Single-Flow Performance, GRO/TSO, and Busy Polling
A single flow remains bound to a core for good reason. If I want to increase the bandwidth of a single flow, I rely on offloads (GRO/TSO), a high core frequency, and proper coalescing. For latency-critical paths, Busy polling help:
Set # to a low value and measure it
sysctl -w net.core.busy_read=25
sysctl -w net.core.busy_poll=25
Busy polling reduces context switches but consumes CPU time. I enable it only where p99 latencies matter, and I always validate the effects on overall utilization and tail latency. I generally keep GRO enabled on servers; I use LRO depending on the role. For middleboxes, I take a conservative approach so as not to disrupt header processing and hash consistency.
Recommendations and Examples: Queues, Affinity, Commands
As a starting point, I choose a queue size that corresponds to the CPU Adjust accordingly, then monitor the load per queue and fine-tune in increments. For 10G, 8–16 queues are often sufficient; for 25G, I usually set a higher number, provided the cores are available. For affinity, I use distinct masks per IRQ so that I can analyze the paths more easily later. The following table provides concise guidelines, which I then verify through measurements. Only the measurement results determine whether I more increase or decrease.
| Link Speed | Typical RX Queues | Sample Commands | Notes |
|---|---|---|---|
| 10 Gbit/s | 8-16 | ethtool -l eth0 | ethtool -L eth0 rx 16 | Coalescing Keep it moderate; check L7 latency |
| 25 Gbit/s | 16–32+ | grep . /proc/interrupts | IRQ masks via echo | NUMA Note: Check the PCIe lanes |
| Multi-25G | Separately per port | Enable vNIC multi-queue (e.g., virtio) | Queues on cores and Workloads divide |
These guidelines are only a starting point, not a goal, because workloads vary greatly. I log changes, take measurements before and after adjustments, and otherwise keep the environment unchanged. As soon as the system remains stable under production load, I freeze the configuration. Later, I repeat the measurements after kernel or driver updates. This way, I stick to RSS Right on track and reliable, reproducible results.
Common Pitfalls and Solutions
Too few Cues Overloading individual cores, or having too many of them, increases administrative overhead and harms the cache hit rate. An unfortunate affinity setting shifts interrupts to cores that are already under load or to the wrong NUMA nodes. An inappropriate hash function also causes dominant flows to clog queues. I solve this step by step: adjust the number of queues, correct affinity, expand hash fields, and fine-tune coalescing. I document every change with Metrics, before I move on to the next lever.
Real-world scenarios
A 10G storage server quickly benefits from 8–12 Cues plus moderate coalescing to ensure smooth bulk transfers. An API server with a high number of connections often requires finer hash tables and lower latency for interrupts. Virtualization hosts benefit significantly when vNIC multi-queue is enabled on the guest side and aligns with the host layout. Container workloads run more smoothly when critical pods run close to the NIC and NUMA memory. I adapt these patterns on a case-by-case basis by PPS, compare retransmissions and queue distribution.
High-Performance Platforms as an Advantage
Hosting setups with consistently configured RSS, multi-queue NICs, and proper affinity provide noticeable headroom during peak loads. Anyone evaluating server offerings should specifically inquire about multi-queue capability, NUMA pinning, and monitoring. A provider that visibly implements these features often achieves noticeably better throughput curves. For high-performance server and hosting solutions, I highly recommend webhoster.de. This focus pays off in Performance and stability, especially when there are many parallel flows.
Summary for practice
I activate Receive Side scaling: Set a reasonable number of queues, bind IRQs to appropriate cores, and check the hash configuration. After that, I optimize coalescing for low latency, ensure NUMA proximity, and distribute workloads consistently. In virtualization, I use multi-queue all the way down to the guests and keep pinning and affinity in sync. Metrics such as PPS, queue load, retransmits, and latency determine the next step. Following this approach allows you to fully utilize 10G and 25G in practice, while maintaining Latency within this framework and reliably generates network returns.
