Skip to content 
I'll show how SO_REUSEPORT speeds up Linux web servers with many concurrent connections and eliminates bottlenecks in the Accept removed. I focus on clear, practical approaches so that you can get more out of multicore systems Performance take out.
Key points
- Accept Bottleneck Avoid and reduce latency
- Multicore Efficiently Utilize Resources Through Kernel Distribution
- Thundering stove significantly reduce
- Architecture Simplify without a user-space dispatcher
- Nginx and use other servers directly
What SO_REUSEPORT Solves Technically
SO_REUSEPORT assigns each worker its own listening socket, so I can use the classic bottleneck avoid at the central Accept. In the past, everything was tied to a single socket, which caused threads to compete and increased wait times. Today, the kernel distributes new connections directly across multiple sockets, which Latency significantly reduces. This eliminates the need for separate dispatcher processes and reduces context switches. Under heavy load, response times remain more consistent because no single listener slows things down.
A Brief Comparison of SO_REUSEPORT and SO_REUSEADDR
SO_REUSEADDR helps me restart quickly, since I can use ports even though TIME_WAIT can bind again. SO_REUSEPORT does something else: it allows multiple listeners simultaneously on the same IP/port combination. Only when I set SO_REUSEPORT before the bind() call does the kernel allow parallel Bind-Operation. The order is important: If a port is occupied without this option, no additional sockets can be added. For parallel workers, SO_REUSEPORT is therefore a key option.
How It Works in the Kernel: Reuse Port Groups and Hash
All sockets with the same IP/port combination and SO_REUSEPORT set are placed in a Group. For each new connection, the kernel calculates a hash based on the source and destination parameters. Based on this, it assigns the connection to an appropriate listener, thereby distributing connections relatively fairly. I benefit from better cache locality because each CPU processes „its“ connections more frequently. For special cases, BPF can Selection further customize it, for example, to implement your own strategies.
Practical Guide: Configuring Nginx Correctly
In Nginx, I enable `reuseport` using the `listen` directive and use multiple Worker-processes. Here's an example: Set `worker_processes` to the number of cores and add `listen 80 reuseport;` to the server block. After that, each worker gets its own listener, and the kernel automatically distributes new connections. For details on the optimal number of workers, see the Nginx Worker Processes. This allows me to achieve higher request rates and more even utilization of the cores.
Making Efficient Use of Multi-Core CPUs
With multiple workers and SO_REUSEPORT, I use Multicore-systems more evenly. I pin workers to cores based on CPU affinity to reduce cache hopping. RSS/RPS on the network card helps distribute incoming packets appropriately across queues. This way, connections are more likely to end up on „appropriate“ cores, which improves the Throughput-rate. This effect is particularly noticeable with many short connections and TLS handshakes.
Monitoring, Rolling Restarts, and Pitfalls
I plan rolling restarts carefully, because closing a listening socket can result in lost backlog-entries. Before I terminate workers, I let their queues run empty and only then take them out of service. For logs, I use separate files for each worker so I can track the distribution later. Monitoring tools must account for multiple processes; otherwise, metrics can be misleading. When it comes to IP binds, I ensure consistency, since 0.0.0.0 and specific IPs would otherwise Conflicts can generate.
SO_REUSEPORT Beyond HTTP
This principle also helps me with UDP-services such as DNS, streaming, or gaming servers. This allows many new packets per second to be distributed across multiple listeners without requiring a user-space load balancer. TCP proxies, gateways, and IoT platforms also benefit from this. It remains important to use the correct number of workers so that the hardware and software operate in sync. I combine this setup with clear Limits for file descriptors and valid timeout values.
Tuning the Network Stack: IRQ, Offloads, Buffers
I'm checking the NIC's IRQ assignments to ensure that queues are assigned to the appropriate CPU-cores. Where appropriate, I use GRO/LRO and offloads, but I always test the latency. I set socket buffers deliberately, since values that are too small cause bottlenecks during peaks, and values that are too large waste memory; more on this at Socket buffer. I also check sysctl parameters such as somaxconn and net.core.somaxconn against the workload profile. I measure the effect of each change in isolation to determine the true Profits to see.
Comparison of Common Web Server Setups
The following table shows typical characteristics of various listener models and helps me with the Choice of the design. I focus on the acceptance path, latency under load, scalability, architectural complexity, and CPU utilization. This allows me to quickly identify which setup best fits my traffic profile. I distinguish theory from practice by checking real-world metrics afterward. The Matrix serves as a starting point for targeted testing.
| Setup | Accept Path | Latency under load | Scaling | Architectural costs | CPU utilization |
|---|---|---|---|---|---|
| A listener without SO_REUSEPORT | A socket | rises early | limited | low | unequal |
| Multiple Workers with SO_REUSEPORT | Kernel-Distribution | more constant | high | low | more evenly |
| Userland Dispatcher | central intake | medium | medium | high | changeable |
| SO_REUSEPORT + BPF Logic | customized selection | very consistent | Very high | medium | very even |
Plan Benchmarks Carefully
I'm testing with and without SO_REUSEPORT to get real Differences to see. Relevant metrics include requests per second, p95/p99 latencies, and CPU utilization per core. I vary the number of workers and identify the sweet spot between context switches and CPU utilization. I select test data that closely mimics real-world conditions, including TLS, keep-alive, and both static and dynamic content. I record the results in a reproducible manner so that I can later Changes can be compared.
Apache: Making Effective Use of the Event MPM
Apache also benefits when I decouple the Accept path and the event-Operate MPM correctly. The choice between Event MPM and Worker MPM depends on the connection profile and available resources. I take into account keep-alives, thread pools, and client limits. This overview helps me get a quick understanding: Event-MPM vs. Worker-MPM. In conjunction with SO_REUSEPORT, I am working specifically on achieving consistent Load per process.
Limitations and Nuances of Distribution
SO_REUSEPORT distributes incoming connections relatively fairly using a hash, but not perfectly evenly. Peak loads can temporarily impact individual workers more heavily if source/destination parameters result in an unfavorable distribution. I therefore monitor worker metrics (accepts, active connections, CPU) and adjust the number of workers, affinities, and RSS queues. Keep-Alive connections remain with the original listener, which provides desired cache locality but can also lead to „sticky“ load patterns. For highly heterogeneous requests (a mix of CPU- and I/O-intensive), I plan buffers to absorb short spikes.
Accept Path in Detail: Backlog, somaxconn, and SYN Queues
I distinguish between the list queue (SYN backlog) and the accept queue. Parameters such as net.ipv4.tcp_max_syn_backlog, tcp_syncookies, and net.core.somaxconn affect how many connection attempts and fully established sockets are maintained. The backlog applies separately to each listener socket—with SO_REUSEPORT, the theoretical buffer capacity is multiplied across all workers. In practice, however, it is limited by the NIC and CPU load. I keep backlogs consistent and measure drop and retransmission rates to detect bottlenecks early.
Nginx Details: accept_mutex, Worker Shutdown, and TLS
As soon as I start using reuseport, I disable `accept_mutex` in Nginx, since the kernel handles fair allocation. For rolling restarts, I set it to „graceful“ and wait for keep-alive connections to time out so that no long transfers are interrupted. On the TLS side, I ensure that workers and instances share ticket keys so that resumption and session IDs work regardless of the assigned listener. I make sure workers don’t get too large (in terms of cache and memory footprint) to avoid cold caches during process switches.
systemd Socket Activation, Containers, and Orchestration
If systemd opens sockets in advance, it must set SO_REUSEPORT; otherwise, parallel binds are blocked. In container environments, I make sure that the desired number of workers actually creates processes per pod/container and that the cgroup CPU allocation matches the affinity strategy. In orchestration platforms, I plan the rolling update strategy so that the Reuseport group remains stable during deployments and does not exclusively block any port. Health checks should not generate unnecessary noise per worker and distort the distribution.
NUMA Awareness and Memory Locality
On NUMA systems, I bind workers to cores on the same NUMA node and ensure that NIC IRQs are directed there whenever possible. I monitor remote memory accesses and page migrations because they drive latency spikes. If the workload scales heavily, it may make sense to have one replica per NUMA node with its own port/frontend; in combination with SO_REUSEPORT, I achieve very stable latencies as long as data and code paths remain node-local.
HTTP/3 and a Focus on UDP
With HTTP/3 (QUIC), I particularly benefit from SO_REUSEPORT in the UDP path: Many handshakes and short-lived connections are distributed without an additional user-space load balancer. I ensure that UDP buffers are sufficiently large and monitor drop counters per queue. Since QUIC logically binds connections to the 5-tuple, the distribution remains stable; nevertheless, I safeguard against potential issues with consistent retry and token strategies to ensure that worker selection remains transparent and high-performing.
eBPF Fine-Tuning for Reuseport
With a Reuseport BPF program, I can further control socket selection—for example, based on destination hostname (SNI), local priorities, or per-worker load. I only use this when the standard hash distribution isn't sufficient, since additional logic increases complexity. For troubleshooting, I verify that BPF programs are actually loaded and running without errors, and I have a fallback strategy ready in case the policy needs to be unloaded.
DDoS resilience and security
SO_REUSEPORT increases the system's capacity—which is both a blessing and a risk. I set rate limits and connection limits per worker to prevent individual processes from becoming overloaded. Combined with SYN cookies, moderate timeouts, and clean L7 limits, I prevent load spikes from permanently tying up resources. I separate logs to detect abuse patterns per worker more quickly, and if necessary, I use iptables/nftables to throttle malicious sources early on.
Debugging and Verification
I check the configuration using `ss -ltnp` (TCP) or `ss -lunp` (UDP) to see if there are multiple listeners on the same IP/port combination. I use `perf`, `top`/`htop`, and `mpstat` to verify consistent CPU usage. Netstat/ss counters, dmesg messages, and NIC drop statistics (ethtool -S) indicate whether queues are overflowing. For more in-depth analysis, tcpdump and Perf events provide insight into accept paths, retransmissions, and retries. Correlation remains key: always examine metrics per worker, per CPU, and per queue.
Avoiding Common Configuration Errors
- A worker without SO_REUSEPORT binds first and blocks everyone else.
- A mix of 0.0.0.0 and specific IP addresses is used—listeners are placed in separate groups.
- accept_mutex is enabled in Nginx despite the reuseport directive—unnecessary serialization.
- Inappropriate backlogs: somaxconn is smaller than the backlog set on the server.
- No shared TLS ticket configuration—resumption rate plummets.
- RSS is incorrectly sized—the IRQ load is concentrated on a few cores.
Capacity Planning: Worker Size and FD Limits
I balance the number of workers against RAM per worker, open files, and the number of connections. Too many processes increase context switching and cache pressure; too few waste parallelism. I set file descriptor limits generously and consistently (ulimit, systemd limits, hard/soft limits), because each worker needs its own FDs for sockets, logs, and upstream connections. I also allocate enough ephemeral ports and monitor TIME_WAIT volumes to ensure that short-term spikes don’t go to waste.
Benchmarks: Common Pitfalls
I warm up servers and caches, calibrate the load generator (to avoid hidden bottlenecks), and separate the control and data networks. Tests run long enough to reliably measure p99/p999, and I vary think times, keep-alive rates, and TLS parameters. I log kernel and server settings to ensure that subsequent runs remain comparable. Where I use eBPF policies, I document their version and effect separately to avoid confusing cause and effect.
Checklist for the start
First, I check the kernel version and make sure that SO_REUSEPORT is available and configured correctly set is. Next, I enable the option in the web server configuration and set up the desired number of workers. I check somaxconn, file descriptor limits, and the NIC queues. Then I run load tests, compare metrics, and iterate. Finally, I fine-tune the logging, restart strategy, and affinity from.
Abstract
SO_REUSEPORT eliminates the Accept bottleneck, distributes new connections via kernel hash, and achieves better performance on multicore systems Throughput I use multiple listeners per port, avoid the „thundering herd“ problem, and save myself the trouble of a separate dispatcher. In Nginx, this is achieved with “listen … reuseport” and an appropriate number of workers. Combined with CPU affinity, proper IRQ distribution, and sensible buffering, I ensure consistent Latencies under load. By reviewing, testing, and fine-tuning these steps, you can improve performance without incurring additional hardware costs in euros.
