NUMA Balancing in Linux: Should You Disable It or Leave It Enabled?

NUMA balancing in Linux determines whether the Kernel Whether memory accesses are automatically localized or whether I manually control their placement. In this guide, I'll explain when to leave Numa balancing enabled and when to disable it for Latency-Turn off security.

Key points

• Automatic Helps with mixed workloads without NUMA tuning.
• Disable in cases of pinning, static policies, or high latency.
• Overhead is caused by scans, faults, and migrations.
• Configuration Control via sysctl or boot parameters.
• Testing and measure instead of guessing, then decide.

NUMA Explained in a Nutshell: Latencies and Locality

In NUMA systems, the hardware organizes memory into several nodes, each of which CPUs are spatially close. Local access takes less time than remote access, which I immediately noticed Latency and I notice the impact on bandwidth. If a process is running on one node but the data is located on another, I lose valuable microseconds per access. This is exactly where the kernel steps in and optimizes the Location in terms of pages. Anyone who understands the basic concept will quickly realize that proximity between processing cores and data is the direct path to constant Performance.

How Automatic NUMA Balancing Works

The kernel tracks which cores a process accesses pages from and triggers specific actions Hint-Faults. This allows it to identify which node is receiving the most traffic and then move the appropriate pages there. These migrations reduce remote accesses and increase local Hit rate. I see this effect particularly in dynamic workloads where threads migrate and memory moves. Anyone who wants to delve deeper can explore the relationship between CPU and memory proximity via CPU/Memory Affinity understand in a practical way.

When to Keep It Active: Typical Workloads

I leave the feature enabled when applications do not have their own NUMA logic and processes frequently change. Typical candidates include application servers, databases with variable workloads, and hosts with many dumpster diving. In setups like this, the automatic system brings pages and threads closer together without requiring me to pin them manually. The proportion of local accesses increases noticeably, especially on multi-socket hosts. For administrators managing heterogeneous services, this provides a good Compromise in terms of time and effort.

When to deactivate: clear criteria

I turn off the automatic mode as soon as I start paddling deliberately or clearly Policies set. If I use numactl, cgroups, or MPOL_BIND/MPOL_PREFERRED, a fixed decision regarding memory paths already exists. In that case, hint faults and migrations create unnecessary Overhead. The same applies to real-time or HFT scenarios, where every microsecond counts and predictability is a priority. Those who delve deeper into the selection of placement rules will benefit from a look at suitable Memory Policies.

Understanding and Measuring Overhead

Automatic balancing creates work: scans, Faults and page migrations consume CPU time. This is hardly noticeable when remote accesses drop significantly, but it’s hardly worth it if the layout is already cached locally. That’s why I always check the actual impact using numastat, perf, and meaningful Benchmarks. What's interesting is the trend over several minutes, not just a brief spike. I'll only keep this mode active once the metrics consistently show that local traffic is increasing and latency is decreasing.

Configuration: Sysctl and Boot Parameters

I check the status via /proc or sysctl and change it immediately if necessary, without a Restart. For testing purposes, simple commands like the ones below—which I run in the console—are sufficient. For long-term use, I set the value in a sysctl file so that it persists after a reboot. If you want to set this at boot time, use the kernel parameter `numa_balancing=enable` or disable. I document every change and note which workload phase I made it during.

cat /proc/sys/kernel/numa_balancing
echo 0 > /proc/sys/kernel/numa_balancing
sysctl -w kernel.numa_balancing=1
# /etc/sysctl.d/90-numa.conf
# kernel.numa_balancing = 0

Container and Virtualization Scenarios

On hosts with many VMs and containers, automatic Localization often showcase their strengths. Processes start and stop, cgroups shift the load, and the kernel keeps memory closer to the active cores. I notice this especially on large multi-socket servers with multiple nodes. I clearly distinguish special cases involving strict pinning of individual instances and specifically disable the automatic behavior in those cases. For a more detailed classification, it helps to look at practical NUMA Optimization in host mode.

Decision Table for Practical Use

The following overview summarizes typical scenarios, their expected impact, and my clear Recommendation. I use them as a starting point, but I never replace them with actual measurements from the real system. Every environment has its own quirks, and I make decisions only after obtaining reproducible Results ... A systematic approach saves time later on when troubleshooting and fine-tuning. Running small test runs before a rollout almost always pays off in Constance and predictability.

Scenario	Typical effects	My recommendation
Standard Workloads Without NUMA Tuning	More local hits, fewer remote ones Reads	Leave active
Databases with fluctuating load	Dynamic page localization, moderate Scans	Leave it active, test it
Hard Real-Time or HFT	Hint-fault latency is a problem Jitter-Goals	Disable, pin manually
Manual pinning via numactl/cgroups	Automatic system conflicts with fixed Policies	Disable
Static Memory Policies (MPOL_BIND, etc.)	Migrations do not bring any real Advantage	Disable
Test/Analysis Environment	A good view of the location and Effects	Leave active, check variants

Guide to Testing and Validation

I'll start with the balancer enabled and log local versus remote Accesses using Numastat. Then I disable the function and repeat the measurements exactly the same way. I evaluate differences not only in terms of average values, but also in terms of Percentiles. Regression tests using production load profiles provide the most reliable results. Only then do I make a final decision in favor of a host, VM, or specific Service.

Common Pitfalls and Myths

A common misconception is that automation replaces every Pinning. That's not true, because fixed latency budgets can hardly accommodate any additional faults. Equally false is the assumption that migrations always free of charge happen. Especially with layouts that are local to begin with, the overhead tends to have a negative rather than a positive effect. Those who avoid myths and measure accurately make decisions with significantly higher Accuracy.

Limitations of Automation and Interactions

AutoNUMA has a significant impact on anonymous pages that a process allocates itself. However, not everything can be meaningfully migrated. Pinned pages (mlock), DMA/device memory, DAX, or RDMA-registered regions remain where they are. Even shared pages (e.g., heavily shared libraries or the page cache) offer only limited benefit from migration because multiple processes compete for Access pattern generate. I also take into account the costs of Transparent Huge Pages (THP): Their migration is more expensive than with 4-KiB pages and can cause load spikes. Those who aim for strict latency targets often combine THP=never or madvise with balancing disabled and clean pinning to avoid surprises.

Another aspect is the interaction with the CPU scheduler. The scheduler tries to place threads where their data is located—and the balancer moves data to where the threads are running. The two complement each other, but in the case of fluctuating load, this can temporarily lead to Oscillations In practice, the scan intervals mitigate these effects; if you encounter extremely erratic load profiles, you can stabilize the situation by using longer scan periods or more stable thread pinning.

Fine-Tuning the Scan Parameters

In addition to the global switch, there are kernel parameters that allow me to fine-tune the aggressiveness of the automatic system. The exact names may vary slightly depending on the kernel version, but the purpose remains the same:

• kernel.numa_balancing_scan_delay_ms: The delay after a start, fork, or exec until the first scan begins.
• kernel.numa_balancing_scan_period_min_ms / _max_ms: Minimum and maximum limits for the scan interval per process address range.
• kernel.numa_balancing_rate_limit_mb: Upper limit per time slot for page migrations to conserve memory bandwidth.
• kernel.numa_balancing_scan_size_mb: Amount of memory marked per scan pass (if available).

In latency-critical setups, I take a conservative approach by increasing the minimum and maximum periods and lowering the rate limits, rather than immediately disabling the automatic mode. This often provides a good middle ground: fewer hint faults, fewer migrations, but still enough responsiveness to actual misplacements.

# Examples (temporary, until reboot)
sysctl -w kernel.numa_balancing_scan_period_min_ms=60000
sysctl -w kernel.numa_balancing_scan_period_max_ms=240000
sysctl -w kernel.numa_balancing_rate_limit_mb=64

In-Depth Metrics and Diagnostics

To make well-informed decisions, I review metrics that directly reveal the underlying mechanism. I regularly use three sources:

• numastat: Ratio of local to remote accesses system-wide and per process.
• /proc//numa_maps: Distribution of a process's memory pages across nodes, including flags such as active, file, and anon.
• /proc/vmstat: Counters such as numa_hint_faults, numa_hint_faults_local, and numa_pages_migrated indicate whether the balancer is active and whether it Success has.

# Overview by Process
numastat -p 

# Detailed View: Which regions are located where?
grep -E 'anon|file' /proc//numa_maps | head

# Kernel-Wide View of AutoNUMA Activity
grep -E 'numa_(hint_faults|pages_migrated)' /proc/vmstat

I look for trends in the results: Is the percentage of local requests rising steadily? Are hint faults decreasing at the same time? If so, a good layout is proving effective. If the local share remains flat despite many migrations, I’m more likely to waste cycles. For latency targets, I also check the 95th and 99th percentiles of response times; small gains in the mean can be offset by Jitter be covered.

Workload Profiles: What Typically Works

Based on real-world experience, certain patterns have emerged regarding when AutoNUMA is usually helpful and when it isn't:

• JVM services and application servers: They often benefit as long as no strict thread-pinning strategy is in place and no aggressive custom NUMA logic is active. Some runtimes offer NUMA options; if I use these strictly, I reduce the automatic behavior or disable it altogether.
• Relational databases: With variable loads and mixed caches, the automatic balancing often works well. However, if I enable dedicated pinning (worker-to-node, shared buffers strictly distributed), I disable the balancing to ensure clean reproducibility.
• In-Memory Stores and Caches: A large, hot working set benefits from local placement. If the instance is single-threaded or strictly pinned, I prevent unnecessary migrations by disabling them.
• HPC/MPI and scientific codes: In most cases, there are clear placement and binding rules (OpenMP/numactl). Here, predictability is more important than automation—I disable NUMA balancing.

Virtualization: vNUMA, Pinning, and Live Migration

When considering the interaction between host and guest, I take both levels into account:

• If the vNUMA topology in the guest matches the host's physical NUMA topology, the guest balancer can make sound decisions. If there is a deviation from this, „incorrect neighborhoods“ arise, which AutoNUMA can only compensate for to a limited extent.
• If I pin vCPUs to host CPUs and bind guest memory to specific nodes, that constitutes an explicit policy—I reduce or disable AutoNUMA at this VM level to prevent duplicate migrations.
• After live migrations, I notice a warm-up phase: Hint faults increase until a new equilibrium is established. During this time, I plan for buffers to Latency-Tips.

On densely populated virtualization hosts, where instances start and stop and cgroups shift the load, the host’s automated mechanisms often provide a net benefit. For dedicated VMs that are sensitive to „noisy neighbors,“ I encapsulate resources properly and set the rules statically.

Pragmatic Target Values and Acceptance Criteria

I'll define what „good“ means up front so I don't end up spending forever fine-tuning:

• General Services: 70–85% local accesses are often sufficient if the variance remains low.
• Latency SLAs: Target >90% locally, clear upper limits for the hint-fault rate, and stable 99th percentiles.
• Bandwidth-intensive: Migrations must not saturate the storage channels—adjust rate limits and time periods accordingly.

I document these thresholds and analyze A/B runs across multiple load phases. I don't make a decision until the results are reproducible.

Troubleshooting checklist

• Sudden latency spikes: Check whether THP migrations or spikes in `numa_hint_faults` correlate. Countermeasure: Increase scan periods, set THP to `madvise/never`, or disable balance if necessary.
• Little effect despite activation: Are the threads heavily pinned, or are there fixed memory policies in place? If so, the automatic mechanism conflicts with the specifications.
• High migration rate, yet still a large number of remote accesses: Check and increase the rate limit; alternatively, stabilize the workload (thread pinning, keep caches warm).
• Unclear measurement values: Use the per-process view with `numastat -p` and `/proc//numa_maps`, rather than just the overall system values.

Details That Are Often Overlooked

• Page-cache-intensive workloads: AutoNUMA is most effective for anonymous pages. Users whose workloads are primarily I/O-bound should not expect miracles from balancing.
• Cgroups and cpusets: `cpuset.mems` restricts which nodes a group is allowed to access. This is a strict boundary within which the automated system operates.
• Memory Hot-Plug/Node Offlining: Dynamic topologies change distances; after changes, it is advisable to run another test and, if necessary, adjust the scan parameters.

Briefly summarized

For general server workloads, I leave the automatic mode on because it comes close to the desired result without any manual intervention Data which results in active cores. For real-time, HFT, manual pinning, or fixed policies, I disable them to avoid overhead and jitter. During testing, I work iteratively: measure, decide, repeat validate. I keep the configuration simple, document every change, and monitor the results using reliable metrics. That way, I take full advantage of NUMA hardware without unnecessary Risks to take into account.