Skip to content 
Memory fragmentation In web hosting, PHP-FPM and MySQL slow down even though there appears to be enough RAM available, because the memory is broken down into many small blocks and larger allocations fail. I demonstrate in a practical way how fragmentation increases the cost of requests, triggers swapping, and why targeted tuning of PHP and MySQL visibly improves loading times, reliability, and scalability.
Key points
- PHP-FPM Recycle: Restart processes regularly via pm.max_requests
- Buffer Dosing: Keep MySQL per-connection buffer conservative
- Swap Avoid: Reduce swappiness, consider NUMA
- tables Maintain: Check for data-free items, optimize as needed
- Monitoring Use: Identify trends early and act
What does memory fragmentation mean in everyday hosting?
In hosting, it hits Fragmentation long-running processes that constantly request and release memory, creating gaps in the address space. Although the total amount of free RAM appears to be large, there are no contiguous blocks available for larger allocations, which slows down allocation attempts. I see this in PHP‑FPM workers and in mysqld, which always appear more „bloated“ after hours. The effect makes each request minimally more expensive and noticeably increases response times under load. This causes peaks such as sales campaigns or backups to become a stumbling block, even though the CPU and network remain unremarkable.
Why PHP-FPM creates fragmentation
Each PHP‑FPM worker loads code, plugins, and data into its own Address space, handles a wide variety of requests and leaves scattered gaps when releasing memory. Over time, processes grow and free up memory internally, but not necessarily to the operating system, which increases fragmentation. Different scripts, import jobs, and image processing amplify this mix and lead to changing allocation patterns. I observe this as a creeping increase in RAM, even though load and traffic appear to be constant. Without recycling, this internal fragmentation slows down allocation and makes planning difficult when visitor numbers are high.
Noticeable consequences for loading times and reliability
Fragmented processes generate more Overhead in memory management, which manifests itself in slower admin backends and hesitant checkouts. WordPress shops and large CMS instances in particular respond sluggishly when many simultaneous requests encounter fragmented workers. This results in timeouts, 502/504 errors, and increased retries on the NGINX or Apache side. I see such situations in metrics such as response time spikes, rising RAM baselines, and sudden increases in swap usage. Ignoring this wastes performance, degrades the user experience, and increases the abandonment rate in critical funnels.
Setting PHP-FPM correctly: limits, pools, recycling
I focus on realistic Limits, separate pools, and consistent recycling to curb fragmentation. I set pm.max_requests so that workers restart regularly without disrupting current visitors. For traffic profiles with load peaks, pm = dynamic is often more suitable, while pm = ondemand saves RAM on quiet sites. I deliberately keep the memory_limit per site moderate and adjust it with real scripts in mind; the topic provides an introduction. PHP memory limit. In addition, I separate heavily loaded projects into separate pools so that one memory hog does not affect all sites.
OPcache, preloading, and PHP allocator at a glance
To reduce fragmentation in the PHP process, I rely on a cleanly dimensioned OPcache. A generous but not excessive opcache.memory_consumption and sufficient interned strings reduce repeated allocations per request. I monitor hit rate, waste, and remaining capacity; if waste increases over time, a planned reload is better than allowing workers to grow uncontrollably. Preloading can keep hot code stable in memory and thus smooth out allocation patterns, provided the code base is prepared accordingly. In addition, I pay attention to the Allocator selectionDepending on the distribution, PHP‑FPM and extensions work with different malloc implementations. Alternative allocators such as jemalloc noticeably reduce fragmentation in some setups. However, I only roll out such changes after testing them, as debugging, DTrace/eBPF profiling, and memory dumps react differently depending on the allocator.
For memory-intensive tasks such as image processing or exports, I prefer separate pools with tighter limits. This prevents the main pool from growing uncontrollably and keeps fragmentation isolated. I also limit memory-hungry extensions (e.g., via environment variables) and use backpressure: requests that require large buffers are throttled or moved to asynchronous queues instead of inflating all workers at once.
Understanding MySQL memory: buffers, connections, tables
With MySQL, I distinguish between global Buffer such as the InnoDB buffer pool, per-connection buffers, and temporary structures, which can grow with each operation. Excessively high values lead to exploding RAM requirements and increased fragmentation at the OS level when connection loads are high. In addition, tables become fragmented by updates/deletes, leaving behind data_free portions that make it harder to utilize the buffer pool. I therefore regularly check the size, hit ratios, and number of temporary disk tables. The following overview helps me to accurately identify typical symptoms and weigh them against countermeasures.
| Symptom | Probable cause | Measure |
|---|---|---|
| RAM steadily increases, swap begins | Buffer pool too large or too many per-connection buffers | Limit pool to appropriate size, reduce via connection buffer |
| Many slow sorts/joins | Missing indexes, excessive sort/join buffers | Check indexes, keep sort/join conservative |
| Large amount of data_free in tables | Heavy updates/deletes, pages fragmented | Targeted OPTIMIZE, archiving, streamlining schema |
| Peaks in temporary disk tables | tmp_table_size too small or unsuitable queries | Raise values moderately, restructure queries |
MySQL Memory Tuning: Choose sizes instead of overloading
I select the InnoDB buffer pool so that the Operating system enough breathing room for file system cache and services, especially on combined servers with web and DB. I scale per-connection buffers such as sort_buffer_size, join_buffer_size, and read_buffer conservatively so that many simultaneous connections do not lead to RAM storms. I set tmp_table_size and max_heap_table_size so that unimportant operations do not require huge in-memory tables. I find further adjustment options under MySQL performance Helpful food for thought. The bottom line remains: I'd rather set things a little tighter and measure than blindly turn it up and risk fragmentation and swapping.
InnoDB in detail: Rebuild strategies and pool instances
To keep MySQL „more compact“ internally, I plan to regularly Rebuilds for tables with a high proportion of writes. A targeted OPTIMIZE TABLE (or an online rebuild via ALTER) merges data and indexes and reduces Data_free. I choose time slots with low load, as rebuilds are I/O-intensive. The option innodb_file_per_table I keep it active because it allows table-controlled rebuilds and reduces the risk of individual „problem children“ fragmenting the entire tablespace file.
I use several Buffer pool instances (innodb_buffer_pool_instances) in relation to pool size and CPU cores to relieve internal latches and distribute accesses. This not only improves parallelism, but also smooths out allocation patterns in the pool. In addition, I check the size of the redo logs and the activity of the purge threads, because accumulated history can tie up memory and I/O, which increases fragmentation at the OS level. It is important to change settings gradually, measure them, and only keep them if latencies and error rates actually decrease.
Avoiding swap: Kernel settings and NUMA
As soon as Linux actively swaps, response times increase by orders of magnitude, because RAM accesses become slow I/O. I significantly lower vm.swappiness so that the kernel uses physical RAM for longer. On multi-CPU hosts, I check the NUMA topology and activate interleaving if necessary to reduce uneven memory utilization. For background information and hardware influence, I find the perspective on NUMA architecture. In addition, I plan for security reserves for page cache, because a starved cache accelerates the fragmentation of the entire machine.
Transparent Huge Pages, Overcommit, and Allocator Selection
Transparent Huge Pages (THP) can lead to latency spikes in databases because merging/splitting large pages happens at inopportune times. I set THP to „madvise“ or disable it if MySQL responds too slowly under load. At the same time, I note that Overcommit: With an overly generous vm.overcommit_memory configuration, you risk OOM kills precisely when fragmentation makes large contiguous blocks scarce. I prefer conservative overcommit settings and regularly check kernel logs for signs of memory pressure.
The Allocator selection At the system level, it's worth taking a look. glibc‑malloc, jemalloc, and tcmalloc behave differently in terms of fragmentation. I always test alternatives in isolation, measure RSS history and latencies, and only roll out changes if the metrics remain stable under real traffic. The benefits vary greatly depending on the workload, extension mix, and OS version.
Recognizing fragmentation: metrics and indicators
I pay attention to slow increases baselines with RAM, more 5xx responses under load, and delays in admin actions. A look at PM statistics from PHP-FPM shows whether children are reaching limits or living too long. In MySQL, I check hit ratios, temporary tables on disk, and data_free per table. At the same time, OS metrics such as page faults, swap-in/out, and memory fragmentation indicators help, depending on the kernel version. By combining these signals, you can recognize patterns early on and take planned action.
Deepening monitoring: How I combine signals
I correlate Application metrics (p95/p99 latencies, error rates) with process metrics (RSS per FPM worker, mysqld memory) and OS values (Pagecache, Slab, Major Faults). In PHP‑FPM, I use the status interface for queue lengths, active/spawned children, and worker lifetime. In MySQL, I monitor Created_tmp_disk_tables, Handler_write/Handler_tmp_write, and Buffer Pool Misses. In addition, I look at process maps (pmap/smaps) to find out if many small arenas have been created. What is important to me is the trend orientationIt is not the individual peak, but rather the gradual shift over hours/days that determines whether fragmentation becomes a real danger.
Practical routine: Maintenance and data management
I tidy up regularly Data on: expired sessions, old logs, unnecessary revisions, and orphaned caches. For highly volatile tables, I plan specific OPTIMIZE windows to merge fragmented pages. I distribute large import jobs or cron waves over time so that not all processes request maximum buffers at the same time. For growing projects, I separate the web and DB early on to isolate memory-hungry patterns. This discipline keeps the working memory more coherent and reduces the risk of burst latencies.
Calculate sizes accurately: Dimension limits and pools
I determine pm.max_children based on the actual RAM available for PHP. To do this, I measure the average RSS of a worker under real load (including extensions and OPcache) and add safety margins for peaks. Example: On a 16 GB host, I reserve 4-6 GB for the OS, page cache, and MySQL. That leaves 10 GB for PHP; at 150 MB per worker, that theoretically results in 66 children. In practice, I set pm.max_children to ~80-90% of this value to leave headroom for spikes, i.e., around 52-58. pm.max_requests I choose to recycle workers before noticeable fragmentation occurs (often in the range of 500–2,000, depending on the code mix).
For MySQL, I calculate the Buffer Pool from the active data size, not from the total DB size. Important tables and indexes should fit in, but the OS cache needs space for binlogs, sockets, and static assets. For the connection buffer, I calculate with the maximum realistic parallelism. If 200 connections are possible, I don't dimension in such a way that several gigabytes per connection explode in total, but set hard limits that are not swap-prone even during peak times.
Decoupling queues, image processing, and background jobs
Many fragmentation problems arise when part-time jobs the same pools as front-end requests. For exports, crawls, image conversions, or search index updates, I use separate FPM pools or CLI jobs with clear ulimitI also limit image operations with GD/Imagick by setting appropriate resource limits so that individual large conversions do not „chop up“ the entire address space. I schedule jobs at different times and give them their own concurrency limits so that they do not overwhelm the front-end path.
Containers and virtualization: Cgroups, OOM, and balloon effects
I observe in containers Memory Limits and the OOM killer in particular. Fragmentation causes processes to run at cgroup limits earlier, even though there is still RAM available on the host. I set pm.max_children strictly to the container limit and keep enough reserve to cushion peaks. I avoid swapping within containers (or on the host) because the additional indirection increases latency peaks. In VMs, I check ballooning and KSM/UKSM; aggressive deduplication saves RAM, but can cause additional latency and distort the fragmentation picture.
Short checklist without bullet points
First, I set a realistic memory_limit per site and observe how peak usage behaves over several days. I then tune PHP‑FPM with appropriate pm values and a sensible pm.max_requests so that fragmented workers run as planned. For MySQL, I focus on an appropriate buffer pool size and conservative per-connection buffers instead of blanket increases. On the kernel side, I lower swappiness, check NUMA settings, and keep reserves free for the page cache. Finally, I evaluate tables with data_free anomalies and plan optimizations outside of day-to-day business.
In summary: What matters in business
I achieve the greatest effect against memory fragmentation by consistently recycling PHP-FPM workers, moderate limits, and clean pools. MySQL benefits from reasonable buffer pool and per-connection buffer sizes, as well as tidy tables. I proactively avoid swapping by paying attention to swappiness and NUMA and reserving free RAM for the file cache. Monitoring reveals creeping patterns before users notice them and allows for calm, planned interventions. Those who use these levers in a disciplined manner keep PHP and MySQL faster, more reliable, and more cost-efficient without immediate hardware upgrades.
