Administration

iotop in Everyday Hosting: Identifying Hard Drive Load Under Linux

With iotop hosting, I can find the process that’s slowing down my hard drives and causing delays in load times, database queries, or backups in a matter of seconds. I use the tool specifically when CPU resources are available but websites are responding slowly and the I/O wait time is rising.

Key points

Real time: See active read/write accesses per process immediately
Polluter: Identify the service that is filling the I/O queue
Context: Categorizing spikes from Cron, backups, and logging
Combination: Use iostat and vmstat to assess the situation
Practice: Carry over findings from maintenance windows and limits

Why I start iotop first when the server seems sluggish

A sluggish server with spare CPU capacity is just begging for a look at the Hard Drive Load. That’s exactly where iotop shines, because I can see, for each process, who is currently reading or writing. A single log file, an import, or an indexing operation can slow down response times even without a hardware failure. I detect such patterns in real time and, when in doubt, terminate the offending task before users abandon the process. This quick focus saves me time when Initial diagnosis and prevents blind flights.

Installation and Startup: The 30-Second Version

Setup takes just a few steps and requires root privileges or the necessary Capabilities. On Debian/Ubuntu, I install iotop using apt install iotop, on RHEL/Alma using yum install iotop respectively dnf install iotop. For the live feed, I'll call iotop open, filter by -o only active processes, and continue with -d 1 a short interval. Example: iotop -o -d 1 shows me who's braking right now. A dry batch output with -b helps me take notes in Logs.

Quick-Start Commands I'm Memorizing

I decide which mode I need based on the situation, and I stay pragmatic and quick about it. iotop -o Shows only truly active processes; this reduces noise. iotop -a It accumulates I/O since startup and helps with jobs that run for a long time. iotop -P summarizes threads at the process level, which provides an overview of Services sharpens. iotop -b -qq -d 2 -n 30 I write to a file when I want to record spikes over a short time window. These little switches give me the necessary Control, without having to go through complicated setups.

Understanding Output: Columns and Their Meanings

To make a good decision, I need clear criteria for determining which values are critical and which are within normal ranges. On iotop, I mainly look at the columns for read, write, and I/O percentages. The IO% column shows me the percentage of time a process spends waiting for I/O in the kernel. SWAPIN% should almost always remain at zero; if it increases, the system becomes bogged down by Outsourcing. With COMMAND, I can quickly see which script or service is causing the issue and whether I need to take action.

Column	What it shows	What I pay attention to
PID / USER	Process ID and User	Who is buying them, and with what? Rights?
DISK READ / WRITE	Current throughput per process	Consistently high MB/s over several seconds are suspicious.
SWAPIN%	Percentage of Time Spent on Swapping	Values between 0 and 1% indicate pressure in the Memory there.
IO%	Percentage of Time Spent in I/O Wait States	High IO% with low MB/s = small, synchronous Writes.
PRIO	Priority/Nice Value	Background jobs, using `ionice` if necessary steam.
COMMAND	Call, including path	Quickly check whether it's log rotation, a backup, or a Import is.

Diagnostic workflow: Run iotop first, then verify with iostat/vmstat

I start with iotop to identify the culprit and back up my findings with system metrics. A high IO% value for a process tells me that this particular service is using the disk. Then I check with iostat -x 1, whether the drive is showing high utilization and latency is increasing. A look at vmstat 1 tells me whether paging or the run queue is distorting the picture. If you want to dive deeper, here's a concise introduction to Analyze I/O Wait, which I noticed when comparing the Metrics helps.

Common Causes in Everyday Hosting and How I Deal with Them

A growing log file is a classic example that fills the I/O queue with many small sync writes and slows down response times. Database workloads with poorly chosen indexes create erratic patterns and slow things down due to random Accesses. Backups during peak hours cause spikes that noticeably impact other services. A search indexing task or a cron job at the wrong time is enough to delay requests. I spread out such jobs, set appropriate log levels, and allow hard writes in Maintenance window run.

Organize Schedules, Cron Jobs, and Logging Neatly

I schedule heavy workloads for off-peak times and manage them using Nice and Ionice values. For backups, I use ionice -c2 -n7, so that interactive processes take priority. I adjust the log level when files grow too quickly and put a strain on the file system. I briefly monitor tasks started overnight in the morning using iotop and rely on logs from batch mode. If you want to see latency trends over time, you can check out Measuring Disk Latency orient and the Baselines Tighten.

SSD, NVMe, and Queue Depth: Why Throughput Alone Isn't Enough

An NVMe drive boosts IOPS, but many small sync writes still cause gaps in response behavior. That’s why I evaluate not only MB/s, but also IO% and the typical request size. When the queue depth is maxed out, requests pile up and latency increases noticeably. This is often evident with iotop, even though the raw throughput looks fine. If you want to delve deeper into this topic, check out the NVMe Queue Depth and sorts the Queues neatly.

Practical Tweaks: Small Adjustments with Immediate Results

I start with the obvious: check the database’s cache hit rate, add indexes, and configure the write-ahead log properly. For files, I set appropriate mount options and enable `noatime` if the workload profile allows it. I evaluate journaling options based on risk, without compromising data security. For backup tools, I choose options that prioritize large, sequential writes. Each of these changes reduces the Friction and alleviates bottlenecks before they affect users.

Automate and Document: iotop in Batch Mode

For recurring spikes, I write iotop output to a file and analyze it afterward. The command iotop -b -o -qq -d 2 -n 120 > /var/log/iotop.log records four minutes without the TUI frame. I combine this with a timestamp prefix or enable log rotation so the files remain manageable. Later, I filter for a conspicuous process name and check the time window. This is how I identify recurring Tips and use that to come up with specific tasks.

Permissions, Kernel Options, and Containers: What I Clarify Up Front

iotop displays all the necessary details only with root privileges or CAP_SYS_ADMIN, which I deliberately use for quick checks. The kernel must provide task stats and accounting functions, which common distributions enable by default. In containers, I often only see processes within the namespace, which limits the view. For cgroups, I also use tools that examine the group as a whole. This helps me understand what iotop is showing and where I need additional Insights need.

A Fine-Tuned Approach Instead of a Sledgehammer: IO-Scheduler, ionice, and Limits

With ionice I reduce the priority of background jobs and give interactive services more breathing room. At the system level, I check whether the I/O scheduler is appropriate for the workload type—for example, BFQ for interactive workloads or MQ variants for NVMe. Rate limits in backup tools protect the rest of the system from side effects. For write-intensive plugins, I implement caching strategies to reduce the load on the database. These steps take very little time but yield noticeable Rest during hectic periods.

Taking a Closer Look: The Limits Inherent in iotop

I always interpret iotop results in context. Not every high IO% value actually means “the disk is full.” Buffered writes first end up in the page cache and are flushed asynchronously by kernel threads (e.g., the write-back worker). Then, in iotop, I might see seemingly harmless MB/s for the process causing the issue, while a kworker or the journaling thread handles the actual load. Encrypted stacks (dm-crypt/LUKS), FUSE-based file systems, or overlay file systems in containers also obscure these mappings. So if only kernel threads are listed at the top, I use the COMMAND and timestamp to determine which user task wrote the data shortly before and where the data is flowing.

With NFS or distributed file systems, a local view is often not enough. While iotop shows me wait conditions, the cause may actually lie on the network or server side. In such cases, I correlate the local metrics with storage latencies or system metrics before hastily restarting services or setting limits.

File Systems and Journaling Options in Everyday Use

I take the characteristics of the file system into account because they shape the iotop images. Under ext4, the journal mode and commit interval influence how “spiky” writes appear: data=ordered is a good standard, writeback increases throughput at the expense of consistency guarantees and journal Makes writes consistent, but more expensive. XFS scales well with many parallel threads and is suitable for large files and high concurrency. Btrfs incorporates copy-on-write, checksums, and, if applicable, compression—which helps with read workloads but can slow things down when there are many small sync writes.

I set the mount options intentionally: noatime or relatime reduce unnecessary metadata writes. barrier/nobarrier I evaluate this solely in terms of the hardware's write cache security. commit=-Intervals determine how often metadata is committed—a higher value smooths out spikes but increases the window of potential losses in the event of a crash. I always evaluate these settings in terms of risk versus response time and test them during maintenance windows.

Understanding the Storage Stack: RAID, LVM, and Caches

I look not only at the process itself, but also at the underlying infrastructure. A RAID 5/6 penalizes small, random writes via read-modify-write, which is evident in iotop as high IO% with meager MB/s. Stripe sizes and alignment in LVM influence whether accesses occur neatly in blocks or are fragmented. Write-back caches on controllers provide a noticeable speed boost, but are only justifiable with a reliable power supply. NVMe with a multi-queue stack delivers low latencies—as long as queue depths, schedulers, and IRQ distribution are properly configured. I therefore check whether the workload matches the storage’s geometry before making any changes to the service itself.

Kernel parameters that smooth the I/O load

When I/O bursts noticeably affect users, I make targeted adjustments to the writeback mechanism:

vm.dirty_bytes / vm.dirty_background_bytes: absolute limits that determine when processes (or flushers) begin writing. I prefer bytes over percentages to manage large RAM systems.
vm.dirty_writeback_centisecs and vm.dirty_expire_centisecs: Control the clock speed and “age” of the pages to be written—useful for spreading out spikes.
vm.swappiness: I keep this setting moderate to prevent unnecessary swapping under load (ideally, SWAPIN% should remain at 0).

I test these adjustments step by step. The goal is to stabilize user latency without sacrificing overall throughput reserves.

Targeted Database Stabilization

For MySQL/MariaDB, I check innodb_buffer_pool_size (cache hit rate), appropriate indexes, and effective flush strategies: innodb_flush_log_at_trx_commit and sync_binlog I choose the appropriate level based on the risk to mitigate commit paths. A value that is too small innodb_log_file_size creates unnecessary checkpoints and I/O spikes. I store temporary files on fast volumes when they're actually in heavy use.

With PostgreSQL, I smooth using checkpoint_timeout, max_wal_size and a sensible Autovacuum configuration. Place the WAL on a fast, consistent volume, don’t run checkpoints too aggressively, and relieve hotspots with indexes—this visibly reduces IO%. In both scenarios, the same rule applies: A single missing index often causes more chaos than any hardware limitation. I measure performance, use iotop to verify the DB process’s write activity, and then decide whether tuning or query optimization takes priority.

Understanding Containers and Cgroups

In container environments, I group processes using -P together to evaluate services rather than threads. iotop primarily shows me what’s visible in the namespace; on the host side, I aggregate by Cgroup when multiple pods/containers share the same volume. I use rate limits (e.g., via Cgroups) to contain “noisy” workloads without stopping them entirely. Overlay layers are worth noting: If a container writes extensively to its overlay, the copy-on-write behavior can result in small, expensive writes. In that case, I offload write paths to dedicated volumes or adjust the write intensity via ionice down.

Network Storage (NFS/Block Storage): When the Network Slows Down

When services access NFS or cloud block storage, I evaluate latencies in two ways: locally and remotely. iotop shows me that a process is waiting—but the cause could lie in the network path, in limits imposed by the remote storage, or in suboptimal mount options. Typical examples include a heavy metadata load on NFS home directories or very small sync writes to block volumes with an IOPS limit. In such cases, I adjust rsize/wsize (NFS), use larger, sequential writes, or distribute hotspots to local SSDs as a cache. It’s important to me not to look at MB/s in isolation: a few MB/s with high IO% indicate wait time, not throughput limits.

From Real Life: My 10-Minute Workflow

Minutes 1–2: iotop -o -d 1 Start, identify the culprits, determine whether read or write operations are dominant, and check IO% and SWAPIN%.
Minutes 3–4: iostat -x 1 In addition: Verify the plausibility of latencies, utilization, and queue depth.
Minute 5: If a specific batch is to blame, with ionice/nice slow down or take a short break.
Minutes 6–7: Classify the pattern (Cron? Backup? Indexing?) and note the schedule/limit.
Minutes 8–9: Check the file system and database context (journal/commit, indexes, flushing).
Minute 10: Start the batch trace (iotop -b -o -qq -d 2 -n 120) and keep track of to-dos.

Automation: Consolidate Batch Outputs

I summarize batch logs in a practical way to identify repetitions. A simple starting point is to tally the results by COMMAND line to see which ones were used most frequently and most intensively. Example: A short awk-Run can sum the measured WRITE/READ values by process name and list the top contributors. This gives me a ranking in seconds, without the need for complex pipelines. For longer-term comparisons, I set the log rotation interval to a short duration and keep the output formats consistent so that I can perform A/B comparisons weeks later.

Briefly summarized

I use iotop to identify, in real time, which service is clogging the I/O queue, and then check system metrics to see how heavily the drive is actually being utilized. Typical culprits include log growth, poorly scheduled cron jobs, database-heavy writes, or parallel indexing that runs at cross-purposes with traffic. With well-organized schedules, appropriate logging, ionice/Nice, and a few storage tweaks, I can reliably reduce wait times. It remains important to document patterns and translate the findings into concrete actions. This is how fast Troubleshooting A lasting speed boost for hosting setups of any size.