Understanding the Redis Replication Backlog: PSYNC, Size, and HA Limits

The Redis Replication Backlog helps determine whether, after a connection failure, a replica will only retrieve missing changes or receive the entire dataset again. By sizing the buffer based on the actual replication volume, you can avoid unnecessary full synchronizations. However, this also requires that the replication history, storage budget, and operational workflows align: A large backlog is no substitute for persistence or a robust failover strategy.

What the Backlog Actually Stores

At Redis Replication The primary processes changes to the data set and transmits a continuous stream of commands to its replicas. This includes not only values written directly by clients; expired or overwritten keys can also trigger changes that must be propagated. The backlog keeps a limited, recent subset of this replication stream in memory. Therefore, it does not contain an additional complete copy of the database, nor is it an archive of write operations of any age.

During normal operation, the replicas follow the current data stream. If a connection is interrupted, the history on the primary continues to grow. After reconnection, the replica attempts to resume from where it left off. The key factor here is whether the required bytes are still available. If that is the case and the replication history matches, Redis can fill in the gap. The data already present on the replica does not need to be completely replaced for this to happen.

The benefits are particularly evident during brief network outages, connection switches, and scheduled maintenance. A full synchronization of a large dataset consumes transmission capacity and computing power; depending on the configuration, additional storage and media loads may also arise. The backlog can reduce this effort, but it cannot account for every type of interruption. A process restart or a changed history requires a different approach than a briefly disconnected TCP connection.

When PSYNC Is Sufficient and When a Full Resync Is Necessary

A Partial resynchronization with PSYNC requires two related pieces of information: the replication ID and the offset. The ID identifies a specific data history. The offset describes a byte position within the replication stream. Therefore, two offsets of the same size from different histories are not automatically comparable. Conversely, a small backlog within the same history may already lie outside the available backlog if its capacity is limited.

Simply put, when the Replica reconnects, it reports how far it has progressed. The Primary checks whether it can provide the data needed next. If the history is unknown or the required section is missing, a Full Resync required. In this process, the replica receives a complete dataset and then the changes that occurred during the synchronization. Depending on the configuration, the transfer can take place with or without an intermediate RDB step to a storage medium.

After a failover, a full resync is not always unavoidable. A promoted replica can also retain the previous replication ID and its valid offset range. This allows additional replicas to resume from the known history under the right conditions. However, this does not provide any guarantee for planning purposes: The relevant range must still be available, and the actual reconnection must match the stored IDs.

Measure the replication rate instead of estimating the database size

The sheer size of the dataset is enough to Backlog Sizing That's not the case. A large database that is primarily read-intensive can generate very little replication traffic. In contrast, a small cache with frequently changing values and many expiration events can continuously transfer significant amounts of data. Even a fixed number of operations per second does not adequately describe the required storage: small key changes and large value overwrites do not result in the same number of bytes.

A practical approximation can be derived from the rate of increase over time of master_repl_offset. Record the value twice on the same primary and divide the difference by the number of seconds that have elapsed. Be sure to check the replication ID as well. After a role change or a restart, do not simply subtract two unrelated measurement points from each other. Furthermore, a single measurement interval provides only an average rate within that interval, not a permanently guaranteed upper limit.

The following query reads diagnostic information. It does not modify any Redis configuration. Run it with the connection and authentication options required for your environment. The call shown below uses the default connection from redis-cli; a different host, port, or TLS connection must be explicitly configured.

redis-cli INFO replication

Monitor across different load phases, such as during normal daily operations, during imports, and during major cache refreshes. Document both typical rates and brief spikes. If many changes occur due to expiring keys, the internal article provides further insight into this topic. Analyzing Redis Key Expiration. This connection is important for planning because not every relevant writing prompt arises directly from a new user request.

Calculate the backlog size in a transparent manner

As Planning Approximation You can multiply the relevant replication rate by the duration of the interruption to be bridged and then add a reasonable buffer. The duration should not only account for the actual network outage. Detection, reconnection attempts, and restoring the connection path can also take time. What constitutes an appropriate buffer depends on observed fluctuations and the desired operational goal, not on a universal percentage.

Here is a deliberately simplified calculation example: For a given load phase, assume 12 MiB per second. The connection could be down for 90 seconds; an additional 30 seconds are factored in as a time buffer. This results in 12 MiB/s × 120 s = 1.440 MiB, or about 1.41 GiB. These numbers are estimates provided for illustrative purposes only and do not represent a Redis benchmark. Before deploying in production, they must be replaced with actual measurements from your application.

The reverse calculation helps in assessing an existing buffer. A fully filled backlog of 256 MiB corresponds, at a constant rate of 12 MiB per second, to approximately 21 seconds of history. At 2 MiB per second, it would be approximately 128 seconds. In a real-world application, however, the rates fluctuate. Such a range is therefore a snapshot and does not guarantee that every disruption of this duration can be partially synchronized.

Also check whether the replica can catch up faster after reconnecting than new changes occur. Increasing the history does not resolve a persistently slow network or a persistently overloaded receiver. If the backlog persists or continues to grow, the cause must be investigated. Otherwise, simply allocating more storage will merely postpone the problem and may put the entire system under storage pressure.

How to Change Redis Configuration in a Clear and Controlled Way

The Parameters repl-backlog-size and repl-backlog-ttl control different things. The first describes the intended backlog size. The second determines, on the primary, after how long without connected replicas the backlog can be released. It is No maximum interruption duration for PSYNC. As long as the buffer is overwritten or some other requirement is not met, a long TTL alone is not enough.

The following lines are commented-out defaults from the Redis 7.2.0 configuration template. The comment characters have been intentionally left in place. Simply copying these lines will not enable any settings; furthermore, the values listed are not general capacity recommendations for production systems.

# Redis 7.2.0: auskommentierte Vorgaben aus redis.conf
# repl-backlog-size 1mb
# repl-backlog-ttl 3600

With repl-backlog-ttl 0 Time-controlled release is disabled after all replicas have been disconnected. This does not preserve an unlimited history: The existing buffer can still be overwritten by new replication data. Furthermore, this does not ensure persistence across arbitrary process restarts. Therefore, carefully evaluate whether the additional memory consumption is appropriate given the expected reconnection behavior.

Before making a change, you should check the actual values in effect and clarify the deployment process. A container environment variable, a managed configuration file, and a setting changed at runtime are not the same thing. If an instance is recreated later, only the adjustments made at runtime may be lost. The following queries are read-only; however, they still require the appropriate access permissions.

redis-cli CONFIG GET repl-backlog-size
redis-cli CONFIG GET repl-backlog-ttl

With Managed Redis, the provider can restrict access to CONFIG restrict or manage settings through a dedicated interface. This is not a reason to bypass security measures. In that case, use the approved management channels and document the selected size along with the underlying replication volume. The change plan should also include the previous values and a realistic fallback plan.

Monitoring: Which Values Go Together

For the Monitoring Redis Replication A single green connection status is not enough. A link may be reconnected while the replica is still catching up or is in the process of loading a complete data set. Conversely, a brief connection interruption does not necessarily have to be critical if the history and catch-up capacity are sufficient. Therefore, evaluate the connection, synchronization status, offset trend, and available history together.

The fields of a INFO-Responses may vary between Redis versions and between primary and replica nodes. Therefore, any evaluation should explicitly handle missing fields rather than silently interpreting them as null or as an error-free state. The labels master and slave For compatibility reasons, they continue to appear in field names; they must not be freely translated in the executable code.

The internal guidelines are used to interpret the backlog Analyzing the Redis Replication Offset a useful addition. In your ongoing monitoring, you should look at historical trends, not just two manually recorded numbers. A widening gap requires a different response than a gap that is steadily narrowing after a brief outage.

Meaningful alerts are based on your operational goals: How long can a replica remain unreachable? How quickly must it catch up? What frequency of full synchronizations is considered unusual? Rigid thresholds that don’t take load profiles and data volumes into account often lead to unnecessary alerts or fail to detect actual performance degradation. In addition to replication metrics, keep an eye on RAM usage, network utilization, and indications of process restarts.

Understanding Storage Budgets and Slow Replicas

The backlog is only part of the overall Redis Storage Requirements. In addition, there are data sets, administrative structures, client buffers, and—depending on the operational state—additional memory during persistence or synchronization operations. Therefore, do not allocate the entire available RAM for user data plus a precisely calculated backlog. The necessary reserve must be determined based on the specific environment and the peak loads that occur there.

Since Redis 7.0, the replica buffer and replication backlog share memory. The INFO documentation therefore notes, among other things, that mem_clients_slaves can be zero if the replica buffers do not exceed the backlog allocation. This does not mean, however, that replication does not consume memory. Consider the values specified for this purpose, such as mem_replication_backlog and mem_total_replication_buffers Keep the context in mind and don't blindly add overlapping quantities.

A common diagnostic mistake is to assume that every failed synchronization is caused by a large backlog. Slow replicas, limited network bandwidth, or exceeded output buffer limits may have other causes. The parameter client-output-buffer-limit replica applies to the client class in question and must not be confused with repl-backlog-size be treated as equivalent. Before changing limits, review logs, version documentation, and the expected impact on other connections.

Why a large backlog does not guarantee high availability

The backlog improves reconnection but does not make the default asynchronous replication lossless. A primary may have already acknowledged a write operation to the client before a replica has processed it. If the primary fails during this period, the operation in question may not be present on the replacement system selected later. The buffer size alone does not eliminate this Risk of Data Loss During Failover not.

Also WAIT does not transform a Redis topology into a system with guaranteed strong consistency. The command can wait for acknowledgments from replicas; actual data safety continues to depend on other factors, particularly persistence and failover behavior. Similarly, min-replicas-to-write and min-replicas-max-lag under their respective conditions, accept new write operations without automatically and permanently saving each individual operation across multiple instances.

For High Availability You therefore need a comprehensive plan that addresses the following: acceptable data loss, permissible downtime, persistence, fault detection, selection of the new primary, and recovery. Sentinel or Redis Cluster can handle tasks other than those of the backlog. Simply increasing the size of a buffer while leaving all other assumptions unchanged does not result in a robust recovery plan.

Test changes and narrow down recurring issues

Start a controlled test in an isolated environment with a comparable Redis version and a predictable write load. Before the interruption, record the replication IDs, offsets, backlog, and memory usage. Then simulate a limited connection interruption without making any unchecked changes to production firewalls or processes. After reconnection, observe whether partial or complete synchronization occurs and how long it takes to catch up.

For each test, change only one relevant variable at a time, if possible. If the backlog, write load, and network conditions all change at the same time, it becomes nearly impossible to pinpoint the cause of the effect. Repeat the test with different interruption durations and different load phases. This way, a single successful reconnection can be used to form a reliable assessment of the system’s behavior. The observed limits should be documented, without inferring from them a guarantee for every future disruption.

If full resyncs occur repeatedly, first check whether the history is compatible at all. Next, check the remaining bytes, the time without a replica, indications of restarts, and the actual catch-up speed. A buffer that is too small is a possible cause, but not the only one. It is particularly important to distinguish between a single, prolonged gap and a replica that consistently falls behind, even when a connection is active.

Ultimately, the correct configuration is the one that covers your defined downtime window under realistic load conditions while leaving enough memory for the rest of the operation. Keep a record of the measurement basis, configuration source, and test date together. After major changes to write behavior, topology, or the Redis version, the sizing should be reevaluated. This ensures that the backlog remains a well-founded operational decision rather than a figure adopted once and for all.