...
web hosting logo

NGINX Rate Limiting: Effective Protection Against Bot Traffic and Attacks

NGINX Rate Limiting stops automated requests, reduces peak load, and protects login, API, and form endpoints against bots and attacks. I'll show you how to define limits and apply them to Bot Protection applies this and uses it to develop a robust security strategy for high-traffic websites.

Key points

Essential These are the key points:

  • Rate limits Thwart malicious attacks and protect backend resources.
  • Burst/no delay handle legitimate spikes without blocking users.
  • zones Separate humans and bots with different limits.
  • Logging provides data to iteratively refine limits.
  • Integration Combined with WAF, DDoS protection, and monitoring, it enhances effectiveness.

Why Rate Limiting Stops Attacks Early

Attackers are banking on high Request Rates, to exploit login forms, overload APIs, or automatically scrape content. I therefore limit the number of requests per key—usually per IP address—and decide whether to throttle them, delay them, or respond with a 429 error. This way, I keep bot traffic away from the CPU, database, and application logic while allowing legitimate users to proceed. Particularly sensitive paths such as /login, /auth, /xmlrpc.php, or resource-intensive searches benefit greatly from this. The source for this method is the Nginx Documentation About the ngx_http_limit_req_module.

How the NGINX Module Works in Practice

The module operates according to the Leaky bucket-Principle: For each key, NGINX stores count values in a zone and compares them to the allowed rate. Typical keys include $binary_remote_addr for IP addresses, tokens for API keys, or values derived via a map. If a client consistently exceeds the rate limit and the burst buffer, NGINX rejects the request before it reaches the backend. This saves processing time and reduces latency for legitimate visitors. I configure the response to return a 429 Too Many Requests status code or, optionally, another Status code um.

Configuration: Explained Step by Step

I start with a zone in the HTTP section, set a moderate rate limit, and enable it specifically on sensitive paths. For short-term spikes, I define a burst, optionally with `nodelay`, to avoid abrupt rejections. Then I test it in staging and analyze the logs before deploying it to production. This way, I avoid the risk of unnecessarily blocking legitimate users. A concise example illustrates the Syntax tangible:

# http {}
limit_req_zone $binary_remote_addr zone=req_limit_per_ip:10m rate=10r/s;

server {
  location /api/ {
    limit_req zone=req_limit_per_ip burst=20 nodelay;
    limit_req_status 429;
  }

  location /login {
    limit_req zone=req_limit_per_ip burst=5;
    limit_req_status 429;
  }
}

Bot Protection with Zones and User-Agent Logic

IP-based limits are rarely sufficient against distributed botnets, so I segment traffic into zones: Humans are assigned more generous limits, while generic crawlers receive stricter ones. Using `map`, I evaluate user-agents, identify exempt bots like Googlebot, and assign them their own, closely monitored limits. For no-name scrapers, I set strict limits on high-cost paths. If I notice patterns, I dynamically increase the strictness until the Rate is back in the clear.

Fine-Tuning: Rate, Burst, NoDelay, and Status Codes

The "Rate" parameter controls the throughput per second, "Burst" allows for short-term buffering, and "nodelay" determines whether I prefer buffering or immediate forwarding. I start off moderately, e.g., 10r/s with a burst of 20 on APIs, and fine-tune based on log analysis. For login routes, for example, I set it to 1r/s with a small burst to slow down brute-force attacks. If limits are exceeded, I return a 429, because clients can use this to cope and the retry logic works properly. In special cases, I use alternative codes if clients request them.

Table Overview: Directives and Usage

The following Table summarizes key guidelines and explains when they are appropriate.

directive Effect Example Typical use
limit_req_zone Specify the key, zone, and Rate firmly limit_req_zone $binary_remote_addr zone=perip:10m rate=10r/s; Per IP, token, or user agent
limit_req Enables the limit in Location/Server limit_req zone=perip burst=20 nodelay; Fine-grained control per path or vHost
limit_req_status Sets the HTTP status code to Exceedance limit_req_status 429; Proper Client Behavior and Retries
map Forwards requests to zones at map $http_user_agent $is_bot {…} Bot/Human Classification Based on User-Agent

Practical Application: Securing a Specific Login Endpoint

I set very strict limits on /login because bots use passwords with high Frequency Test it out. 1r/s with burst 3 prevents massive brute-force attacks without being too harsh on real users. I also log repeated failed attempts so I can temporarily block IP addresses. Combined with 2FA and, optionally, CAPTCHA, this significantly reduces the load on the database and session handling. This way, I keep failed attempts to a minimum and ensure that the Access ready and stable.

Best Practices: Providing APIs in a Fair and Controlled Manner

APIs require clear Odds, so that individual clients don't monopolize the entire throughput. For general routes, I set 10r/s and a burst of 20; for high-traffic endpoints, I use stricter limits. If tokens or API keys are available, I limit access per token rather than per IP. This ensures fairness among customers and prevents abuse. For a more in-depth look, see my note on API Rate Limiting, which provides a broader context for the concept.

Monitoring, Logging, and Iterative Refinement

I'm logging 429 responses, including Key (e.g., IP or token) and path to detect patterns. Spikes on a few paths indicate scraping or brute-force attacks; distributed traffic suggests botnets. Using this data, I set limits only where necessary and minimize false positives. Dashboards showing rates, error rates, and latency let me see the impact of every change. This way, the Performance high, while protection increases.

Integration into a comprehensive protection strategy

I consider rate limiting to be a strong first step shift, but I combine it with WAF rules, IP reputation, and TLS hardening. To counter volume-based attacks, I use upstream DDoS protection that filters network-level traffic before NGINX has to step in. I continuously monitor metrics, set up alerts for unusual spikes, and respond by updating rules. This creates a resilient protective network made up of multiple building blocks. These provide a practical overview: DDoS Strategies.

Specific Configuration Patterns for Bots vs. Humans

I use `map` to sort visitors into categories and direct them to their own zones. Known crawlers are given moderate limits, while generic agents are given stricter ones. I remain stricter for paths like /search or /report, as they consume a lot of CPU. In the event of repeated violations, I do not increase the limits; instead, I impose a temporary block or transfer the check to a bot detection module. This ensures that the Misuse Rate low, without interfering with search engines.

Example: Two Zones and User-Agent Mapping

The following excerpt shows the separation based on User agent and the assignment of appropriate limits. I combine this with differentiated status codes and logging fields to accurately measure the effect. Bots with generic agents end up in the strict zone. Humans or verified crawlers use the more lenient zone. This approach provides predictable Throughputs Per class:

map $http_user_agent $is_bot {
  default 0;
  "~*googlebot"     0;
  "~*bingbot" 0;
  "~*crawler|scraper|bot" 1;
}

limit_req_zone $binary_remote_addr zone=human:10m rate=10r/s;
limit_req_zone $binary_remote_addr zone=bot:10m   rate=1r/s;

server {
  location / {
    if ($is_bot) {
 limit_req zone=bot burst=5;
    }
    if ($is_bot = 0) {
 limit_req zone=human burst=20 nodelay;
    }
    limit_req_status 429;
  }
}

Error Handling: Communicating Error Code 429 Correctly

I provide a clear Answer with a note on when it makes sense to try again. For APIs, this includes a valid Retry-After header so that clients can apply backoff. Human users receive a brief explanation without technical details. This reduces the number of support tickets and ensures that the behavior is easy to understand. A clean UX makes limits acceptable and prevents frustration.

Hosting Provider, Network, and Kernel: Strengthening the Foundation

High levels of legitimate traffic and defensive measures require reliable Resources and sensible defaults at the network level. I make sure to use the latest NGINX versions, allocate sufficient RAM for zones, and enable protection against transport-layer attacks. To guard against SYN floods, enabling TCP SYN Cookies in the kernel to prevent connections from getting stuck. Overall, this relieves NGINX of unnecessary load. This way, I focus limits on the HTTP layers and keep the Throughput stable.

In a nutshell: How I Use NGINX Rate Limiting Effectively

I limit requests to key, I isolate critical paths and keep bots at bay with strict zones. Burst and nodelay help allow legitimate traffic spikes without encouraging abuse. Using 429 logs, I continuously calibrate the settings and tighten limits only where necessary. Combined with WAF, DDoS protection, monitoring, and kernel hardening, this creates a robust security strategy. Those who implement it consistently will significantly reduce bot traffic and preserve Performance even under load.

Elements Often Missing in Practice

In many setups, several key components are missing that significantly enhance the effectiveness of rate limiting:

  • Actual Client IP Addresses Behind Proxies: Without proper Real IP handling, NGINX often limits the load balancer's IP address—and those limits then apply to all users behind it.
  • Dry Runs: Limits are activated „blindly.“ It’s better to first log only how often a limit would have been triggered.
  • Fine-Grained Keys: Instead of limiting access based solely on IP address, it's worth setting limits per API token, session, or user to ensure fairness.
  • Interaction with limit_conn: Concurrent connections and request rates account for different patterns of abuse.
  • Specific Exceptions: Health checks, webhooks, or internal services often require more lenient limits or no limits at all.

Reverse Proxy: Securely Evaluate the Actual Client IP

If NGINX is behind a load balancer, I set the Real-IP directives so that $binary_remote_addr reflects the actual client. I only trust networks that I own, and I enable recursive evaluation:

http {
  # Trusted proxy IP ranges (example)
  set_real_ip_from 10.0.0.0/8;
  set_real_ip_from 192.168.0.0/16;
  # Add public LB/CDN ranges as needed

  real_ip_header X-Forwarded-For;
  real_ip_recursive on;

  limit_req_zone $binary_remote_addr zone=perip:20m rate=10r/s;
}

Without this setting, a limit would otherwise affect many innocent users at once. After setting it up, I check the access logs to see if the expected client IP appears.

Key Strategy: IP, Users, Tokens, and Path

The key you choose determines fairness and effectiveness. Here are some tried-and-true patterns:

  • Pro IP ($binary_remote_addr): Quick to set up, good for /login and anonymous endpoints.
  • Per API token: Ensures fairness among customers; protects against NAT bundling. I extract tokens using `map`.
  • Per path class: Set separate limits for high-cost endpoints, e.g., /search has a higher limit than /status.
map $http_authorization $api_token {
  default "";
  "~*^Bearer\s+(.+)$" $1;
}

limit_req_zone $api_token zone=per_token:30m rate=5r/s;

server {
  location /api/ {
    # Only applies if a token is present
    limit_req zone=per_token burst=10;
    limit_req_status 429;
  }
}

Important: High key cardinality consumes memory in the zone. Plan for buffers and monitor memory usage.

Storage and Zone Sizing

The zone stores metadata for each active key. The storage usage per entry is a few dozen bytes plus overhead. From this, I conclude:

  • For a large number of concurrent IPs/tokens, I choose larger zones, such as 50–100 MB.
  • I tend to start with a generous setting and check the NGINX logs: „shared memory zone is full“ indicates that the settings need to be fine-tuned.
  • Unused keys expire after a short period of inactivity; peaks are more important than the daily average.

Using Burst and Nodelay Precisely

Without nodelay NGINX queues excess traffic within the burst buffer and delayed Requests. With nodelay Valid burst requests are allowed through immediately, while excess requests are rejected. My approach:

  • Interactive Trails (HTML): Preferably without `nodelay`, to create short wait times instead of hard 429 errors.
  • APIs: Often with `nodelay`, so that clients clearly receive a 429 response and apply a backoff.
  • Expensive Endpoints: A short burst to smooth out backend peaks.

Dry Run, Log Level, and Evaluation

Before I enable limits, I activate Dry Run and adjust the log level. That way, I can see the effect without any risk:

server {
  location /api/ {
    limit_req zone=perip burst=20;
    limit_req_dry_run on; # log only, do not block
    limit_req_log_level notice;  # less severe than 'error'
  }
}

I then analyze the access data for 3–7 days, identify hotspots, adjust the rate/burst, and only then disable the dry run.

429 Clean Transport: HTML, JSON, and Retry-After

To ensure a good user experience, I distinguish between browsers and API clients and use Retry-After. Here's how I clearly communicate limits:

map $http_accept $wants_json {
  default 0;
  "~*application/json|/json"    1;
}

server {
  error_page 429 = @rate_limited;

  location @rate_limited {
    add_header Retry-After 2 always;
    if ($wants_json) {
 add_header Content-Type application/json;
      return 429 '{"error":"too_many_requests","retry_after":2}';
    }
    return 429 "Please try again later.";
  }
}

APIs can respond programmatically, and users receive a clear message.

Combining `limit_req` and `limit_conn`

limit_req addresses throughput per time slot, limit_conn limits the number of simultaneous connections. To counter downloads, chatty clients, or HTTP/2 floods, I combine both:

limit_conn_zone $binary_remote_addr zone=perip_conn:10m;

server {
  location /api/ {
    limit_req  zone=perip burst=20 nodelay;
    limit_conn zone=perip_conn 20;  # max. 20 concurrent connections per IP
  }
}

This way, I prevent a small number of clients from tying up resources by maintaining too many concurrent connections, even though they are adhering to the rate limit.

Exceptions, Health Checks, and Internal Routes

Not every path needs limits. Health checks (/healthz), internal webhooks, and payment callbacks are assigned their own locations without a `limit_req`—or with more lenient values:

server {
  # No limits for health checks
  location = /healthz { return 200 "ok"; }

  # Soft limits for payment callbacks
  location /webhooks/pay/ {
    limit_req zone=perip burst=5;
  }

  # Strict protection for login
  location = /login {
    limit_req zone=perip rate=1r/s burst=3;
  }
}

Granular exceptions reduce false positives and keep integrations stable.

More Robust Zone Routing Without "If" Magic

When it comes to separating „bot vs. human,“ I prefer internal redirects using named locations. This makes the configuration clear and predictable:

map $http_user_agent $is_bot {
  default 0;
  "~*googlebot|bingbot" 0;
  "~*crawler|scraper|bot" 1;
}

limit_req_zone $binary_remote_addr zone=human:20m rate=10r/s;
limit_req_zone $binary_remote_addr zone=bot:10m   rate=1r/s;

server {
  error_page 418 = @bot;

  location / {
    if ($is_bot) { return 418; }   # internal redirect
    limit_req zone=human burst=20 nodelay;
    limit_req_status 429;
    try_files $uri $uri/ /index.html;
  }

  location @bot {
    limit_req zone=bot burst=5;
    limit_req_status 429;
  }
}

As a result, bots end up deterministically in the strict zone, while humans end up in the relaxed one—without both limits taking effect at the same time.

Testing, Measuring, Fitting: A Pragmatic Process

  • Staging: Select a conservative rate/burst setting, enable dry run, and run the synthetic load against the hot path.
  • Smoke tests: Use curl or Lasttools to generate short bursts and check the 429/delay behavior.
  • Production Pilot: Start by applying this to individual locations and monitor the logs closely.
  • Iterative Sharpening: Set limits only where patterns are evident; minimize false alarms.
# Example: Fast Burst Test with curl
for i in {1..50}; do curl -s -o /dev/null -w "%{http_code}\n" https://example.com/login & done; wait

Minute-based rather than second-based rates and granular paths

NGINX allows rates in seconds or minutes (r/s, r/m). For login abuse, I often set the limit to 60r/m instead of 1r/s to allow short, legitimate double-clicks but cap continuous firing. Expensive paths are given tighter limits than cheap ones. Example:

limit_req_zone $binary_remote_addr zone=perip_min:20m rate=60r/m;

server {
  location /search/ {
    limit_req zone=perip_min burst=10;   # stricter
  }
  location /status {
    # no limit – inexpensive and used internally
    return 200;
  }
}

Pitfalls and How I Avoid Them

  • Wrong Key: When using proxies without a real IP address, I accidentally limit all users at once.
  • Zones That Are Too Small: „zone is full“ leads to unpredictable behavior—allow for ample capacity.
  • A limit for everything: Different paths require different values; a one-size-fits-all approach leads to frustration.
  • No monitoring: Without a 429 analysis, misconfigurations go unnoticed.
  • Over-Whitelist: Exceptions that are too broad open the floodgates—use a targeted, temporary, and transparent whitelist.

Special Considerations with HTTP/2, SSE, and Caching

HTTP/2 bundles requests over a small number of connections; limit_conn remains relevant nonetheless, because streams consume resources. Server-Sent Events or long downloads rarely trigger rate limits (few requests), but they do take time—in these cases, I set limits in parallel using `limit_conn` or implement bandwidth strategies. Where possible, I offload the load by using Caching (e.g., static assets, frequent GET requests), so that limits are triggered less often and users receive faster responses.

Surgical Checklist

  • Real IP correct, keys defined (IP/token/user)
  • Zones are generously sized; metrics and logs are available
  • Rate/burst adjusted per path class; nodelay set intentionally
  • Dry run tested; 429 communication (Retry-After) implemented
  • Exceptions for Health/Webhooks, Combination with limit_conn
  • Iterative Re-sharpening and Alerts for Anomalies

Current articles