Rate Limiting: Protecting APIs Without Punishing Users

Your API does not die from one big attack. It dies from one customer's broken retry loop at 2 a.m., quietly starving everyone else. Rate limiting is the difference between a bad night for one integration and a bad quarter for your whole platform, and most teams bolt it on so crudely that it punishes exactly the users they wanted to protect.

Rate limiting is a product surface, not a firewall rule

A rate limiter has four jobs, and only one of them is security:

• Protect capacity. Your database can handle N queries per second. The limiter is the contract that keeps demand under N.
• Enforce fairness. One tenant should not be able to consume the headroom of fifty others. This is the noisy neighbor problem, and it is the most common reason production APIs feel "randomly slow."
• Control cost. Every request has a price: compute, egress, third party API calls, LLM tokens. Limits are your budget enforcement.
• Blunt abuse. Credential stuffing, scraping, enumeration. Necessary, but genuinely hostile traffic is a minority of what your limiter touches on a normal day.

The mistake is designing for job four and shipping it against jobs one through three. That is how you get paying customers hitting opaque 403s because their nightly sync crossed an invisible line. Treat the limiter as part of your API's developer experience: documented, predictable, observable. Everything below follows from that framing.

Pick the algorithm for the traffic you actually have

There are four algorithms that matter. Each has a specific failure mode. Choose deliberately.

Fixed window

Count requests per key per window (say, 100 per minute), reset the counter at the boundary. It is one Redis INCR with an EXPIRE, which is why everyone starts here. The flaw: a client can send 100 requests at 11:59:59 and 100 more at 12:00:01. That is double your intended rate in two seconds, which is precisely when your backend gets hurt. Fine for coarse abuse ceilings. Wrong for capacity protection.

Sliding window counter

Keep the current and previous window counts, then weight the previous one by how much of it still overlaps: previous * (overlap fraction) + current. It approximates a true sliding log at a fraction of the memory, and it kills the boundary burst problem. In our experience this is the right default for per user HTTP API limits. Two counters per key, no timestamp lists, accurate enough that nobody notices the approximation.

Token bucket

Each key owns a bucket with capacity B that refills at rate R tokens per second. A request spends one token (or more, we will get to that). This is the only algorithm on the list that models what users actually do: work in bursts, then go quiet. A CLI tool that fires 30 requests in one second and nothing for an hour is a legitimate client, and a token bucket with B=50, R=5 welcomes it while still capping sustained throughput at 5 rps. If your API serves developers, this is almost always the answer.

Leaky bucket

Requests enter a queue that drains at a constant rate. It shapes traffic rather than just policing it, producing a perfectly smooth output stream. Use it in front of fragile downstreams: a legacy SOAP service, a payment provider with strict limits, a database that falls over under bursts. The tradeoff is latency, because queued requests wait. That is usually wrong for interactive endpoints and exactly right for background pipelines.

Rule of thumb: token bucket at the user boundary, leaky bucket at the fragile downstream boundary, sliding window when you want simple and honest, fixed window only as a blunt outer ceiling.

Enforce at the right layers, plural

One limiter is never enough. Layer them coarse to fine:

• Edge (CDN, WAF, nginx). Cheap, per IP, brutal. Its job is to absorb floods before they touch your infrastructure. An nginx example that most teams underuse:

limit_req_zone $binary_remote_addr zone=perip:20m rate=30r/s;

location /api/ {
    limit_req zone=perip burst=60 nodelay;
    limit_req_status 429;
}

burst=60 nodelay is the important part: it grants token bucket semantics (immediate bursts up to 60) instead of rigidly spacing requests 33ms apart, which would punish every browser that fires parallel requests.

• Gateway (Kong, Envoy, API Gateway). Per API key limits, tier enforcement, and quota accounting live here. This layer knows who the caller is; the edge does not.
• Application. Per endpoint and cost aware limits. Only your code knows that POST /reports/generate costs 400x what GET /health costs. Weight tokens accordingly: cheap reads spend 1, expensive aggregations spend 50. One limit, honest accounting.

IP based keys deserve a warning: mobile carriers and corporate networks put thousands of users behind one address via CGNAT. Aggressive per IP limits at the edge will block entire offices. Keep edge limits loose (floods only) and do identity based limiting behind authentication. This layering also matters for security posture more broadly; we cover the adjacent controls in API security best practices.

Distributed counters that do not lie

The moment you run more than one instance, in memory counters undercount by a factor of your instance count. You need shared state, and in practice that means Redis. Two rules keep it correct and fast:

Rule one: make check and update atomic. The naive GET, decide, SET sequence is a race that leaks requests under concurrency, which is exactly when limits matter. Use a Lua script so Redis executes the whole decision as one unit:

-- KEYS[1] = bucket key; ARGV = rate, capacity, now, cost
local tokens = tonumber(redis.call('HGET', KEYS[1], 'tokens') or ARGV[2])
local ts     = tonumber(redis.call('HGET', KEYS[1], 'ts') or ARGV[3])
tokens = math.min(ARGV[2], tokens + (ARGV[3] - ts) * ARGV[1])
local allowed = tokens >= tonumber(ARGV[4])
if allowed then tokens = tokens - ARGV[4] end
redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', ARGV[3])
redis.call('EXPIRE', KEYS[1], 3600)
return allowed and 1 or 0

Note the pattern: tokens are refilled lazily from the elapsed time, so there is no background job and no per second writes for idle keys. The EXPIRE matters too. Without a TTL, every API key that ever existed lives in Redis forever, and your limiter becomes a slow memory leak.

Rule two: decide what happens when Redis is down. Fail closed and a cache outage becomes a full API outage you inflicted on yourself. Fail open and your protection vanishes exactly when the system is already degraded. Our default: fail open at the identity layer with an alert, fail closed only for unauthenticated endpoints and known attack surfaces like login. If limiter latency is a concern (a Redis round trip on every request adds up), run a small local token bucket per instance as a pre filter and let the shared limiter be the source of truth. Slight overcounting tolerance in exchange for resilience is usually the right trade.

Tell users what happened and when to come back

This is where "without punishing users" is won or lost. A limiter that answers with a bare 429 (or worse, a 403 or connection reset) forces client developers to guess, and guessing produces aggressive retry loops that make your problem worse. The contract:

HTTP/1.1 429 Too Many Requests
Retry-After: 12
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 12
Content-Type: application/problem+json

{"type": "https://api.example.com/errors/rate-limited",
 "title": "Rate limit exceeded",
 "detail": "100 requests per minute per API key. Retry after 12 seconds."}

The specifics that matter:

• Use 429, not 403. 403 means "never," 429 means "not right now." Client SDKs and proxies treat them completely differently.
• Always send Retry-After. It converts blind exponential backoff into informed waiting. Send the standard RateLimit-* headers on every response, not just rejections, so well behaved clients can self regulate before hitting the wall.
• Document limits per tier and per endpoint, and ship backoff with jitter in your official SDKs. Full jitter (sleep(random(0, base * 2^attempt))) prevents synchronized retry waves. If you publish an SDK without backoff built in, you have shipped a thundering herd generator.
• Make writes idempotent with idempotency keys so a retried request is safe. Otherwise users fear retrying, and fear produces worse client code.

Clear limit communication is a subset of a bigger discipline we have written about in designing APIs developers love. The principle is identical: the API should never make the client guess.

Failure modes we keep seeing

• Synchronized resets. Every client's window resets at the top of the minute, so the top of the minute becomes your new peak load. Stagger windows per key by hashing the key into an offset.
• Limiting after the expensive work. If the check happens after authentication, deserialization, or a database call, an attacker spends your resources for free. Check as early in the stack as identity allows.
• One global bucket per tenant. A tenant's batch import then starves their own interactive users. Split buckets by workload class: interactive, background, webhooks.
• Punishing your own webhooks and cron consumers. Internal and partner traffic needs distinct keys and generous, separately monitored limits.
• Unbounded key cardinality. Keying by URL path plus query string means every unique query mints a Redis key. Key on identity plus route template, never raw URLs.
• No shadow mode. Teams ship limits derived from guesswork and instantly break their three biggest customers. Always run in log only mode first and measure who you would have blocked.

That last point deserves emphasis: you cannot tune what you cannot see. Per key rejection metrics, top talkers dashboards, and alerts on rejection rate spikes are not optional extras, they are the tuning loop. Our guide to observability with logs, metrics, and traces covers the plumbing.

Rollout checklist

1. Measure reality first. Pull two to four weeks of traffic per identity and per endpoint. Your limits must sit above real usage of legitimate clients, typically with 2x to 5x headroom over their p99.
2. Choose keys. API key or user ID behind auth, IP only at the edge, separate keys per workload class.
3. Choose algorithms per layer. Token bucket for users, leaky bucket in front of fragile downstreams, blunt fixed window ceilings at the edge.
4. Weight expensive endpoints with token costs instead of maintaining twenty separate limits.
5. Ship in shadow mode. Log would be rejections for at least a week, review the list of affected keys, then talk to those customers before you flip enforcement on.
6. Ship the contract: 429s, Retry-After, RateLimit-* headers, error docs, SDK backoff with jitter.
7. Wire the dashboards: rejection rate per tier, top rejected keys, limiter latency, Redis health, fail open events.
8. Load test the limiter itself. It is a distributed system component with its own saturation point. Find it before your customers do.

How Innovation T can help

Innovation T designs and builds this layer for real production systems: multi tier rate limiting, Redis backed quota services, gateway configuration, and the observability to tune it all without breaking paying customers. We have done it for SaaS platforms, fintech APIs, and high traffic public endpoints, and we ship the client side contract (headers, SDKs, docs) along with the enforcement.

If your API is growing faster than your confidence in its guardrails, look at our software and cloud engineering services or talk to our team. A two week engagement is usually enough to go from "we have nginx defaults" to a limiter your customers never have to think about.