Rate Limiting

Rate limiting is a critical mechanism for controlling the rate at which clients consume API resources. It serves three primary purposes: protecting against abuse (preventing malicious or buggy clients from overwhelming the service), ensuring fair usage (preventing one client from monopolizing shared resources), and preventing resource exhaustion (bounding the total request rate to what the system can handle). Every production API should implement rate limiting -- without it, a single client can consume all available capacity, a bug in a client library can generate unbounded requests, and denial-of-service attacks can take down the entire service.

The five fundamental rate limiting algorithms each offer different trade-offs. The fixed window counter divides time into fixed windows (e.g., 1-minute intervals) and counts requests per window. It is simple to implement (a single counter per client per window) but has the boundary burst problem: a client can send its full quota at the end of one window and the beginning of the next, effectively doubling the rate. The sliding window log tracks the exact timestamp of every request and counts requests within a trailing window. It is perfectly accurate but requires storing every timestamp, consuming memory proportional to the rate limit times the window size. The sliding window counter is a hybrid that combines the fixed window counter of the current and previous windows, weighted by their overlap with the sliding window. It eliminates the boundary burst problem with minimal memory overhead -- just two counters per client.

The token bucket algorithm is the most widely deployed rate limiting mechanism. A bucket holds tokens, refilled at a constant rate (e.g., 10 tokens per second). Each request consumes one token. If the bucket is empty, the request is rejected. The bucket has a maximum capacity (burst size), allowing clients to send short bursts up to the bucket size as long as they have accumulated tokens. This is ideal for APIs that need to allow occasional bursts while maintaining a sustained rate limit. The leaky bucket works inversely: requests enter a queue (bucket) and are processed at a constant rate. If the queue is full, new requests are rejected. The leaky bucket produces the smoothest output rate but does not allow any bursting, which can be too restrictive for interactive APIs.

In distributed systems, rate limiting must be coordinated across multiple server instances. The standard approach uses Redis as a centralized counter store: each server checks and increments the client's counter in Redis before processing the request. Redis INCR and EXPIRE operations provide atomic counter manipulation, and Lua scripts ensure that the check-and-increment is atomic. Rate limits are typically organized in tiers: per-IP (protecting against anonymous abuse), per-API-key or per-user (ensuring fair usage among authenticated clients), per-endpoint (protecting expensive operations), and global (bounding total system load). Rate limit responses use standardized HTTP headers: X-RateLimit-Limit (the limit), X-RateLimit-Remaining (remaining requests), X-RateLimit-Reset (when the limit resets), and Retry-After (how long to wait before retrying). Enforcement happens at the API gateway (centralized, language-agnostic), application middleware (per-service, customizable), or service mesh sidecar (infrastructure-level, no code changes).

A highway on-ramp meter (traffic light at the ramp) is a rate limiter. It controls how many cars enter the highway per minute to prevent congestion. Without the meter, all cars merge at once, causing stop-and-go traffic that slows everyone down. The meter releases cars at a controlled rate (token bucket -- each green light is a token), allowing the highway to flow smoothly. During rush hour, the meter is stricter (lower rate limit); during off-peak, it is more lenient. Cars waiting at the meter experience a brief delay, but the overall highway throughput is higher than if all cars merged freely and caused gridlock.