Benchmarking Redis Rate Limiters: Token Bucket vs. Sliding Window in Python

Why Rate Limiting Matters in Distributed Systems

Any public-facing API requires rate limiting. Without it, your system is vulnerable to brute-force attacks, DDoS attempts, and noisy neighbor problems where a single aggressive client degrades the experience for everyone else.

Designing a rate limiter for a single server is relatively simple. Designing one for a globally distributed microservices architecture handling millions of requests per second is a classic system design challenge. In this article, we'll break down the architecture and algorithms required.

1. Choosing the Right Algorithm

The heart of a rate limiter is the algorithm it uses to count requests.

Token Bucket

• How it works: Imagine a bucket holding tokens. Tokens are added at a fixed rate. Every request costs one token. If the bucket is empty, the request is dropped.
• Pros: Memory efficient, allows for short bursts of traffic.
• Cons: Tricky to tune burst size vs. sustained rate.

Fixed Window Counter

• How it works: Divides time into fixed windows (e.g., 12:00 to 12:01). Increments a counter for every request. Resets at the start of the next window.
• Pros: Very easy to implement.
• Cons: Spike traffic at the edges of the window. A client could send their entire quota at 12:00:59 and again at 12:01:01, effectively doubling their allowed rate in a short span.

Sliding Window Log

• How it works: Keeps a timestamp log of every request for a user. Drops requests if the log size exceeds the limit within the time frame.
• Pros: Highly accurate.
• Cons: Consumes significant memory and compute to maintain and prune logs. Not viable for massive scale.

Sliding Window Counter (The Winner)

• How it works: A hybrid approach. It tracks the counter for the previous fixed window and the current fixed window, and calculates a weighted average based on the current time overlapping the window.
• Pros: Smooths out edge spikes, highly memory efficient, highly accurate.

2. Architecture for Distributed Systems

In a multi-server setup, maintaining state becomes the primary challenge. If a load balancer routes User A's first request to Server 1 and their second to Server 2, how do they share the rate limit counter?

Option A: Sticky Sessions

Route all traffic for a specific user to the same server. The server keeps the rate limit state in local memory.

• Verdict: Bad idea. It unevenly distributes load and fails gracefully when a server crashes.

Option B: Centralized Datastore (Redis)

All servers read and write to a centralized Redis cluster. Redis is fast enough (in-memory) to handle the latency requirements.

• Verdict: The standard approach. However, it introduces network latency and race conditions.

3. Handling Race Conditions in Redis

If two requests for the same user hit two different API servers simultaneously, both might read the counter as 4, both increment to 5, and both allow the request, bypassing the limit.

Solution: Lua Scripts

Redis allows you to execute Lua scripts atomically. You can write a script that fetches the counter, checks the limit, increments it, and updates the TTL in a single atomic operation.

-- Simple Token Bucket Lua Script
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local current = tonumber(redis.call('get', key) or "0")

if current + 1 > limit then
    return 0 -- Rejected
else
    redis.call('INCRBY', key, 1)
    redis.call('EXPIRE', key, 60)
    return 1 -- Allowed
end

4. Performance Optimization: Local Cache + Async Sync

For ultra-high throughput where even Redis latency is unacceptable, you can use a hybrid approach.

• API servers keep a local, in-memory rate limiter.
• They sync their local counts to the central Redis cluster asynchronously every few seconds.
• Tradeoff: This trades strict accuracy for performance. A client might exceed their limit slightly between sync windows, but the system remains highly available and fast.

Architectural Insights & Key Takeaways

Building a distributed rate limiter requires balancing strict enforcement against system latency. For most modern APIs, combining a Sliding Window algorithm implemented via atomic Redis Lua scripts provides the best balance of accuracy, performance, and scalability.

You Might Also Like

• Real-Time Data Streaming with Apache Kafka: A Production Guide
• LangGraph in Production: Human-in-the-Loop, Checkpointing & State Persistence
• Hierarchical Multi-Agent Systems in LangGraph: Supervisors, Subgraphs & State Machines
• Quantum Computing for Software Engineers: What You Actually Need to Know in 2026

Frequently Asked Questions