Flash Sale — Virtual Queue + Token Bucket

The virtual queue with token bucket architecture is the industry-standard approach for flash sale systems handling millions of concurrent users. It solves the fundamental problem that makes the naive database lock approach collapse: the thundering herd, where all users hit inventory simultaneously, overwhelming the database with lock contention.

The key architectural insight is demand shaping. Instead of allowing 500K users to compete for database row locks simultaneously, a virtual queue serializes access: users enter a FIFO queue backed by a Redis sorted set, and a TokenWorker releases purchase tokens at a controlled rate (1000/sec). Only token-holders can attempt a reservation. This reduces backend load by 500x — the database never sees more than 1000 concurrent purchase attempts regardless of how many users are waiting.

The demand shaping separates two concerns that the naive approach conflates: fairness and inventory management. The virtual queue provides fairness via FIFO ordering — users who arrived first get tokens first. Inventory management uses Redis atomic DECR — a single-threaded operation that returns the new stock count in one round-trip (~1ms), eliminating the need for database row locks entirely. If the DECR result is >= 0, the reservation succeeds. If negative, the item is sold out.

The numbers make the case compelling. In the naive approach, 50K concurrent users generate 50K SELECT FOR UPDATE transactions, saturating the 100-connection pool in milliseconds. With the virtual queue, those 50K users generate 50K ZADD operations to Redis (which handles 500K ZADD/sec easily on a single node), and only 1000/sec token-gated reservation attempts reach the inventory system. The database handles order persistence at 1000 inserts/sec — well within its capacity.

Kafka decouples reservation from payment processing. When a reservation succeeds, SaleService publishes a checkout event to Kafka and immediately returns 'reserved' to the user (~40ms total). CheckoutWorker asynchronously processes payment (~200ms), updates the order status, and releases stock back via Redis INCR if payment fails. This means users see confirmation in 40ms instead of 240ms, and payment gateway failures do not block the reservation path.

The architecture scales horizontally at each tier. QueueService scales with queue-join traffic (500K RPS). ReservationService scales with token-gated traffic (1K/sec). The virtual queue capacity scales with Redis cluster size. Kafka throughput scales with partition count. Each tier can be scaled independently.

The primary limitation is that the virtual queue does not address bots. Automated scripts can join the queue microseconds after sale start, grabbing positions ahead of human users. CAPTCHA at queue join and device fingerprinting add bot mitigation but are not part of this architecture. The Tiered variant addresses bot detection via a dedicated BotDetector service with device fingerprinting.

The reservation TTL mechanism adds another dimension of complexity. When a user reserves an item, the stock is decremented in Redis but payment has not yet been processed. If the user abandons the checkout (closes the browser, payment card is declined, network timeout), the reserved stock must be returned to the pool. The 5-minute TTL on reservations triggers a background expiry process that detects abandoned reservations, INCRs the Redis counter to release the stock, and updates the order status to EXPIRED in PostgreSQL. This TTL-based release mechanism is essential for preventing inventory 'leakage' — without it, abandoned reservations would permanently drain available stock.

The WebSocket connection cost for real-time queue position updates is a significant infrastructure consideration. With 10M users in the queue, maintaining 10M persistent WebSocket connections requires substantial memory and network resources. Production implementations often use long polling (30-second intervals) instead of WebSocket to reduce connection overhead, accepting slightly less responsive queue position updates as the trade-off.

This architecture appears in interviews at Amazon, Shopify, Ticketmaster, Nike, and Walmart. Interviewers expect candidates to explain demand shaping, articulate why Redis DECR replaces database locks, reason about the FIFO fairness guarantees, and identify bot vulnerability as the motivation for the tiered approach.

The virtual queue flash sale system uses 9 components organized into three layers: traffic ingestion (Client, API Gateway, QueueService), inventory reservation (ReservationService, InventoryCache, QueueCache, OrderDB), and async processing (CheckoutStream, TokenWorker, CheckoutWorker).

All traffic enters through the API Gateway, which performs JWT authentication (~3ms), enforces a 600K RPS rate limit, and routes queue operations to QueueService and reservation operations to ReservationService. The API Gateway is the single entry point — no traffic bypasses it.

The queue layer handles the traffic burst. When the sale opens, 500K users/sec hit POST /api/v1/sales/{id}/queue. QueueService writes each user into QueueCache (Redis sorted set with ZADD, scored by join timestamp) and returns their queue position. This is the highest-traffic operation — 500K RPS of simple ZADD writes. QueueService runs 20 pods with 100 threads each for 400K sustained throughput.

The token release mechanism is the demand shaping engine. TokenWorker runs a controlled loop, reading the next batch of users from QueueCache via ZPOPMIN, generating time-limited purchase tokens (5-minute TTL), and writing them back to QueueCache. It releases 1000 tokens/sec — this rate is configurable per sale. Token-holders poll or receive WebSocket notifications that their token is ready.

The reservation layer handles token-gated purchases. ReservationService validates the purchase token against QueueCache, performs an atomic DECR on InventoryCache (Redis), and if the result is >= 0, the reservation succeeds. The service persists the order to OrderDB with status RESERVED and publishes a checkout event to CheckoutStream (Kafka). At 1000/sec token-gated traffic, the backend operates at sustainable capacity with massive headroom.

The async processing layer handles payment. CheckoutWorker consumes checkout events from Kafka, calls the external payment gateway (~200ms), updates order status to CONFIRMED or FAILED in OrderDB, and releases stock back via Redis INCR on payment failure. The 5-minute reservation TTL ensures abandoned reservations expire and stock returns to the pool.

InventoryCache (Redis) is the zero-overselling guarantor. Each item's stock is stored as a simple integer counter. DECR is atomic and single-threaded — no lock contention, no race conditions. At 1000 DECR/sec, Redis handles this trivially. The counter going negative means sold out — the service INCRs it back immediately.

QueueCache (Redis sorted set) serves dual purpose: FIFO queue storage for user positions and token storage for purchase admission. ZADD gives O(log N) insertion, ZPOPMIN gives O(log N) dequeue — both sub-millisecond at 10M entries.

OrderDB (PostgreSQL) is the durable persistence layer. Each reservation creates an order record with status RESERVED. CheckoutWorker updates the status to CONFIRMED after successful payment or FAILED after payment decline. The database uses 16 partitions with 3 replicas for durability and throughput. Strong consistency prevents double reservations. At 1K inserts/sec (token-gated), the database operates well within capacity — a dramatic improvement over the naive approach where the database was the bottleneck.

The separation of concerns across services is a key architectural strength. Each component has a clear responsibility: QueueService manages the waiting line, TokenWorker controls admission rate, ReservationService handles inventory and orders, CheckoutWorker processes payment. This decomposition enables independent scaling, independent deployment, and independent failure isolation. If CheckoutWorker falls behind (payment gateway slowdown), reservations continue unaffected — Kafka buffers the backlog.