Production-grade global multiplayer platform: multi-region game servers with rollback netcode (client-side prediction + server reconciliation), Kafka-powered replay and spectate streaming to S3, dedicated anti-cheat service, integrated voice relay, and persistent player progression.
The global competitive platform architecture is the production-grade solution for multiplayer games operating at worldwide scale — the architecture behind League of Legends, Valorant, Fortnite, and Apex Legends. It solves the two critical limitations of the single-region authoritative approach: network latency perception and geographic unfairness.
The first problem is perceived input latency. In the single-region authoritative model (v1), players feel the full network round-trip time (50-100ms) between pressing a button and seeing the result on screen. For casual games this is acceptable, but for competitive FPS and fighting games, 80ms of input lag makes the game feel sluggish and unresponsive. Players describe it as 'playing through mud.' The global variant solves this with rollback netcode — a technique where the client predicts the outcome of local inputs immediately (applying them to a local simulation copy) and corrects when the server's authoritative state arrives. If the prediction was correct (which it is 90%+ of the time for movement), the player sees zero input lag. If the prediction was wrong, the client rolls back to the server state and replays subsequent inputs, causing a brief visual correction.
The second problem is geographic unfairness. A single-region deployment means players in Tokyo connecting to US-East servers experience 150-200ms RTT, while players in New York enjoy 10ms. This geographic lottery determines competitive outcomes more than skill. The global variant deploys game server fleets in 5+ regions (US-East, US-West, EU-West, AP-Northeast, AP-Southeast) and the matchmaking service routes players to their nearest region. Most players connect with sub-30ms ping, creating a level playing field regardless of continent.
Beyond latency optimization, the global platform adds several production-critical features. A dedicated anti-cheat service consumes sampled player inputs from a Kafka stream and performs statistical analysis — detecting impossible movement speeds, inhuman aim trajectories, and suspicious action timing. Unlike inline validation that must complete within the 16.67ms tick budget, the dedicated service can spend seconds on deep analysis and flag cheaters mid-match or post-match.
Every game state tick (60 per second per match) streams through Kafka, enabling two powerful features: live spectating (viewers subscribe to a match's state stream with approximately 2-second delay) and replay recording (the full tick stream is archived to S3 for post-match replay, highlights, and tournament review). At 150K concurrent matches, the game state stream handles 9 million events per second — requiring 256 Kafka partitions.
VoiceRelay handles real-time team voice communication. Instead of peer-to-peer voice (which creates O(N^2) connections in a team), a centralized relay mixes audio from all teammates into a single stream per player. Each player sends one audio stream and receives one mixed stream, adding approximately 20ms of latency — imperceptible for voice communication.
ProgressionService persists player data (XP, rankings, match history) to PostgreSQL with strong consistency. Match results flow through Kafka from the game server fleet, enabling async processing without blocking the match-end screen. The progression system supports leaderboards, seasonal rankings, and detailed match history with per-player stats.
This is the architecture interviewers at gaming companies expect senior candidates to describe. The progression from P2P to authoritative to global demonstrates deepening understanding of real-time systems, and the specific technical choices (rollback netcode, regional routing, Kafka streaming for replay) signal production experience with latency-sensitive distributed systems.
The global competitive platform uses 10 components organized into five functional areas: matchmaking (MatchmakingService, MatchmakingCache), game simulation (GameServerFleet), data streaming (GameStateStream, ReplayStore), player services (ProgressionService, ProgressionDB, VoiceRelay), and security (AntiCheatService). The architecture is designed for multi-region deployment, though the simulation models a single region.
All matchmaking and profile traffic enters through MatchmakingService (15 ECS Fargate pods, 100 threads each). Unlike the v1 architecture with API Gateway and Load Balancer, the global variant routes directly to specialized services for lower latency. MatchmakingService handles three endpoints: queue for matchmaking (writes to Redis sorted set partitioned by region), poll matchmaking status (reads from Redis), and get player profile (reads from ProgressionDB with Redis cache). When enough players at similar skill levels in the same region accumulate, MatchmakingService forms a match and directly allocates a GameServerFleet instance.
MatchmakingCache (Redis, 8-node cluster, 26 GB) stores regional matchmaking queues as sorted sets, active session state, and hot player profiles. Partitioning queues by region ensures matchmaking operates independently per region. The 60-second TTL on queue entries is shorter than v1's 300 seconds because the global system enables cross-region fallback after 60 seconds for sparse regions.
GameServerFleet is the fleet of dedicated game server instances — one per active match. Each instance runs the authoritative simulation at 60 Hz with rollback netcode support. The server maintains a rolling buffer of recent game states (typically 10-20 frames). When a client's predicted state diverges from the server's authoritative state, the server sends a correction. The client rolls back to the corrected state and replays all inputs since that state, producing a smooth visual correction rather than a jarring teleport. This technique makes the game feel responsive at up to 150ms RTT.
GameStateStream (Kafka, 256 partitions) is the central nervous system of the platform. It carries three topic types: (1) game-state-tick — every authoritative game state snapshot at 60 Hz per match, consumed by spectator delivery and replay archiver; (2) match-result — end-of-match results consumed by ProgressionService; (3) anticheat-input-stream — sampled player inputs consumed by AntiCheatService. At 150K concurrent matches, the stream handles 9M events/sec for game state ticks alone.
ReplayStore (S3) archives complete match replays. The replay archiver consumer reads game-state-tick events from Kafka and writes compressed replay files to S3. Each 10-minute match generates approximately 100MB of state data. S3 Standard storage with lifecycle policies transitions replays to S3 Glacier after 30 days. ProgressionService serves replay metadata and generates pre-signed S3 URLs for client download.
AntiCheatService (30 worker instances) consumes sampled input streams from Kafka and performs deep statistical analysis: movement speed anomaly detection, aim trajectory analysis (detecting inhuman flick patterns), action timing validation, and impossible state detection. Running asynchronously means analysis can take 1-5 seconds without affecting the game server's tick budget. Flagged players are kicked mid-match via callback to GameServerFleet.
ProgressionService (8 pods, 100 threads each) handles player progression and replay access. It consumes match-result events from Kafka, calculates XP, updates player levels and MMR in ProgressionDB (PostgreSQL, 64 partitions, 3 replicas), and writes match history records. It also serves profile reads, leaderboard queries, and replay metadata.
VoiceRelay (50 pods) handles team voice communication. Players send Opus-encoded audio packets via UDP, and VoiceRelay mixes audio from all teammates into a single outbound stream per player. Running in the same region as the game server ensures minimal added latency (~20ms). Audio is ephemeral and not persisted.
Choice
Clients predict local inputs immediately; server corrects on authoritative state arrival
Rationale
Without prediction, players feel 50-100ms input delay (full RTT). With rollback netcode, the client applies inputs to a local simulation copy instantly — the game feels like 0ms latency. When the server's authoritative state arrives, the client compares it to the predicted state. If they match (90%+ of the time), no correction is needed. If they differ, the client rolls back to the server state and replays subsequent inputs. The result is responsive controls with rare, subtle visual corrections instead of constant perceived lag.
Choice
Deploy GameServerFleet in 5+ regions; MatchmakingService routes to nearest region
Rationale
A player in Tokyo connecting to US-East has 150-200ms RTT — unplayable for competitive games. Regional deployment ensures most players connect with sub-30ms ping. MatchmakingService detects the player's region (via IP geolocation or client measurement) and queues them in the regional sorted set. Cross-region matching is only enabled after 60 seconds if the regional queue is too sparse, accepting higher latency for shorter queue times.
Choice
Every game state tick published to Kafka for live spectating and S3 replay archival
Rationale
Publishing every tick to Kafka enables two high-value features with a single infrastructure investment. Live spectators subscribe to a match's partition and receive state updates with ~2 second buffering delay. The replay archiver writes the full tick stream to S3 for permanent storage. This dual-use pattern is more efficient than separate spectate and replay pipelines. The trade-off is Kafka throughput — 9M events/sec requires 256 partitions.
Choice
Separate service consuming sampled inputs from Kafka for deep statistical analysis
Rationale
Anti-cheat analysis is CPU-intensive (statistical modeling, trajectory analysis, anomaly detection) and must not consume the game server's 16.67ms tick budget. Running as a separate Kafka consumer means analysis can take 1-5 seconds without any gameplay impact. Detection latency of 1-5 seconds is fast enough to kick cheaters mid-match. The sampling approach (not every tick, but periodic input batches) reduces Kafka throughput from 9M to 500K events/sec.
Choice
S3 Standard for recent replays, automated transition to Glacier after 30 days
Rationale
At 100MB per match and 5K match completions/sec, replay storage grows at 500TB/month. S3 Standard costs $0.023/GB for recent replays that players actively watch. After 30 days, lifecycle policies transition to Glacier at $0.004/GB — an 83% cost reduction. The write-once, read-occasionally access pattern is ideal for object storage. Pre-signed URLs enable direct client download without proxying through application servers.
Choice
VoiceRelay mixes team audio centrally instead of peer-to-peer WebRTC mesh
Rationale
WebRTC P2P creates O(N^2) connections in a team — a 5-player team needs 10 bidirectional audio streams. VoiceRelay reduces this to O(N): each player sends one stream and receives one mixed stream. At 500K concurrent voice streams, the relay architecture scales linearly with player count. The added ~20ms latency is imperceptible for voice communication but would be unacceptable for game state (which is why game traffic uses direct UDP to the game server).
Target RPS
100K RPS (matchmaking + profiles) + 150K concurrent matches
Latency (p99)
<30ms regional game latency; <5ms perceived with prediction
Storage
500 TB/month replays (S3) + 2 TB player data (PostgreSQL)
Availability
99.9% (multi-region, replicated, Kafka durability)
| Operation | Time | Space | Notes |
|---|---|---|---|
| Regional matchmaking (POST /api/v1/matchmaking/queue) | O(log N) ZADD + O(log N + M) ZRANGEBYSCORE per region | O(1) per queued player (~500 bytes in Redis) | N = regional queue size (~300K per region). Cross-region fallback after 60s doubles the search space. |
| Game tick with rollback (server-side, 60 Hz) | O(P) per tick + O(B * P) for rollback where B = buffer depth | O(B * S) state buffer per match (B = 20 frames, S = game state size) | Rollback is triggered only on prediction mismatch — O(P) per tick in the common case. |
| Replay archival (Kafka consumer to S3) | O(T) per match where T = total ticks (60 Hz * duration) | O(100MB) per match replay file | 10-minute match = 36,000 ticks. Compressed to ~100MB. Written as a single S3 PutObject on match completion. |
| Anti-cheat analysis (per input sample batch) | O(W * log W) for statistical analysis where W = input window size | O(W) per analysis window (~1000 input samples) | Running standard deviation, trajectory analysis, and anomaly scoring on sliding windows. |
Persistent player profiles with progression data, skill ratings, and competitive tier. Written by ProgressionService after each match. Read for profile pages, leaderboards, and matchmaking MMR lookups. Partitioned by player_id across 64 shards with 3 replicas. Strong consistency prevents double XP on retry.
Indexes: idx_players_skill ON (skill_rating) — leaderboard queries, idx_players_region_skill ON (region, skill_rating) — regional leaderboards
64 partitions x 3 replicas. ~1.5M rows at 500 bytes each = ~750MB. Read-heavy from profile/leaderboard queries.
Per-player per-match records with detailed stats. One row per player per match. Written by ProgressionService on match-result consumption. Read for match history pages and replay lookups. Partitioned by player_id for efficient per-player history queries. Secondary index on match_id for cross-player lookups.
Indexes: idx_history_player ON (player_id, played_at DESC) — per-player history, idx_history_match ON (match_id) — cross-player match lookup
Grows at ~5K rows/sec (match completion rate x avg players per match). Partitioned with automatic archival after 12 months.
Active matchmaking session state in Redis, partitioned by region. Stores queue position, assigned match ID, and game server address. Written by MatchmakingService (queue entry) and GameServerFleet (server assignment). 60s TTL for abandoned queue entries.
1.5M active sessions x 500 bytes = ~750MB working set. Regional sorted sets for skill-based matchmaking.
Complete match replay files stored as compressed binary in S3. Contains tick-by-tick game state (60 ticks/sec) for client-side replay rendering. Written once by replay archiver Kafka consumer. Read via pre-signed S3 URLs. S3 Standard for 30 days, then Glacier lifecycle.
500TB/month new data. Lifecycle: S3 Standard (30 days) then Glacier. Pre-signed URLs for client download.
Emitted by GameServerFleet at 60 Hz per match. Contains the full authoritative game state snapshot. Consumed by spectator delivery (live viewing) and replay archiver (S3 storage). Partitioned by match_id across 256 partitions. Expected 9M msg/sec at peak.
Key Schema
match_id: string (partition key for per-match ordering)
Value Schema
{ match_id: string, tick_number: number, game_state: object (full snapshot), timestamp: string (ISO) }
Emitted by GameServerFleet when a match completes. Contains per-player stats and match metadata. Consumed by ProgressionService for XP/MMR updates and match history recording. Partitioned by match_id. Expected 5K msg/sec at peak.
Key Schema
match_id: string (partition key)
Value Schema
{ match_id: string, duration_sec: number, region: string, results: Array<{ player_id: string, kills: number, deaths: number, placement: number, xp_earned: number }> }
Sampled player input batches forwarded by GameServerFleet to AntiCheatService. Contains movement vectors, aim angles, and action timing for statistical anomaly detection. Partitioned by player_id. Expected 500K msg/sec (sampled from 9M ticks).
Key Schema
player_id: string (partition key for per-player analysis continuity)
Value Schema
{ player_id: string, match_id: string, inputs: Array<{ tick: number, movement: vec3, aim: vec2, actions: string[] }>, tick_range: string }
| Variant | Tier | Latency | Throughput | Cost | Complexity | Reliability |
|---|---|---|---|---|---|---|
| Naive (Client-Hosted P2P) | T1 | 0ms host / 50-150ms guests | ~8K RPS (lobby ops only) | $100/month (1 DB, 3 pods) | Low — 3 components, no game servers | ~99% backend; match dies if host disconnects |
| Authoritative (Dedicated Servers) | T2 | 50-100ms all players (equal) | 60K RPS lobby + 100K matches | $15,000/month (dedicated game servers) | Medium — 10 components, Kafka, Redis | 99.9% (replicated, no host dependency) |
| Global Platform (Multi-Region + Rollback) | T3 | <30ms regional + client-side prediction | 100K RPS + 150K matches globally | $50,000/month (multi-region fleet) | High — 10+ components, rollback netcode, 5 regions | 99.9% (multi-region, replay, anti-cheat) |
This template is for educational and illustration purposes only. It may not represent the optimal production design for this problem. Real-world systems involve additional considerations (compliance, specific cloud provider constraints, organizational requirements) not captured here. Use this as a starting point for discussion, not as a production blueprint.
The client maintains a local copy of the game simulation and a buffer of recent authoritative states from the server. When the player presses a movement key, the client immediately applies it to the local simulation — the character moves with zero perceived latency. Simultaneously, the input is sent to the server. The server processes it authoritatively and sends back the resulting state. When the client receives the server state (50-100ms later), it compares it to what it predicted. If the prediction was correct (which it is 90%+ of the time for simple movement), nothing happens — the player already sees the correct state. If the prediction was wrong (e.g., the server rejected the movement because of a collision the client did not know about), the client rolls back to the server state, replays all inputs since that state, and updates the display. This correction appears as a subtle visual 'snap' rather than constant input lag.
At 150K concurrent matches publishing 60 ticks/sec each, the stream handles 9 million events per second. Kafka partitions are the unit of parallelism — each partition is consumed by one consumer thread. To handle 9M events/sec with reasonable per-partition throughput (~35K events/sec per partition), 256 partitions provide the necessary parallelism. Partitioning by match_id ensures all ticks for a given match land on the same partition, preserving ordering for spectators and replay archiving. The trade-off is metadata overhead — 256 partitions per topic requires more ZooKeeper/KRaft coordination.
The anti-cheat service is fully decoupled from the game server via Kafka. GameServerFleet samples player inputs (not every tick — periodic batches) and publishes them to the anticheat-input-stream topic. AntiCheatService consumes these samples and runs three types of analysis: (1) movement speed validation — checking if a player's position change exceeds the maximum possible speed; (2) aim trajectory analysis — statistical detection of inhuman aim patterns (snap-to-target, perfect tracking); (3) timing analysis — detecting impossible reaction times or input automation. Analysis takes 1-5 seconds per sample batch. When a cheat is detected, the service sends a kick command to the relevant GameServerFleet instance via direct API call.
MatchmakingService implements a tiered fallback. For the first 30 seconds, it searches only the player's region with a tight MMR range (+/- 200). From 30-60 seconds, it widens the MMR range (+/- 500) within the same region. After 60 seconds, it enables cross-region matching, searching adjacent regions (e.g., US-East player can match with US-West or EU-West) with a latency warning displayed to the player. The player can accept the higher-latency match or continue waiting. This balances competitive fairness (tight MMR, low latency) with reasonable queue times. Sparse regions like Oceania may default to cross-region matching during off-peak hours.
S3 Standard at $0.023/GB costs approximately $11,500/month for the first month. With lifecycle policies transitioning to Glacier after 30 days ($0.004/GB), the cost structure is: 500TB in Standard ($11,500) + accumulated Glacier storage. After 12 months, total storage is ~6PB, with ~500TB in Standard and ~5.5PB in Glacier. Monthly cost stabilizes at approximately $11,500 (Standard) + $22,000 (Glacier) = $33,500/month. Production games reduce this by 80% with selective recording — only storing ranked matches, tournament matches, and anti-cheat flagged matches.
Voice audio and game simulation have fundamentally different resource profiles. Game simulation is CPU-bound (physics, collision detection, state validation) while voice is I/O-bound (audio mixing, UDP relay). Integrating them on the same instance means CPU spikes in complex game states (100 projectiles, explosions) could cause voice audio dropouts. Separate instances allow independent scaling: voice traffic scales with the number of talking players (~3.3 per match on average) while game server resources scale with game complexity. Additionally, voice relay instances can be shared across matches in the same region, while game server instances are dedicated per match.
Sign in to join the discussion.
Ready to design your own Multiplayer Game Server?
Open the simulator, place components on the canvas, wire them up, and run a traffic simulation to see how your architecture performs under real load.
Open Simulator