VetoraReal-Time Chat — Naive (HTTP Polling + SQL)
The simplest chat system: clients poll every 2 seconds via HTTP GET, all messages stored in PostgreSQL. Demonstrates why push-based delivery (WebSocket) is essential once you exceed a few hundred concurrent users.
Problem Statement
Real-time chat is one of the most frequently asked system design interview questions because it touches nearly every core distributed systems concept: persistent connections, message ordering, delivery guarantees, presence detection, and storage at scale. The naive HTTP polling approach is where every candidate should start — it establishes the baseline that makes the improvements in WebSocket-based architectures measurable and concrete.
The fundamental challenge of chat is delivering messages with low perceived latency. Users expect messages to appear 'instantly' — any delay beyond 200-300ms feels sluggish. The naive approach uses HTTP long-polling: each client sends a GET request every 2 seconds asking 'are there any new messages since my last check?' The server queries PostgreSQL, returns any new messages (usually none), and the client waits 2 seconds before polling again. This creates an average delivery delay of 1 second (half the polling interval) and a worst case of 2 seconds.
The polling model's fatal flaw is not the delivery delay — it is the wasted load. In a typical chat workload, users send messages infrequently (a few per minute during active conversation) but poll continuously (every 2 seconds). This means 95% or more of poll responses return empty results. Each empty poll still requires an HTTP connection, a database query (SELECT with index scan), query planning, and response serialization. At 1,000 concurrent users polling every 2 seconds, the database handles 500 queries per second — the vast majority returning zero rows. The database connection pool saturates, write latency for actual message sends degrades, and the system hits a hard ceiling.
This template makes the polling waste visible and quantifiable. Run the simulation at 500, 1,000, and 2,000 concurrent users and watch the database utilization climb from comfortable to saturated. The comparison with the Kafka Fan-Out variant (WebSocket push) demonstrates a 10x reduction in database load for the same message throughput — because push-based delivery only generates database traffic when actual messages exist.
Chat system design appears in interviews at WhatsApp, Slack, Discord, Telegram, Meta, Google, and virtually every company with a messaging product. Interviewers expect candidates to start with the polling approach, identify its scaling limitations, and propose WebSocket push as the first optimization. This template provides the concrete numbers to support that discussion.
Architecture Overview
The naive chat system is a four-component linear architecture: Client, Load Balancer, Chat Service, and PostgreSQL database. There is no WebSocket server, no message queue, no cache, no presence service, and no separation between the message send path and the message receive path.
All traffic enters through the Load Balancer, which distributes requests across Chat Service pods using round-robin. The Load Balancer adds approximately 1.5ms of routing latency and supports up to 5,000 RPS — well above the system's actual ceiling, which is determined by the database. The Load Balancer handles both message sends (POST) and message polls (GET) identically since both are standard HTTP requests.
The Chat Service is a stateless REST API running on 3 pods with 50 threads each. It handles three operations: (1) message send — validate the request, INSERT into the messages table, UPDATE the conversations table's last_message_at field for sort ordering; (2) message poll — SELECT from messages WHERE conversation_id IN (user's conversations) AND created_at > since_timestamp; (3) conversation list — SELECT from conversations JOIN conversation_members WHERE user_id = requesting_user ORDER BY last_message_at DESC. The service processing time is approximately 8ms (request parsing, validation, response serialization), but total response time is dominated by database operations.
PostgreSQL stores four tables: users, conversations, conversation_members, and messages. The messages table is the hottest — it receives INSERTs on every send and SELECT scans on every poll. A B-tree index on (conversation_id, created_at) supports poll queries but causes write amplification on every INSERT (the index must be updated alongside the heap row). The conversations table receives UPDATE operations on every message send (updating last_message_at), creating row-level lock contention when multiple users send to the same conversation simultaneously.
The system has no redundancy at the data layer. A single PostgreSQL primary handles all reads and writes. If the database fails, both message sending and receiving stop entirely. There is no message cache, so every poll query hits the database directly. There is no deduplication — if the client retries a message send (network timeout), the server records two messages. There is no presence tracking — users cannot see whether conversation partners are online. There is no delivery receipt mechanism — senders never know if recipients received their messages.
The concrete scaling ceiling is approximately 1,000 concurrent users. At this point, 500 poll queries per second plus message send writes consume 60-70% of the database connection pool. Beyond 1,000 users, connection wait times spike, poll latency degrades from 15ms to 100ms+, and message send latency exceeds the 200ms SLO. The system degrades gracefully (slower, not failing) until approximately 2,000 users, where connection pool exhaustion causes outright errors.
Architecture Preview
Request Flow — HTTP Polling Chat
This sequence diagram shows the two primary flows in the naive chat system: message sending and message polling. The critical insight is the polling waste — the recipient's client polls every 2 seconds regardless of whether new messages exist. In a typical chat workload, 95% of polls return empty results, yet each empty poll still consumes a database connection, executes an index scan, and returns an HTTP response. The sender's message may sit in the database for up to 2 seconds before the recipient discovers it on the next poll cycle.
Step-by-Step Walkthrough
- 1Sender sends POST /messages with conversation_id and message content to the Load Balancer
- 2Load Balancer routes to a Chat Service pod via round-robin (~1.5ms routing overhead)
- 3Chat Service executes INSERT INTO messages to persist the message (~60ms with index update on conversation_id, created_at)
- 4Chat Service executes UPDATE conversations SET last_message_at = now() to update sort order (~15ms with row-level lock if another send is concurrent)
- 5Chat Service returns 201 Created — total ~90ms at low load, higher under contention
- 6Every connected client polls GET /messages?since=<last_seen_timestamp> every 2 seconds
- 7Chat Service executes SELECT from messages WHERE conversation_id IN (...) AND created_at > since — ~12ms even when returning zero rows (index seek cost)
- 895% of poll responses contain zero new messages but still consume a database connection and HTTP round-trip
Pseudocode
// Message send — synchronous dual-write
async function sendMessage(conversation_id, sender_id, content):
// Persist message
msg = await db.execute(
"INSERT INTO messages (message_id, conversation_id, sender_id, content, created_at)
VALUES (gen_random_uuid(), $1, $2, $3, now())
RETURNING message_id, created_at",
[conversation_id, sender_id, content]
) // ~60ms (INSERT + B-tree index update)
// Update conversation sort order
await db.execute(
"UPDATE conversations SET last_message_at = $1
WHERE conversation_id = $2",
[msg.created_at, conversation_id]
) // ~15ms (row lock if concurrent sends)
return { status: 201, message_id: msg.message_id }
// Message poll — runs every 2 seconds per connected client
async function pollMessages(user_id, since_timestamp):
// Get user's conversations
conv_ids = await db.execute(
"SELECT conversation_id FROM conversation_members WHERE user_id = $1",
[user_id]
)
// Fetch new messages across all conversations
messages = await db.execute(
"SELECT * FROM messages
WHERE conversation_id = ANY($1)
AND created_at > $2
ORDER BY created_at ASC LIMIT 50",
[conv_ids, since_timestamp]
) // ~12ms even when returning 0 rows (index seek)
return messages // 95% of the time: []Database Schema (ER Diagram)
The schema reveals the fundamental tension of the naive approach: the messages table serves both write-heavy (message sends) and read-heavy (polling) workloads simultaneously on a single database instance. The B-tree index on (conversation_id, created_at) is essential for poll query performance but creates write amplification — every INSERT must update the index tree structure in addition to appending the heap row. The conversations table's last_message_at field creates row-level lock contention when multiple users send to the same conversation concurrently.
Step-by-Step Walkthrough
- 1The users table stores account data — static after creation, rarely queried except during authentication
- 2The conversations table tracks metadata and last_message_at for sort ordering. Every message send triggers an UPDATE to last_message_at, creating row-level lock contention in active group chats
- 3conversation_members is the many-to-many join table. Queried on every poll (to determine which conversations to check) and every conversation list request
- 4The messages table is the hottest table: INSERT on every send, SELECT on every poll. The B-tree index on (conversation_id, created_at DESC) enables efficient poll queries but adds write amplification to every INSERT
- 5No read_receipts table — the naive approach has no delivery receipt mechanism. No presence table — online status is not tracked
Pseudocode
-- Schema DDL
CREATE TABLE users (
user_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
username VARCHAR(255) UNIQUE NOT NULL,
created_at TIMESTAMPTZ DEFAULT now()
);
CREATE TABLE conversations (
conversation_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
type VARCHAR(10) NOT NULL CHECK (type IN ('direct', 'group')),
name VARCHAR(255),
last_message_at TIMESTAMPTZ DEFAULT now(),
created_at TIMESTAMPTZ DEFAULT now()
);
CREATE INDEX idx_conv_last_msg ON conversations (last_message_at DESC);
CREATE TABLE conversation_members (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
conversation_id UUID REFERENCES conversations(conversation_id),
user_id UUID REFERENCES users(user_id),
joined_at TIMESTAMPTZ DEFAULT now(),
UNIQUE (conversation_id, user_id)
);
CREATE INDEX idx_members_user ON conversation_members (user_id, conversation_id);
CREATE TABLE messages (
message_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
conversation_id UUID REFERENCES conversations(conversation_id),
sender_id UUID REFERENCES users(user_id),
content TEXT NOT NULL,
message_type VARCHAR(10) DEFAULT 'text',
created_at TIMESTAMPTZ DEFAULT now()
);
CREATE INDEX idx_messages_conv_time ON messages (conversation_id, created_at DESC);
-- Poll query (runs every 2 seconds per user)
SELECT * FROM messages
WHERE conversation_id = ANY(
SELECT conversation_id FROM conversation_members WHERE user_id = $1
) AND created_at > $2
ORDER BY created_at ASC LIMIT 50;
-- Cost: ~12ms even for 0 results (index seek on conversation_id)Key Design Decisions
HTTP Polling (2-Second Interval)
Choice
Clients poll GET /messages?since=timestamp every 2 seconds
Rationale
HTTP polling is the simplest possible delivery mechanism — any HTTP client can implement it with a setInterval and a fetch call. No WebSocket library, no persistent connection management, no reconnection logic. The 2-second interval is a pragmatic trade-off: shorter intervals (500ms) increase server load 4x, longer intervals (10s) make the chat feel unresponsive. The fundamental cost is that N users generate N/2 queries per second to the database regardless of message activity.
Single PostgreSQL Database
Choice
One database for users, conversations, and messages
Rationale
A single PostgreSQL instance eliminates replication lag, split-brain, and failover complexity. ACID transactions ensure that a message INSERT and the corresponding conversations UPDATE are atomic. The cost is that poll reads and message writes compete for the same resources — connection pool, buffer cache, WAL bandwidth, and I/O. This mutual degradation is the primary scaling bottleneck.
No Message Cache
Choice
Every poll query hits PostgreSQL directly
Rationale
A Redis cache for recent messages (last 50 per conversation) would reduce database poll load by 80-90% since most polls request recent messages. The naive approach skips this for simplicity — at the cost of making the database the hard bottleneck. Adding a cache is the second-easiest optimization after switching to WebSocket (and often the first thing interviewers expect candidates to propose).
No Presence Tracking
Choice
No online/offline indicators
Rationale
Presence requires either WebSocket connection events (user connects = online, disconnects = offline) or explicit heartbeats with TTL-based expiry. In the polling model, the server has no persistent connection to detect disconnect events. You could infer presence from poll frequency ('online if polled within 10 seconds'), but this is unreliable and adds query complexity. The naive approach omits presence entirely.
No Delivery Receipts
Choice
No sent/delivered/read status tracking
Rationale
Delivery receipts require the server to track per-message, per-recipient state (sent, delivered, read) and propagate state changes. In the polling model, the recipient would need to send an acknowledgment on each poll that returns messages, and the sender would need to poll for receipt updates — doubling the already-heavy poll traffic. Modern chat systems use WebSocket push for receipts, which is free once the connection exists.
Scale & Performance
Target RPS
~500 sustained (polling ceiling)
Latency (p99)
1-2s delivery delay (polling interval)
Storage
~2 GB/month at modest scale
Availability
~99% (single DB, no redundancy)
Time & Space Complexity
| Operation | Time | Space | Notes |
|---|---|---|---|
| Send message (POST /messages) | O(1) INSERT + O(log N) index update | O(1) per message (~500 bytes with index) | N = messages in conversation. Index update cost grows logarithmically as conversation history grows. |
| Poll messages (GET /messages?since=) | O(log N + K) index scan | O(K) result set | N = messages in conversation, K = new messages since last poll. 95% of polls return K=0 but still pay O(log N) index seek. |
| List conversations (GET /conversations) | O(M log M) sort by last_message_at | O(M) conversation list | M = user's conversation count. JOIN with conversation_members adds cost. |
Database Schema (HLD)
Append-only message log — every chat message is an INSERT. The hottest table: receives writes on every send and SELECT scans on every 2-second poll. B-tree index on (conversation_id, created_at) enables efficient poll queries but causes write amplification — each INSERT updates both the heap and the index. At 1K users, poll queries alone generate 500 index scans/sec.
Indexes: idx_messages_conv_time ON (conversation_id, created_at DESC)
No deduplication — a retried message send creates a duplicate row. At 200 msgs/sec, table grows ~3 GB/day with indexes.
Conversation metadata with last_message_at for sort ordering. Updated on every message send (UPDATE SET last_message_at = now()), creating row-level lock contention when multiple users send to the same conversation simultaneously. Also queried on conversation list (JOIN with conversation_members).
Indexes: idx_conv_last_msg ON (last_message_at DESC)
Row-level locks on UPDATE cause contention in active group chats. A 500-member group with 10 concurrent senders serializes on this lock.
Many-to-many join table linking users to conversations. Queried on every poll (to determine which conversations the user belongs to) and on conversation list. Composite index on (user_id, conversation_id) for fast membership lookups.
Indexes: idx_members_user ON (user_id, conversation_id)
At 1K users with avg 20 conversations each, table has ~20K rows. Small and fast, not a bottleneck.
User account table. Queried on authentication and conversation member lookups. Small table, rarely a bottleneck.
Indexes: idx_users_username ON (username)
Static data — rarely written after account creation.
What-If Scenarios
1,000 users join simultaneously (thundering herd)
Impact
All 1,000 users start polling within the same 2-second window. Database receives 500 poll QPS spike plus conversation list queries. Connection pool saturates, poll latency spikes to 500ms+, message sends queue behind polls.
Mitigation
Jitter the poll interval (randomize between 1.5-2.5 seconds) to spread load. Better: switch to WebSocket push to eliminate polling entirely.
500-member group message (fan-out storm)
Impact
One message sent to a 500-member group. No fan-out mechanism — all 500 members discover the message on their next poll. 500 poll queries within 2 seconds all hit the same conversation_id index range. Database handles the load but with elevated latency.
Mitigation
In the polling model, group messages are naturally 'rate-limited' by the poll interval. The real problem is the 2-second delivery delay for 500 people. WebSocket + Kafka fan-out solves this with async push.
Database primary failure
Impact
Single PostgreSQL instance — no read replicas, no failover. Both message sending and polling stop completely. All connected users see 'connection error' on their next poll attempt. Mean time to recovery depends on RDS automated backup restoration: 5-30 minutes.
Mitigation
Add a read replica for poll queries (reduces primary load and provides read availability during primary failure). Better: multi-AZ RDS deployment with automatic failover (~30 seconds downtime).
User reconnects after 24 hours offline
Impact
Client polls with since=24_hours_ago. The SELECT query scans a large range of the messages index across all user conversations. At 200 msgs/sec, 24 hours = 17.3M messages. Even with index, the query may scan thousands of rows across multiple conversations, taking 500ms-2s.
Mitigation
Paginate reconnect queries (LIMIT 50 per conversation). Add a Redis cache for recent messages to serve the common case. Implement conversation-level unread counts to avoid scanning all conversations.
Failure Modes & Resilience
| Component | Failure | Impact | Mitigation |
|---|---|---|---|
| PostgreSQL Database | Connection pool exhaustion | New poll queries and message sends wait for a free connection. Latency spikes from 15ms to 500ms+. If queue fills, requests are rejected with 503 errors. Both read and write paths fail simultaneously. | Connection pooler (PgBouncer) to multiplex connections. Read replica for poll queries. Cache layer (Redis) to reduce direct DB hits. |
| Chat Service | Pod crash (1 of 3) | Load Balancer detects unhealthy pod within 10 seconds and stops routing. Remaining 2 pods absorb the traffic (150 → 100 threads each). At low load, no user impact. At high load, thread pool approaches saturation. | Auto-scaling group with min 3 pods. Health check interval reduced to 5 seconds. Graceful shutdown drains in-flight requests. |
| Load Balancer | ALB availability zone failure | ALB is multi-AZ by default — single AZ failure is handled automatically. Cross-AZ failover adds 2-5ms latency. No user-visible impact unless both AZs fail. | Multi-AZ deployment (default for ALB). DNS failover to secondary region for catastrophic regional failure. |
| PostgreSQL Database | Slow query from conversation list JOIN | A user with 500+ conversations triggers a heavy JOIN query that holds a connection for 2-5 seconds. At 10 concurrent slow queries, 10% of the connection pool is locked, degrading all other operations. | Query timeout (SET statement_timeout = 500ms). Paginate conversation list (LIMIT 20). Denormalize: store conversation list as a JSON column on the users table. |
Scaling Strategy
Vertical scaling first: upgrade the RDS instance from db.r7g.xlarge to db.r7g.2xlarge (8 vCPU, 64 GB) to double the connection pool and query throughput. This buys time to approximately 2K concurrent users. Horizontal scaling: add a read replica to offload poll queries from the primary (poll queries are read-only). This approximately doubles the poll capacity. Beyond 3-4K users, the polling model itself is the bottleneck and no amount of database scaling helps — the correct scaling strategy is to switch to WebSocket push (the Kafka Fan-Out variant).
Monitoring & Alerting
Key metrics to monitor: (1) Database connection pool utilization — alert at 70%, critical at 90%. This is the primary bottleneck indicator. (2) Poll query latency p99 — should stay under 30ms; spikes indicate DB contention. (3) Empty poll ratio — track the percentage of polls returning zero messages to quantify waste. (4) Message send latency p99 — alert if exceeding 200ms SLO. (5) Active poll QPS — should equal approximately (concurrent_users / 2); sudden drops indicate client disconnections. Dashboard: Grafana with PostgreSQL metrics (pg_stat_activity for connection usage, pg_stat_user_tables for table I/O). Alerting: PagerDuty integration for connection pool exhaustion and latency SLO breaches.
Cost Analysis
Monthly cost at 1K concurrent users: RDS db.r7g.xlarge (4 vCPU, 32 GB) ~$350/month. ECS Fargate 3 pods (2 vCPU, 4 GB each) ~$180/month. ALB ~$25/month + $0.008/LCU-hour. Total: ~$555/month. Cost per concurrent user: ~$0.55/month. This is relatively expensive for the scale — the Kafka Fan-Out variant serves 10x more users on approximately 3x the infrastructure cost, yielding a 3x improvement in cost-per-user. The naive approach's cost inefficiency comes from the database being oversized to handle poll query load that the WebSocket variant eliminates entirely.
Security Considerations
Security in the naive chat is straightforward but limited: (1) Authentication: session token or JWT in the Authorization header on every HTTP request. Polling means re-authentication every 2 seconds per user — ensure token validation is cached (not hitting the auth DB on every poll). (2) No end-to-end encryption: messages are stored in plaintext in PostgreSQL. Server operators can read all messages. Adding E2E encryption (Signal protocol) would prevent server-side search but is orthogonal to the architecture. (3) Rate limiting: essential to prevent poll abuse. Limit each user to 1 poll/second (above the 0.5/sec design rate) to prevent clients from polling more aggressively. (4) SQL injection: use parameterized queries for the since timestamp and conversation_id filters in poll queries. (5) Message content validation: sanitize HTML/script content to prevent XSS when messages are rendered in web clients.
Deployment Strategy
Blue-green deployment: deploy the new Chat Service version to a parallel ECS service (green), run integration tests against the green deployment, then switch the ALB target group from blue to green. Rollback is instant (switch back to blue). Database migrations must be backward-compatible since blue and green share the same PostgreSQL instance. For the naive architecture, zero-downtime deployment is straightforward because the service is stateless — no WebSocket connections to drain, no in-memory state to migrate. The 2-second poll interval means clients automatically discover the new version on their next poll cycle.
Real-World Examples
- •Early-stage startup MVPs (YC companies often start with polling for simplicity)
- •Internal team chat tools (Basecamp's original chat used polling before switching to WebSocket)
- •IRC web clients (classic IRC over HTTP used polling before EventSource/WebSocket)
- •Email notification checks (Gmail's original new-mail check was HTTP polling at 60-second intervals)
Solution Comparison
| Variant | Tier | Latency | Throughput | Cost | Complexity | Reliability |
|---|---|---|---|---|---|---|
| Naive (HTTP Polling) | T1 | 1-2s delivery delay | ~1K concurrent users | $555/month (1K users) | Low — standard HTTP, no WS | 99% (single DB, no failover) |
| Kafka Fan-Out (WebSocket) | T2 | <100ms delivery | 10K-1M concurrent users | $2,500/month (100K users) | Medium — WS, Kafka, Redis | 99.9% (replicated components) |
| Multi-Region Global (CRDT) | T4 | <50ms intra-region, <500ms cross-region | 500K+ concurrent users | $15,000+/month (multi-region) | Very High — CRDT, multi-region Kafka | 99.99% (multi-region failover) |
This template is for educational and illustration purposes only. It may not represent the optimal production design for this problem. Real-world systems involve additional considerations (compliance, specific cloud provider constraints, organizational requirements) not captured here. Use this as a starting point for discussion, not as a production blueprint.
Frequently Asked Questions
Why is real-time chat the most popular system design interview question?
Chat combines almost every distributed systems concept in one problem: persistent connections (WebSocket vs polling), message ordering (wall clock vs logical clocks), delivery guarantees (at-least-once vs exactly-once), presence detection (heartbeats, TTLs), storage scaling (time-series writes, range reads), fan-out (group messages to N recipients), and offline delivery (push notifications). Companies like WhatsApp, Slack, Discord, and Telegram ask it because it maps directly to their production systems.
At what user count does HTTP polling break down?
In this simulation, the inflection point is around 1,000 concurrent users. At that point, poll queries (500 QPS) consume 60-70% of the database connection pool. Message send latency starts climbing as writes compete with poll reads for connections. By 2,000 users (1,000 poll QPS), the connection pool is saturated and the system starts returning errors. The exact number depends on database instance size, but the pattern is universal: polling creates O(N) load regardless of message volume.
How does the 2-second polling interval affect user experience?
A 2-second polling interval means messages have an average delivery delay of 1 second and a worst case of 2 seconds. In a fast conversation, this creates a noticeable lag — you type a message, your partner doesn't see it for 1-2 seconds, they respond, you don't see the response for another 1-2 seconds. Total round-trip conversation latency is 2-4 seconds versus sub-200ms with WebSocket push. Users describe polling-based chat as 'laggy' or 'broken' compared to push-based alternatives.
Why not use HTTP long-polling instead of short-polling?
Long-polling (server holds the request open until a new message arrives or a timeout expires) reduces wasted empty responses but creates different problems: each user holds a server thread/connection open for up to 30 seconds, consuming connection pool slots. At 1,000 users, that is 1,000 held connections — more resource-intensive than 500 short poll queries per second. Long-polling is a half-measure; WebSocket is the correct solution for push-based delivery.
What is the first optimization you would make to this architecture?
Switch from HTTP polling to WebSocket push. This single change eliminates 95% of the wasted poll queries and reduces message delivery latency from 1-2 seconds to under 50ms. The Kafka Fan-Out variant demonstrates this: same message throughput, 10x lower database load, sub-100ms delivery. After WebSocket, the next optimization is adding a Redis cache for recent messages to further reduce database read load during scrollback and reconnect.
Related Templates
Discussion
Sign in to join the discussion.
Ready to design your own Real-Time Chat?
Open the simulator, place components on the canvas, wire them up, and run a traffic simulation to see how your architecture performs under real load.
Open Simulator