Email Service — Queue-Based Pipeline (Kafka + Workers)

The queue-based email pipeline is the standard production architecture used by email delivery platforms handling millions of emails per day. It solves the fundamental problem that makes the naive synchronous approach unworkable: the 50-500ms SMTP blocking I/O that exhausts the thread pool at a few hundred emails per second.

The key architectural shift is decoupling the API response from SMTP delivery. Instead of blocking the caller until the SMTP handshake completes (200ms+ per email), the SubmitService validates the request, generates a message_id, publishes to a Kafka topic, and returns 202 Accepted in under 15ms. The actual email processing — template rendering, suppression checking, and SMTP delivery — happens asynchronously in dedicated worker pools, each consuming from their own Kafka topic.

The multi-stage pipeline architecture recognizes that email delivery has distinct processing phases with different failure modes and scaling requirements. Template rendering is CPU-bound (10ms per email for variable substitution and HTML generation). Suppression checking is memory-bound (Bloom filter lookup against 10B addresses in ~1ms). SMTP delivery is I/O-bound (50-500ms per email, dependent on the receiving ISP's response time). Separating these into distinct stages connected by Kafka queues means a slow ISP does not block template rendering, a rendering bug does not affect delivery of already-rendered emails, and each stage can scale independently.

The suppression check is the critical compliance addition over the naive approach. A Bloom filter in Redis holds 10 billion suppressed email addresses (hard bounces, unsubscribes, spam complaints) in approximately 12GB of memory, providing O(1) lookups in ~1ms with a 0.1% false positive rate. Without this check, the sender IP's bounce rate would climb above 5% within days, triggering ISP blacklisting. The Bloom filter trades a small false positive rate (occasionally suppressing a valid address) for massive memory efficiency — a traditional hash set would require 100GB+ for 10B entries.

The per-stage Kafka queues provide natural backpressure and retry isolation. During a 165K/sec bulk campaign burst, rendered emails queue up in the render-to-send Kafka topic while send workers drain at their sustainable rate (limited by SMTP I/O). If SMTP delivery fails for a batch of emails (soft bounce from an ISP), only the send stage retries — the email is not re-submitted or re-rendered. If template rendering fails (missing template, invalid variables), only the render stage retries — already-rendered emails in the send queue continue processing.

This architecture handles 12K emails per second sustained (transactional — order confirmations, password resets) and absorbs 165K/sec bulk campaign bursts through Kafka buffering. The submit tier (15 pods) handles the API burst; render workers (40) process templates at 160K/sec; send workers (80) deliver at 128K/sec with Kafka queue absorbing the overflow. The pipeline drains a 100M-email campaign in approximately 15 minutes.

The primary limitation of this variant is the lack of IP reputation management. All emails go through a single IP pool without transactional/bulk separation. A high-bounce-rate bulk campaign degrades the sending IP's reputation, which also affects transactional emails (password resets, 2FA codes) that share the same IPs. The Pipeline variant solves this with separate transactional and bulk delivery paths, per-IP reputation scoring, and DKIM/SPF/DMARC signing.

Email pipeline design appears in system design interviews at Amazon (SES), Mailchimp (Intuit), SendGrid (Twilio), Postmark, and any company operating email infrastructure. Interviewers expect candidates to explain why async decoupling is necessary, articulate the per-stage scaling rationale, and reason about the suppression list's role in deliverability.

The queue-based email system uses 10 components organized into a three-stage pipeline: submit, render, and send. Each stage has its own Kafka topic acting as a buffer and retry queue between stages.

All traffic enters through the API Gateway, which validates API keys (~3ms), enforces per-tenant rate limits (200K RPS cap), and routes to the SubmitService. The API Gateway is the single entry point for all email operations — transactional sends, bulk campaign submissions, and delivery status queries. Rate limiting at this layer prevents a single tenant from monopolizing outbound capacity during a campaign burst.

The SubmitService is a stateless REST API running on 15 pods with 100 threads each (1,500 concurrent connections). It handles three operations: (1) POST /api/v1/emails — validate request, generate message_id, publish to SubmitStream, return 202 Accepted; (2) POST /api/v1/campaigns — fan out a campaign's recipient list into individual messages published to SubmitStream; (3) GET /api/v1/messages/{id}/events — query delivery events from EventDB. The critical design choice is that the API response is decoupled from delivery — the client gets a message_id immediately and can poll for status later.

SubmitStream (Kafka, 64 partitions) is the first pipeline queue. It carries submitted emails from SubmitService to RenderWorker. During campaign bursts (165K/sec), SubmitStream absorbs the traffic spike while render workers process at their sustainable rate. Partitioned by message_id for even load distribution.

RenderWorker (40 instances) is the second pipeline stage. It consumes from SubmitStream, fetches the email template from TemplateCache (Redis, ~2ms hit rate of 98%), hydrates per-recipient variables (name, order ID, etc.), and publishes the fully rendered email to RenderStream. For raw HTML emails (no template), this is a pass-through. CPU-bound at ~10ms per email, 40 workers handle 4K renders/sec each = 160K/sec total.

RenderStream (Kafka, 64 partitions) is the second pipeline queue. It carries rendered emails to SendWorker. Partitioned by recipient_domain so all emails to the same ISP (gmail.com, yahoo.com) land on the same partition, enabling per-ISP send rate awareness.

SendWorker (80 instances) is the final pipeline stage. It consumes from RenderStream, checks the recipient against SuppressionCache (Bloom filter of 10B addresses, ~1ms), and if not suppressed, delivers via SMTP (~50ms per email). Delivery events (sent, bounced, suppressed) are written to EventDB. I/O-bound at ~50ms per email, 80 workers handle 1.6K sends/sec each = 128K/sec total. During campaign bursts, the gap between render throughput (160K/sec) and send throughput (128K/sec) is absorbed by the RenderStream Kafka queue.

TemplateCache (Redis, 3 nodes) caches email templates for fast hydration. 100K templates at ~10KB each = ~1GB working set. 98% hit rate means render workers rarely need a database fallback. SuppressionCache (Redis, 6 nodes) holds the Bloom filter of 10B suppressed addresses in ~12GB of memory. EventDB (DynamoDB, 64 partitions) stores delivery event logs for status queries and analytics.