Email Service — Multi-Stage Pipeline (IP Reputation + Webhooks)

The multi-stage pipeline with IP reputation management is the production-grade architecture used by email delivery platforms like Amazon SES, SendGrid, Postmark, and Mailchimp. It solves the two critical limitations of the queue-based approach: (1) the lack of transactional/bulk path separation that allows a bad bulk campaign to destroy transactional deliverability, and (2) the absence of per-IP reputation scoring that enables automatic detection and quarantine of degraded sending IPs.

The fundamental insight is that transactional emails and bulk marketing emails have fundamentally different requirements. Transactional emails (password resets, 2FA codes, order confirmations) are time-sensitive — a password reset that arrives 30 minutes late is useless. They must be sent from high-reputation IPs that are never contaminated by bulk campaign bounce rates. Bulk marketing emails (newsletters, promotions) are latency-tolerant but volume-heavy — sending 100 million emails in a single campaign requires careful per-ISP rate shaping to avoid triggering spam filters.

Sharing a single delivery path means a bulk campaign with a 3% bounce rate (a bad but not unusual recipient list) degrades the shared IP pool's reputation score, which then throttles or blocks transactional emails that share those IPs. A user waiting for a password reset email that never arrives because a marketing campaign poisoned the IP is the worst-case failure mode for an email platform. Separating transactional and bulk into dedicated Kafka streams with dedicated sender pools and dedicated IP pools eliminates this cross-contamination entirely.

Per-IP reputation scoring is the second critical addition. The system tracks bounce rate, complaint rate, and spam trap hits for every sending IP address. When an IP's bounce rate exceeds 5%, it is automatically quarantined — removed from the active sending pool until its metrics recover. This prevents a single bad campaign from permanently destroying an IP's reputation. IP warming schedules control how quickly new IPs ramp up volume (starting at 1K/day, doubling weekly until reaching full capacity). Without warming, a new IP sending 100K emails on day one would be immediately flagged by ISPs as suspicious.

DKIM/SPF/DMARC signing via a dedicated SigningService adds cryptographic authentication to every outbound email. DKIM (DomainKeys Identified Mail) signs the email headers and body with an Ed25519 private key, allowing receiving ISPs to verify the email was not tampered with in transit. SPF (Sender Policy Framework) declares which IPs are authorized to send on behalf of the domain. DMARC (Domain-based Message Authentication, Reporting, and Conformance) ties DKIM and SPF together with a policy for handling authentication failures. Without these, deliverability drops from 99%+ to under 50% — most ISPs classify unauthenticated emails as potential spam.

Bounce and complaint feedback loops close the reputation management loop. ISPs publish feedback loop (FBL) reports when recipients mark emails as spam. The BounceProcessor consumes these reports along with SMTP bounce notifications, updates the sending IP's reputation metrics in ReputationDB, and adds permanently-failed addresses to the SuppressionCache. This creates a self-correcting system: bad addresses are automatically suppressed, degraded IPs are automatically quarantined, and healthy IPs continue serving traffic.

Webhook delivery provides real-time event notifications to senders. Instead of requiring senders to poll an events API at high frequency, the WebhookWorker pushes delivery events (sent, bounced, complained, opened, clicked) to the sender's configured webhook URL as they occur. This reduces API polling load by 100x and enables senders to react to bounces and complaints in real time — updating their own databases, pausing campaigns with high bounce rates, or triggering re-engagement flows.

This architecture appears in senior system design interviews at Amazon (SES), Google (Gmail infrastructure), Microsoft (Outlook), Twilio (SendGrid), and any company operating email at scale. Interviewers expect candidates to articulate why transactional/bulk separation is critical, explain the IP reputation lifecycle (warming, monitoring, quarantine), and reason about the trade-offs of adding operational complexity (12+ components) for deliverability guarantees.

The multi-stage pipeline uses 12+ components organized into three layers: an API layer that accepts and classifies emails, dual delivery pipelines for transactional and bulk traffic, and a feedback layer for bounce processing and webhook delivery.

The API layer consists of the Client, API Gateway, Load Balancer, and EmailService. The API Gateway authenticates API keys (~3ms), enforces per-tenant rate limits (200K RPS cap), and routes to the Load Balancer. The EmailService (20 pods, 100 threads each = 2,000 concurrent) handles all inbound requests. For transactional emails (POST /api/v1/emails), the service validates the request, optionally renders a template from TemplateDB, and publishes to TransactionalStream (Kafka). For bulk campaigns (POST /api/v1/campaigns), it fans out the recipient list and publishes individual messages to BulkStream (Kafka). Both return 202 Accepted immediately.

The dual delivery pipelines are the core architectural innovation. TransactionalStream (Kafka, 32 partitions, partitioned by message_id) feeds TransactionalSender (20 workers). BulkStream (Kafka, 64 partitions, partitioned by recipient_domain) feeds BulkSender (100 workers). Both sender types follow the same processing flow: check SuppressionCache (Bloom filter, ~1ms), call SigningService for DKIM/SPF/DMARC headers (~5ms), deliver via SMTP (~50ms), write delivery events to ReputationDB. The critical difference is that BulkSender also reads per-IP reputation scores from ReputationDB to select sending IPs and applies per-ISP rate limits — throttling sends to ISPs where the IP's bounce rate is elevated.

TransactionalSender uses a dedicated pool of high-reputation IPs that are reserved exclusively for transactional traffic. These IPs have consistently low bounce rates (<0.5%) because transactional emails go to addresses that the user has actively used (they just logged in, placed an order, requested a password reset). BulkSender uses a separate pool of IPs, some of which may be in warming stages. If a bulk campaign degrades one of these IPs, it has zero impact on transactional deliverability.

The SigningService (10 pods, 200 threads each) centralizes DKIM key management. Ed25519 private keys are stored in AWS Secrets Manager and rotated quarterly. Both TransactionalSender and BulkSender call SigningService before every SMTP delivery. Signing adds ~5ms per email but is non-negotiable for deliverability — unsigned emails are classified as spam by most ISPs.

The feedback layer consists of BounceProcessor and WebhookWorker. BounceProcessor (10 workers) consumes SMTP bounce notifications and ISP FBL complaint reports. Hard bounces (550 — invalid address) immediately add the address to SuppressionCache and increment the sending IP's bounce counter in ReputationDB. Soft bounces (421 — temporary failure) increment a per-address counter; 3 soft bounces promote to hard bounce. IPs exceeding 5% bounce rate are auto-quarantined in ReputationDB. WebhookWorker (20 workers) reads delivery events from ReputationDB and POSTs them to tenant-configured webhook URLs with HMAC signatures for verification, retrying with exponential backoff on failure.

SuppressionCache (Redis, 6 nodes) holds the Bloom filter of 10B suppressed addresses (~12GB). ReputationDB (PostgreSQL, 32 partitions) stores per-IP reputation metrics, delivery event logs, webhook configurations, and IP warming schedules. TemplateDB (PostgreSQL, 16 partitions) stores email templates with per-tenant namespacing.