Instagram — CDN + Presigned Upload (Async Processing)

The CDN + presigned upload approach to Instagram represents the industry-standard architecture for media-heavy applications at production scale. It solves the three critical bottlenecks of the naive approach: API server bandwidth saturation from proxying image bytes, slow synchronous thumbnail generation blocking upload responses, and absence of CDN causing global latency penalties for image delivery.

The key architectural insight is that the API server should never touch image bytes. When a user uploads a photo, the client first calls POST /api/v1/media/upload, which returns a presigned S3 URL in approximately 2ms. The client then uploads the 2MB image directly to S3 using this presigned URL — the API server is completely bypassed for the actual data transfer. This eliminates the 1GB/sec bandwidth bottleneck (at 500 uploads/sec) that makes the naive approach unscalable. The API server's role is reduced to a lightweight URL-signing operation.

Once the image lands in S3, an event is published to Kafka (upload-complete topic). ImageWorker consumers download the original from S3 and create three resized variants: thumbnail (150px, ~15KB), medium (600px, ~80KB), and large (1080px, ~250KB) using libvips. Each variant is uploaded back to S3 and made available via CloudFront CDN. The entire processing pipeline is asynchronous — the upload response returns in under 200ms, and variants are available within 5-10 seconds. The client shows a loading placeholder during this brief interval.

The feed system pre-computes per-user feeds in Redis. When a user creates a post, PostService writes the post record to PostgreSQL and pushes the post_id to each follower's feed cache in Redis. FeedReader serves feed requests by reading from Redis via LRANGE — at 92% cache hit rate, most feed reads complete in under 5ms from cache. On cache miss, FeedReader falls back to querying PostgreSQL.

CloudFront CDN serves all image reads from 200+ edge locations worldwide. At 95% cache hit rate, only 5% of image reads reach the S3 origin. Popular images (viral posts, celebrity content) stay hot at edge locations with long TTLs (86400s for immutable media). A user in Tokyo receives images in approximately 10ms from the nearest edge location instead of 180ms from the S3 origin.

The primary trade-offs of this approach are eventual consistency (image variants may not be ready for 5-10 seconds after upload), chronological feed ordering (no ML ranking), and celebrity write amplification (a user with 50M followers triggers 50M Redis cache writes per post). The V2 variant addresses all three: hybrid fanout eliminates celebrity write amplification, ML ranking replaces chronological ordering, and Cassandra provides linear-scale post storage.

Interviewers expect candidates to explain the presigned URL pattern (why the API server should not proxy image bytes), reason about async image processing trade-offs (eventual consistency vs upload latency), discuss CDN cache hit rates and their impact on origin load, and identify the celebrity write amplification problem as the motivation for the hybrid fanout approach in V2.

The CDN + presigned upload architecture uses ten components organized into four layers: traffic ingestion (ReadClient, UploadClient, ApiGateway, MainLB), application services (FeedReader, PostService), data stores (FeedCache/Redis, PostDB/PostgreSQL, MediaStore/S3), async pipeline (ImageStream/Kafka, ImageWorker), and content delivery (CloudFront CDN).

The upload path (write) handles 15% of traffic. UploadClient sends POST /api/v1/media/upload to the ApiGateway, which authenticates the JWT token (~3ms) and routes to PostService via MainLB. PostService generates a presigned S3 URL (~2ms) and returns it to the client. The client uploads the 2MB image directly to MediaStore (S3) — the API server never sees the image bytes. After successful S3 upload, PostService publishes an upload-complete event to ImageStream (Kafka). ImageWorker consumes the event, downloads the original from S3, creates three resized variants using libvips, uploads them back to S3, and updates PostDB with CDN URLs.

The read path handles 85% of traffic. ReadClient sends GET /api/v1/feed to the ApiGateway, which routes to FeedReader via MainLB. FeedReader checks FeedCache (Redis) for the pre-computed feed — at 92% hit rate, the feed is served from cache in ~2ms. The response contains post metadata with CDN URLs for image variants. The client fetches images directly from CloudFront CDN (~10ms at edge). On cache miss, FeedReader queries PostDB (PostgreSQL) and reconstructs the feed.

FeedCache (Redis) stores pre-computed per-user feeds as sorted lists of post IDs. When a user creates a post, PostService writes the post_id to each follower's feed list in Redis. This fanout-on-write approach enables O(1) feed reads but creates O(followers) write amplification per post — the critical limitation that the V2 variant addresses with hybrid fanout.

PostDB (PostgreSQL) is the source of truth for posts, users, follows, and media metadata. Indexed by (author_id, created_at) for feed reconstruction on cache miss. 32 partitions with 3 replicas for durability.

ImageStream (Kafka) decouples the upload path from image processing. 16 partitions for ImageWorker parallelism. Messages are small (~512 bytes) containing media_id and S3 key — the actual image is read from S3 by the worker.

CloudFront CDN is configured as an S3 origin for the image bucket. 200+ edge locations provide sub-100ms image delivery worldwide. Long TTL (86400s) on immutable media means images are cached at edge for 24 hours. Cache invalidation is not needed because image URLs include the media_id (immutable — a new upload creates a new URL).