Pastebin — Object Storage + NoSQL TTL

The Object Storage + NoSQL TTL architecture is the advanced Pastebin design that addresses the two fundamental limitations of the RDBMS+Cache approach: large paste performance and TTL cleanup. By separating paste content (S3) from metadata (DynamoDB) and adding a CDN for edge caching, this variant handles arbitrarily large pastes, global-scale traffic, and automatic expiry without background jobs.

The key architectural insight is that paste content and paste metadata have fundamentally different access patterns and storage requirements. Content is a blob — ranging from a few bytes to 10MB — that is write-once and read-many. S3 is purpose-built for this: unlimited object size, 99.999999999% durability, and no impact on metadata query performance regardless of blob size. Metadata is small (approximately 200 bytes per paste), queried frequently, and needs TTL-based expiry. DynamoDB is purpose-built for this: single-digit millisecond reads at any scale, and native TTL that automatically deletes expired items without application intervention.

The separation solves the PostgreSQL TOAST problem from the RDBMS approach. In PostgreSQL, a 10MB paste fragments across approximately 1,250 TOAST pages. Reading such a paste requires fetching all pages — slower than a single S3 GET. More critically, when expired pastes are deleted, their TOAST pages must be vacuumed separately, consuming CPU and I/O during maintenance windows. With S3, a 10MB paste is a single object — reading and deleting it are both constant-time operations regardless of size.

DynamoDB native TTL eliminates the need for a background cleanup job. Each paste's metadata row includes an expires_at field containing a Unix epoch timestamp. DynamoDB automatically scans for and deletes items whose TTL has passed — within 48 hours of expiry, without any application logic. In the RDBMS approach, if the nightly cleanup job fails, expired pastes remain accessible and continue consuming storage. With DynamoDB TTL, expiry is a platform-level guarantee.

The CDN layer (CloudFront) adds edge caching for popular public pastes. While most Pastebin pastes have small audiences, some go viral — error logs shared on social media, code snippets referenced in blog posts, configuration files linked in documentation. Without a CDN, these viral pastes hammer the origin at thousands of requests per second. CloudFront absorbs this traffic at edge locations globally, serving cached content in approximately 5ms. The CDN applies only to public pastes — password-protected pastes bypass CloudFront entirely via Cache-Control: no-cache headers.

This architecture supports 10,000+ sustained RPS with global distribution. S3 and DynamoDB both scale automatically — there is no connection pool ceiling as in PostgreSQL. The trade-off is higher complexity (7 components vs 5 in the RDBMS approach) and higher per-request latency for small pastes (S3 GET at approximately 50ms vs PostgreSQL indexed read at approximately 12ms). For services where most pastes are small and traffic is moderate, the RDBMS+Cache approach is more cost-effective. For services handling large pastes, global traffic, or requiring zero-maintenance TTL cleanup, the Object Storage approach is the right choice.

The Object Storage + NoSQL TTL system uses seven components organized into a multi-tier architecture: Client, CDN (CloudFront), Load Balancer, PasteService, Redis HotCache (for metadata), DynamoDB (for metadata with native TTL), and S3 (for paste content). The key architectural decision is the separation of content storage from metadata storage, with each using the optimal service for its access pattern.

All traffic enters through the CDN (CloudFront). For public paste reads, CloudFront checks its edge cache first. At approximately 30% hit rate for popular pastes, about one-third of read traffic never reaches the origin. On CDN miss, the request routes to the Load Balancer, which distributes across PasteService pods. Paste creation requests and password-protected paste reads bypass the CDN and go directly to the Load Balancer.

PasteService is the central coordinator running on 4 pods with 100 threads each. The write path executes two sequential operations: (1) upload paste content to S3 as an individual object keyed by paste_id (approximately 100ms for a 5KB paste), then (2) write metadata to DynamoDB with the s3_key reference and native TTL set to the configured expiry timestamp (approximately 10ms). Content is written to S3 first because if the DynamoDB write fails, an orphaned S3 object is less harmful than a metadata entry pointing to nonexistent content. On successful write, PasteService also populates the Redis HotCache with the metadata for immediate read availability.

The read path has three tiers. First, PasteService checks the Redis HotCache for paste metadata (approximately 85% hit rate, 1ms latency). On cache miss, it queries DynamoDB for metadata (approximately 5ms). If the metadata exists (not TTL-deleted), PasteService fetches the paste content from S3 (approximately 50ms). The metadata cache stores only small records (paste_id, syntax, expires_at, s3_key, is_public) — content is always fetched from S3, keeping the Redis working set small and hit rate high.

DynamoDB runs in on-demand capacity mode, scaling automatically with traffic. The paste_metadata table has paste_id as the partition key and includes an expires_at field configured as the DynamoDB TTL attribute. DynamoDB automatically deletes expired items in the background — typically within a few hours of expiry, guaranteed within 48 hours. No background job, no cron, no operational burden. The application-level expiry check in PasteService is a safety net for the lag between expiry time and actual DynamoDB deletion.

S3 stores paste content as individual objects in a single bucket. S3 lifecycle policies are configured to delete objects older than the maximum paste lifetime (1 year), catching any orphaned objects from failed DynamoDB writes. S3 provides 99.999999999% (11 nines) durability and scales to any request rate without connection pool constraints. GET latency is approximately 50ms — higher than PostgreSQL's 12ms indexed read, but consistent regardless of object size.