Replication Lag and Read-Your-Writes

Replication lag is an inherent consequence of asynchronous replication. When a leader commits a write, it streams the change to followers, but those followers apply the change with some delay -- typically milliseconds under normal conditions, but potentially seconds or minutes during high write load, long-running transactions, schema migrations, or I/O saturation on followers. This delay creates a window during which different replicas have different versions of the data, and clients reading from lagging followers receive stale results. While eventual consistency guarantees that all replicas will converge to the same state given enough time, the application must handle the inconsistency window correctly to avoid confusing or incorrect user experiences.

The most commonly encountered anomaly is the read-your-writes violation. A user submits a form (write to leader), and the page immediately refreshes (read from a follower). If the follower has not yet received the write, the user sees the old data and believes their submission was lost. This is particularly jarring for profile updates, message sends, and order placements -- operations where users expect to see their action reflected instantly. The solution is a read-your-writes guarantee: for data that the current user has recently modified, route the read to the leader (or a synchronous follower known to have the latest data). This can be implemented via sticky sessions (route all requests from a user to the leader for a short window after a write), version tokens (include the write's position in the replication log with the response, and have followers wait until they have reached that position before responding), or application-level routing (always read 'own' data from the leader).

Monotonic read violations are a subtler problem. A user makes two sequential reads that happen to be routed to different followers with different amounts of lag. The first read hits a relatively up-to-date follower and returns recent data. The second read hits a lagging follower and returns older data. From the user's perspective, time went backward -- they saw a comment appear and then disappear, or a counter go from 42 to 38 and back to 42. The solution is monotonic reads: ensure that a given user always reads from the same replica (sticky sessions), or include a monotonic version token that prevents a replica from serving data older than what the client has already seen.

Consistent prefix reads address causal ordering violations: seeing an effect before its cause. If User A writes 'What is the score?' and User B writes 'It is 3-1' (causally dependent on seeing A's question), a third observer reading from a lagging follower might see B's answer before A's question, which is confusing. Consistent prefix reads guarantee that if a sequence of writes happens in a certain order, any reader will see them in that same order. This is harder to guarantee in partitioned or sharded databases where different shards may replicate at different speeds. Solutions include writing causally related data to the same partition or using causal consistency protocols with logical timestamps.

You update your profile picture on a social media app (write to leader). You immediately visit your profile page (read from a nearby follower). The follower has not received the update yet, so you see your old photo. Panicking, you upload the photo again. Now there are two uploads. Later, both replicas catch up and show the new photo -- but the double upload is a wasted operation caused by the lag. A read-your-writes guarantee would have routed your profile page read to the leader (or waited until the follower was caught up), showing you the new photo on the first try and avoiding the confusion.