Event Stream

Overview

An Event Stream (also known as a message queue or event bus) is a durable messaging platform that enables asynchronous, decoupled communication between services. Producers publish events (messages) to topics, and consumers subscribe to those topics to receive events. Unlike direct service-to-service calls, event streams decouple producers from consumers in time (the consumer does not need to be available when the event is published) and space (the producer does not need to know who consumes the event). This decoupling is the foundation of event-driven architectures and is essential for building scalable, resilient microservices.

Pub/sub (publish-subscribe) is the primary communication pattern. A producer publishes an order_placed event, and multiple independent consumers process it: an inventory service reserves stock, a notification service emails the customer, an analytics service records the conversion, and a fraud detection service checks for suspicious patterns. Each consumer reads the event independently at its own pace. Adding a new consumer requires zero changes to the producer — this is the power of event-driven loose coupling.

Partitioning enables horizontal scaling of both throughput and consumption. A topic is divided into partitions, and each partition maintains strict ordering of events. Events are assigned to partitions based on a partition key (e.g., order_id, user_id), ensuring all events for the same entity are processed in order. Consumers in a consumer group each handle a subset of partitions, providing parallel processing. Scaling throughput means adding partitions and consumers. Vetora models partition distribution and shows how partition count affects throughput and ordering guarantees.

Consumer groups enable multiple independent processing pipelines on the same event stream. Each consumer group maintains its own offset (position) in the stream, reading events independently. The inventory group and the analytics group each process all events, but each group can have multiple members that share the partition workload. If a consumer in a group fails, its partitions are automatically redistributed to healthy members (consumer rebalancing). Vetora models consumer group behavior, including rebalancing delays and their impact on processing latency.

Event replay is a unique advantage of durable event streams like Kafka. Unlike traditional message queues that delete messages after delivery, event streams retain messages for a configurable period (days, weeks, or indefinitely). Consumers can reset their offset to replay historical events — invaluable for rebuilding derived state after a bug fix, onboarding a new service that needs to catch up on historical events, or debugging production issues by replaying the exact sequence of events that caused a failure.

When to Use

Key Parameters in Vetora

Real-World Examples

Frequently Asked Questions

Related Components