Introduction
The transport/delivery mechanism for continuously moving individual records (“Event(s)”) from producers to consumers as they happen, in order, often with persistence/replay capability.
- It’s about getting data from point A to point B reliably and in real time — not necessarily transforming it.
- Events are usually immutable facts: “user clicked button X at time T.”
Think of event streams as a flow of events traveling through a river. Each event is captured and can be processed by downstream consumers.
When we build applications there are times where the amount of data to process is unknown and we want to capture it (e.g. user interactions with a shopping cart online), we can collect this data and then process it in real-time downstream (e.g. for analytics or reporting for example).
Data loses value over time. Many companies want to process information in real-time to make decisions. Event streaming can help.
Use Cases
Event streams can be great for processing real-time data.
Some common examples where you may find event-streaming is handling IOT events, gaming, real-time applications or using streams to collect user information for analytics/metrics downstream.
You can also connect streams together to process information (Event Stream processing), streams read output from other streams and can process this into new information.
Log based brokers
Some brokers use a log-based approach to capture messages/events in an append only system. Think of this as a ledger, new information is added onto the stream, and consumers can consume this information from either the start or choose a starting location (usually an offset).
Examples
- Apache Kafka
- Amazon Kinesis
- Google Pub/Sub (Publisher-Subscriber Architecture)
These event streaming platforms. They durably store and route streams of events between systems.