What is streaming data?
Streaming data is data generated continuously as a sequence of small events and processed as each event arrives, instead of being gathered into files and processed on a schedule. Typical sources are machine sensors, point-of-sale terminals, GPS units on trucks, website clicks, payment authorizations, and the change feeds that change data capture produces from databases. A streaming platform receives the events, keeps them in order, and lets many consumers read them independently. Processing jobs filter, aggregate, or enrich the events as they pass. Most companies stream the few sources that need a fast reaction, batch everything else, and land both in the same lakehouse.
Why streaming data matters
Some decisions lose their value if they wait for the nightly load. A payment processor scoring a transaction for fraud must answer before the authorization completes. A plant monitoring bearing vibration wants an alert while the machine can still be stopped safely. Streaming data pairs naturally with real-time inference: the event arrives, a model scores it, and the action follows in the same second. A data historian applies the same idea to equipment readings in a plant.
What to watch for
- Confirm the business decision needs sub-minute data before paying the engineering cost of streaming, because most reports do not.
- Design for events that arrive late or twice, since networks and devices guarantee neither order nor uniqueness.
- Define how long the stream retains events, which sets how far back a new consumer can replay.