The term "stream processing" might sound intimidating to many people. We often hear statements like:
- "Stream processing is too difficult to learn and use!" 😱
- "Stream processing is very expensive!" 😱
- "I don’t see any business use cases for stream processing!" 😱
However, we believe this isn't true. ❌
Streaming data is everywhere, generated from operational databases, messaging queues, IoT devices, and many other sources. People can leverage modern stream processing technology to easily address classic real-world problems, using SQL as the programming language.
In this repository, we provide a series of executable demos demonstrating how stream processing can be applied in practical scenarios:
- Getting started ✅
- Install Kafka, PostgreSQL, and RisingWave, and run minimal toy examples
- Querying and processing event streaming data ✅
- Directly query data stored in event streaming systems (e.g., Kafka, Redpanda)
- Continuously ingest and analyze data from event streaming systems
- Bringing analytics closer to operational databases ✅
- Offload event-driven queries (e.g., materialized views and triggers) from operational databases (e.g., MySQL, PostgreSQL)
- Real-time ETL (Extract, Transform, Load) ✅
- Perform ETL continuously and incrementally
- Solution demonstrations ✅
- A collection of demos showcasing how stream processing is used in real-world applications
We use RisingWave as the default stream processing system to run these demos. We also assume that you have Kafka and/or PostgreSQL installed and possess basic knowledge of how to use these systems. These demos have been verified on Ubuntu and Mac.
All you need is a laptop 💻 - no cluster is required.
Any comments are welcome. Happy streaming!
Join out Slack community to engage in discussions with thousands of stream processing enthusiasts!