ksqlDB HOWTO: Filtering

Using ksqlDB you can filter streams of data in Apache Kafka and write new topics in Kafka populated by a subset of another.

ksqlDB uses SQL to describe the stream processing that you want to do. With the WHERE clause you can define predicates to filter the data as you require.

For example:

CREATE STREAM ORDERS_NY AS
SELECT *
FROM ORDERS
WHERE ADDRESS_STATE='New York';

-----

💾 Run ksqlDB yourself: https://ksqldb.io?utm_source=youtube&...
☁️ Use ksqlDB as a managed service: https://www.confluent.io/confluent-cl...

👾 Demo code: https://github.com/confluentinc/demo-...

🤔 Questions? Join the Confluent Community at https://confluent.io/community/ask-th...

-----

⏱ Time codes

00:00:00 Intro
00:00:27 Listing topics on the Kafka cluster in ksqlDB
00:00:53 Consuming from a Kafka topic with ksqlDB using PRINT
00:01:44 Creating a stream in ksqlDB on an existing topic using Avro schema
00:02:20 Show ksqlDB stream schema with DESCRIBE
00:02:33 Nested objects in ksqlDB
00:02:47 Querying a stream in ksqlDB
00:03:16 Accessing nested fields in ksqlDB
00:03:29 Filtering in ksqlDB
00:04:09 Using predicates in ksqlDB
00:04:35 Persisted a continuous query output into a new stream
00:04:53 CREATE STREAM … AS SELECT
00:05:20 Changing auto.offset.reset in ksqlDB
00:06:13 Creating a filtered stream in ksqlDB
00:06:20 Changing the properties of the target topic (name, partition count, etc)
00:07:06 Viewing the filtered topic properties
00:07:27 Viewing the schema of the new stream
00:07:41 Viewing the original and filtered topics together
00:08:39 Recap & Summary