ksqlDB HOWTO: Integration with other systems

Using ksqlDB you can pull data in from other systems (e.g. databases, JMS message queues, etc etc), and push data down to other systems (NoSQL stores, Elasticsearch, databases, Neo4j, etc etc).

This is done using Kafka Connect, which can be run embedded within ksqlDB or as a separate cluster of workers. ksqlDB can be used to create and control the connectors. For example:

CREATE SINK CONNECTOR SINK_ELASTIC_ORDERS_01 WITH (
'connector.class' = 'io.confluent.connect.elasticsearch.ElasticsearchSinkConnector',
'topics' = 'ORDERS_ENRICHED',
'connection.url' = 'http://elasticsearch:9200',
'type.name' = '_doc'
);

-----

💾 Run ksqlDB yourself: https://ksqldb.io?utm_source=youtube&...
☁️ Use ksqlDB as a managed service: https://www.confluent.io/confluent-cl...

👾 Demo code: https://github.com/confluentinc/demo-...

🤔 Questions? Join the Confluent Community at https://confluent.io/community/ask-th...

-----

⏱ Time codes

00:00:00 Introduction
00:00:30 Listing streams in ksqlDB
00:00:49 Printing a topic's contents in ksqlDB
00:01:03 Creating a sink connector in ksqlDB
00:01:56 Checking connector status in ksqlDB
00:02:13 Embedded vs external connect worker in ksqlDB
00:03:12 Inspecting the data in Elasticsearch that's been streamed from ksqlDB
00:04:33 Recap