Quix Streams—a Python-Kafka Library for Data-Intensive Workloads (Tomas Neubauer) | RTA Summit 2023

All Sessions: https://rtasummit.com | This talk will introduce Quix Streams, an open-source Python library for data-intensive workloads on Kafka.

We will discuss the unique problems that this library is designed to solve, and how it was shaped by the challenges building a Kafka-based solution for Formula 1 cars at McLaren—a solution that needed to process a colossal firehose of sensor data coming in at thousands of samples per second. We’ll also explain why we decided to combine a Kafka API approach with a stream processing library and provide developers with a familiar Pandas DataFrame-like interface.

You’ll see how the library can simplify tasks such as:
– Subscribing to topics, deserializing incoming messages into table rows
– Running calculations on a rolling window of messages
– Using memory states to apply different functions such as aggregation or filtering
– Automatically outputting the results of calculations into downstream topics
– Managing state without the hassle of checkpointing and queues

LEARN MORE
► Learn how StarTree Cloud can power your real-time analytics: https://stree.ai/cloud-rtas23
► Learn about Apache Pinot: https://dev.startree.ai

CONNECT
Subscribe on YouTube: https://youtube.com/c/startreedata?su...
Community Slack: https://stree.ai/slack
Twitter:   / startreedata  
Linkedin:   / startreedata  
GitHub: https://github.com/startreedata
Site: https://startree.ai

ABOUT STARTREE
When you hear “decision maker,” it's natural to think, “C-suite,” or “executive.” But these days, we’re all decision-makers. So, yes, the CEO is a crucial decider, but the franchise owner, the student, or the big box shopper all have important choices to make, too. The difference is, those traditional decision-makers have access to relevant data to guide their thinking. All the others? Flying blind.

Our vision is to change that. We believe every decision-maker should have access to fast, fresh, actionable insights. We’re motivated to unlock what’s possible when you put the right information in the right hands at the right time. Like the restaurant owner who’s able to call in an extra cook because she can see the surge in orders coming. Or the holiday shopper who has plenty of time to pick a different gift for his loved one because he sees his original order is delayed at the moment the supplier registers a procurement delay. Or the use-case we can’t even envision yet, because it is sitting in your data, waiting to be set free.

Exposing timely information to real decision makers in intuitive apps that not only inform, but allow you to act on the information being shared. That’s how we’re unleashing the power of user-facing analytics.

#apachepinot #realtimeanalytics #startree