GPU Pipeline Optimization Explained | Async UDFs, CUDA Streams & Pinned Memory

🖥️ Whiteboard Deep Dive into GPU Pipeline Optimization

In this deep dive, Srinu Lade   / srinivas-lade   (Software Engineer working on Daft’s execution engine) breaks down how to optimize GPU pipelines for ML and multimodal data processing. Using architectural diagrams, he explains why sequential CPU→GPU execution creates bottlenecks and how techniques like async UDFs, CUDA streams, and pinned memory unlock parallelism.

What you’ll learn:
How GPU workloads flow: host↔device transfers, VRAM, kernel execution
Why Python UDFs are a bottleneck — and how async execution improves throughput
Using CUDA streams to overlap transfers and compute for better utilization
How GPU internals (H2D/D2H engines + compute units) enable pipeline parallelism
Reducing OS overhead with pinned memory reuse in PyTorch workflows
How Daft abstracts these optimizations into a high-level API for data/ML engineers

Our aim is to abstract away these low-level complexities and provide a high-level API in Daft that delivers optimized GPU execution out-of-the-box for ML workloads.

—

Daft. Simple and reliable data processing for any modality and scale.

Explore → https://daft.ai/
Build → https://docs.daft.ai/
Connect → https://www.daft.ai/slack
Contribute → https://github.com/Eventual-Inc/Daft
Learn → https://daft.ai/blog

pip install daft