The Future of Data Warehousing with Databricks

Franco Patano explains why SQL and Data warehousing are a solved problem and AI remains a tough challenge.

Chapters
• 0:00 - Introduction to Franco Patano, Databricks' Data Warehousing Expert
• 0:40 - Databricks' Evolution: From Spark to Lakehouse and Data Intelligence
• 1:40 - The Genesis of the Lakehouse (2018-2020)
• 3:00 - Databricks SQL's Performance Journey (2021-2022)
• 4:00 - Performance & Scalability Advancements
• 5:25 - Key Engineering Innovations for Performance
• 6:30 - Introducing Streaming Tables & Materialized Views for Real-Time Warehousing:
• 7:45 - Streaming Tables: Automatic State Management
• 8:40 - Materialized Views: Efficient Incremental Computation (Project Enzyme)
• 9:50 - Databricks SQL's Market Position vs. Competitors
• 11:50 - Demo: Ingesting Data with Unity Catalog & Streaming Tables
• 13:10 - Demo: AI-Generated Metadata & End-to-End Lineage
• 14:30 - Demo: SQL Syntax for Streaming Tables and Materialized Views
• 15:00 - Demo: AI-Powered Metadata & Discovery in Unity Catalog
• 15:50 - Demo: Text-to-Visualization in Lakeview Dashboards
• 17:35 - Recap of the Data Flow
• 18:25 - Demo: Lineage for Root Cause Analysis & Governance
• 20:15 - Conclusion & Call to Action

Youssef Mrini & Quentin Ambard