Deep Dive into Delta Tables on Fabric

Deep Dive into Delta Tables on Fabric

Speaker: Chen Hirsh

Session Overview:

Delta tables are the foundations of the Lakehouse, and understanding how they work internally, and how to optimize them for best performance, is crucial to the Fabric data engineer or data developer.
In this session, I will explain the Apache Parquet format, that Delta tables are based on, and how column store and compression make it the default choice in most Datalakes.


We would also talk about the Parquet format disadvantages, and how the Delta format overcame those with the introduction of the transaction log. I would demonstrate advanced features in Delta tables based on the transaction log like easily cloning a table without copying any data and using the log to “time travel” to see older versions of the table, query it, and even restore the table to an earlier state.
Like most data technologies, delta tables need to be maintained to improve performance. We will see how to optimize for the best file sizes, and how to use partitions, Zorder and Vorder to help the query engine read only the necessary files.


At the end of the session, participants should have a clear understanding of Delta tables internals and uses, and how to maintain and optimize them for best performance.


💻 Level up your data skills by subscribing to our newsletter (it's free!) https://levelupyourdata.com/newsletter/

#microsoftfabric #dataanalytics #levelupyourdata