Unity Catalog, Delta Sharing and Data Mesh on Databricks Lakehouse

In this technical deep dive, we will detail how customers implemented data mesh on Databricks and how standardizing on delta format enabled delta-to-delta share to non-Databricks consumers.

Current state of the IT landscape
1. Data silos (problems with organizations not having connected data in the ecosystem)
2. A look back on why we moved away from data warehouses and choose cloud in the first place
3. What caused the data chaos in the cloud (instrumentation and too much stitching together) ~ periodic table list of services of the cloud
How to strike the balance between autonomy and centralization
Why Databricks Unity Catalog puts you in the right path to implementing data mesh strategy
What are the process and features that enable and end-to-end Implementation of a data strategy
How customers were able to successfully implement the data mesh on out of the box Unity Catalog and delta sharing without overwhelming their IT tool stack
Use cases
1. Delta-to-delta data sharing
2. Delta-to-others data sharing
How do you navigate when data today is available across regions, across clouds, on-prem and external systems
1. Change data feed to share only “data that has changed”
Data stewardship
1. Why ABAC is important
2. How file based access policies and governance play an important role
Future state and its pitfalls
1. Egress costs
2. Data compliances

Talk by: Surya Turaga and Thomas Roach

Connect with us: Website: https://databricks.com
Twitter:   / databricks  
LinkedIn:   / databricks  
Instagram:   / databricksinc  
Facebook:   / databricksinc