AWS Aurora Distributed SQL internals with Marc Brooker - ​

The GeekNarrator memberships can be joined here:    / @thegeeknarrator  
Membership will get you access to member only videos, exclusive notes and monthly 1:1 with me.

Here you can see all the member only videos:    • Members-only videos  

------------------------------------------------------------------------------------------------------------------------------------------------------------------
About this episode:
------------------------------------------------------------------------------------------------------------------------------------------------------------------

In this episode of the Geek Narrator podcast, host Kaivalya Apte interviews Marc Brooker, a distinguished engineer at AWS, about Aurora D-SQL. They discuss Marc's journey at AWS, the evolution of Aurora D-SQL, and the customer-centric approach that led to its development.

Marc explains the choice of PostgreSQL as the foundation for DSQL, the architecture of the database, and the importance of snapshot isolation and concurrency control. The conversation goes into the technical aspects of DSQL, including the write process and how atomicity is maintained, providing listeners with a comprehensive understanding of this innovative database solution. This conversation also goes deep into the intricacies of database design, focusing on fault tolerance, replication strategies, and the role of Firecracker VMs in enhancing scalability. Marc Brooker discusses the architecture of Aurora D-SQL, emphasizing the importance of transaction management, the challenges of active-active deployments, and the trade-offs involved in database design. The discussion also highlights various use cases for Aurora DSQL, including its suitability for micro-services and serverless architectures, while addressing scenarios where it may not be the best fit.

Chapters

00:00 Introduction to Aurora DSQL and Marc Brooker's Journey
03:38 The Evolution of Aurora DSQL at AWS
09:24 Customer-Centric Development and Technological Enablers
12:50 Why PostgreSQL? The Choice Behind DSQL
16:39 High-Level Architecture of DSQL
22:07 Understanding Snapshot Isolation and Concurrency Control
28:45 The Write Process and Atomicity in DSQL
38:50 Designing Fault Tolerance in Databases
47:38 Replication and Transaction Commit Strategies
54:35 Active-Active Deployment and Fault Tolerance
01:00:14 Role of Firecracker VM in Scalability
01:09:27 Use Cases and Trade-offs of Aurora D-SQL

Marc's Blog: https://brooker.co.za/blog/
Marc on Aurora DSQL : https://brooker.co.za/blog/2024/12/03...
AWS's documentation on Aurora DSQL : https://aws.amazon.com/rds/aurora/dsq...

------------------------------------------------------------------------------------------------------------------------------------------------------------------
Like building real stuff?
------------------------------------------------------------------------------------------------------------------------------------------------------------------
Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.

https://app.codecrafters.io/join?via=...

------------------------------------------------------------------------------------------------------------------------------------------------------------------
Link to other playlists. LIKE, SHARE and SUBSCRIBE
------------------------------------------------------------------------------------------------------------------------------------------------------------------
If you like this episode, please hit the like button and share it with your network.
Also please subscribe if you haven't yet.

Database internals series:    • Write-ahead-logging  

Popular playlists:
Realtime streaming systems:    • Realtime Streaming Systems  

Software Engineering:    • Software Engineering  

Distributed systems and databases:    • Distributed Systems and Databases  

Modern databases:    • Modern Databases  

Stay Curios! Keep Learning!

#sql #postgres #databasesystems #aws #awsdevelopers #spanner #google #cockroachdb #yugabytedb #cap #scalability #WAL #DistributedSystems #Cloud #aurora