How AWS S3 Hit 1PB/s Using Hard Drives… This Is WILD!

Amazon S3 still relies on tens of millions of hard drives, and this choice is the key to how it scales to mind-bending speeds. In this video we break down why HDDs remain cheaper, how AWS squeezes massive throughput out of them, and how clever engineering solves their biggest limitations. You’ll learn how S3 handles hot partitions, parallelizes reads, and reaches performance levels that seem impossible for such old hardware.

🔗 Relevant Links
how AWS S3 serves 1 petabyte per second on top of slow HDDs: https://bigdata.2minutestreaming.com/...

❤️ More about us
Radically better observability stack: https://betterstack.com/
Written tutorials: https://betterstack.com/community/
Example projects: https://github.com/BetterStackHQ

📱 Socials
Twitter:   / betterstackhq  
Instagram:   / betterstackhq  
TikTok:   / betterstack  
LinkedIn:   / betterstack  

📌 Chapters:
00:00 Introduction
00:49 S3 By The Numbers
01:13 But Why Hard Disk Drives??
01:50 How HDD Physics Limit Performance
02:48 Sequential Access and the Log Structure
03:27 Why Reads Are Slow on HDDs
04:14 Massive Parallelism Explained
04:40 Erasure Coding in S3
05:27 The Hot Partition Problem
06:20 Power of Two Random Choices
06:47 Rebalancing Data Across Drives
07:16 Workload Decorrelation
06:10 Final Thoughts