DEVOURED
Parting the Clouds: The Rise of Disaggregated Systems (47 minute video)
Cloud databases are increasingly disaggregating compute from storage to enable elastic scaling, cost efficiency, and simplified fault isolation.
What: Murat Demirbas analyzes the transition from monolithic databases to architectures like Amazon Aurora, Alibaba PolarDB, and Google AlloyDB, which separate stateless compute nodes from shared, persistent log and page stores.
Why it matters: The industry is moving toward treating the data center as a giant computer where the network acts as a backplane, allowing compute and storage to scale independently based on demand.
Deep dive
- Monolithic Origins: Traditional databases coupled compute and storage, creating scaling bottlenecks.
- Disaggregation Drivers: Compute fluctuates while storage grows steadily; decoupling allows scaling compute to zero during idle periods.
- Network as Backplane: High-speed networks, RDMA, and CXL technologies make separation viable by reducing I/O latency.
- Log-as-Database: Aurora popularized using the redo log as the primary data exchange, reducing network traffic.
- Role Specialization: Paxos roles (Proposers/Acceptors/Learners) mirror modern distributed database functions, enabling granular resource pooling.
- Tradeoffs: Disaggregation shifts bottlenecks from CPU/disk to the network, requiring prefetching, buffering, and query pushdown to mitigate latency.
- Future Trends: Memory disaggregation via CXL and self-assembling database systems that auto-configure based on agentic workload bursts.
Decoder
- Disaggregation: The architectural practice of separating compute resources from storage resources to allow independent scaling.
- Paxos/Raft: Consensus algorithms used to ensure data consistency and fault tolerance across distributed nodes.
- Redo Log / WAL (Write-Ahead Log): A file that records all changes made to the database before they are applied to the main data files, ensuring crash recovery.
- RDMA (Remote Direct Memory Access): A networking feature that allows one computer to access memory on another without involving either system's operating system or CPU.
- CXL (Compute Express Link): An open-standard interconnect that allows CPUs to access memory and devices (like accelerators or storage) with very low latency.
- LSM Trees (Log-Structured Merge Trees): A data structure optimized for write-heavy workloads, common in modern distributed databases.
Original article
Full article content is not available for inline reading.
Explore all 67 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi