DEVOURED
Azure Maia 300 Architecture Deep Dive
Microsoft's new Maia 300 AI accelerator features 288 GB of HBM4 memory, delivering 18 TB/s of bandwidth and 7x greater token throughput.
What: The Maia 300 utilizes TSMC's N2P node, scales up to 12,288 chips per superpod, and operates in racks consuming 167 kW of power.
Why it matters: This highlights the industry's shift toward custom silicon designed specifically for massive model inference, trading general-purpose flexibility for extreme memory bandwidth and throughput.
Decoder
- HBM4: High Bandwidth Memory, the fourth generation of vertically stacked DRAM that provides extreme data rates for AI workloads.
- TSMC N2P: A 2-nanometer process node, representing the most advanced mass-production semiconductor fabrication technology currently entering the market.
Original article
Microsoft's third-generation AI accelerator uses TSMC N2P with 288 GB HBM4 at 18 TB/s. Racks hold 96 accelerators at 167 kW, scaling to 12,288-chip superpods, with up to 7x token throughput.
Explore all 62 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi