DEVOURED
Z1T
Extropic's Z1 chip utilizes probabilistic sub-threshold CMOS technology to achieve massive energy efficiency gains for transformer inference.
What: Extropic released the Z1 chip, which uses probabilistic bits (pbits) to perform sparse neural computations, claiming a 100x efficiency gain over H100 GPUs for specific transformer tasks.
Why it matters: This represents a move toward hardware-software co-design where models are built to match the specific physical constraints of silicon, rather than relying on dense matrix-multiplication accelerators like GPUs.
Takeaway: Developers can access open-source training recipes and weights for the Z1T sparse transformer model on Hugging Face to test sparse neural network performance.
Deep dive
- The Z1 chip functions as a graphical model of pbits with fixed hardware connectivity.
- Employs 'dyadic quantization' (dy4p) to map continuous values to stochastic bit streams.
- Uses Gated Convolutional Attention (GCA) as a transformer alternative suitable for sparse hardware.
- Z1T model achieves ~295 nJ per token compared to ~40.9 µJ on an H100.
Decoder
- Probabilistic sub-threshold CMOS: A type of hardware design that operates transistors in the 'sub-threshold' region (very low voltage) to exploit natural physical noise for stochastic computation.
- Pbit: A binary stochastic variable that flips between states based on neighboring bias, used in Ising-model computing.
Original article
Full article content is not available for inline reading.
Explore all 68 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi