DEVOURED
Brains maximize bits per ATP using 100+ specialist retinal cells and 1Hz firing rates—evolution discovered Shannon entropy millions of years before the math existed.
What: Paras Chopra's notes on 'Principles of Neural Design' argue brains optimize information processing per unit of ATP (energy). Key strategies: send only surprising data (predictive coding), compute locally to minimize wiring, use 1Hz average spike rates in V1 visual cortex, encode natural data distributions via log-scale firing matching Shannon entropy, and deploy 100+ specialist retinal cell types instead of generalist neurons.
Why it matters: This inverts standard software engineering wisdom—biological systems complicate and specialize to optimize efficiency, while engineers simplify and generalize. Evolution discovered mathematically optimal solutions (Shannon entropy, Fourier transforms in the inner ear) through energy constraints alone, suggesting that biological trade-offs could inspire more efficient AI architectures.
Deep dive
- Central thesis: brains maximize information (bits) processed per ATP molecule consumed, driving all neural design through energy efficiency constraints
- Predictive coding: only transmit surprising/unpredicted signals to save energy on redundant information
- Local computation: brain regions exist because local processing minimizes expensive long-distance wiring (language regions, visual cortex)
- Sparse firing: neurons fire at ~1Hz average in V1 visual cortex to minimize ATP cost; faster rates require thicker, more expensive axons
- Two sparsity types: lifetime sparsity (how often individual neurons fire across stimuli) and population sparsity (what % of neurons activate per stimulus, typically few percentage in V1)
- Optimal encoding: spike rates encode equal-probability bins of natural data (often log scale), maximizing Shannon entropy before humans formalized the math
- Chemical/analog computing: dendrites perform complex multi-layer perceptron-like computation using cheap analog signals versus expensive digital spikes
- Irreducible sizing: components shrink to physics limits where noise begins to dominate
- Specialization over generalization: retina alone has 100+ cell types; specialist morphology enables efficient computation and sparse communication (downstream neurons decode meaning from which neuron fired, not just rate)
- Continuous adaptation: firing rates adjust to ambient conditions (bright room, loud concert) to maintain efficiency rather than fixed scales
- Pattern generators: brain sends sparse commands to local circuits near muscles rather than precise micromanagement signals
- Eye/retina preprocessing: performs extensive local computation before sending compressed summaries to brain
Decoder
- ATP (Adenosine Triphosphate): The energy currency of cells; producing it requires metabolic work, making efficiency critical
- Predictive coding: Neural strategy where only prediction errors (surprising signals) are transmitted, not redundant expected signals
- V1: Primary visual cortex, the first cortical area processing visual information from the eyes
- Spike rate: How frequently a neuron fires electrical signals (action potentials), measured in Hz
- Axons/dendrites: Axons transmit signals out from neurons; dendrites receive incoming signals and perform computation
- Shannon entropy: Mathematical measure of information content; maximum entropy encoding is most efficient
- Pattern generators: Local neural circuits near muscles that execute complex movement patterns from simple command signals
Original article
Consuming energy and producing ATP is hard, so brains try to maximize information retrieval and computing using the least amount of energy possible.
Explore all 73 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi