DEVOURED
An operationalization of opaque serial depth
Redwood Research proposes NLS depth as a way to quantify how much unverbalized 'opaque' reasoning an AI model is performing.
What: The authors define Natural-Language-rooted node-Separated (NLS) depth to track how deep a computation graph is before it hits a human-interpretable bottleneck (like CoT tokens), flagging architectures that move reasoning into opaque latent states.
Why it matters: This gives oversight bodies a way to distinguish between 'legible' models using Chain-of-Thought and 'opaque' models that use advanced techniques like COCONUT, which are significantly harder to monitor.
Deep dive
- NLS depth provides a mathematical definition of 'opaque serial depth' for AI systems.
- Traditional Transformers have low NLS depth because their reasoning is routed through human-readable natural language tokens.
- Architectures like 'COCONUT' or full-bandwidth transformers have high NLS depth because they perform multi-step reasoning in opaque latent space.
- NL-rooted nodes are defined by natural language initialization, non-expanding output spaces, and non-differentiable sampling.
- NLS depth is designed to be computable before training begins, based purely on architecture specifications.
- It flags architectures like Mamba or looped transformers as moderately concerning, while latent-state models are categorized as high-risk.
- Tracking NLS depth serves as a necessary complement to, but not a replacement for, existing monitorability research.
Decoder
- NLS Depth: A metric calculating the length of the longest computational path that avoids human-interpretable bottlenecks (nodes that output tokens initialized from natural language priors).
- COCONUT: An AI architecture where models reason by feeding hidden states directly back into themselves, rather than using natural language chain-of-thought tokens.
- Chain-of-thought (CoT): The process where an AI model generates a sequence of intermediate tokens to 'think' through a problem before arriving at an answer.
Original article
Full article content is not available for inline reading.
Explore all 56 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi