DEVOURED
Subquadratic launched a 12-million-token context window model that beats GPT-5.5 on retrieval benchmarks, but Magic.dev raised $500M on similar promises in 2024 and still hasn't shipped publicly.
What: Miami-based Subquadratic launched a model with a 12-million-token context window using Subquadratic Selective Attention (SSA) architecture, scoring 83% on MRCR v2 retrieval (vs GPT-5.5's 74%) and 82.4% on SWE-bench. The company raised $29M at a $500M valuation from investors including Tinder co-founder Justin Mateen and plans a 50-million-token model by Q4 2026.
Why it matters: If the benchmarks hold in production, this would be the first commercially viable solution to the quadratic attention trap that has kept every frontier model capped below 2M tokens, potentially eliminating RAG and agentic decomposition as necessary workarounds—but Magic.dev's invisible 100M-token model launched with $500M in funding suggests the gap between lab results and deployment remains a chasm.
Deep dive
- Subquadratic claims 12M-token window with 83% on MRCR v2 (vs GPT-5.5's 74%), 82.4% on SWE-bench (vs Anthropic's 81.4%), and 92.1% needle-in-a-haystack at 12M tokens where no frontier model operates
- Core problem: standard transformer attention scales O(n²)—doubling context quadruples compute because every token compares with every other token, capping frontier models around 1-2M tokens
- Subquadratic Selective Attention (SSA) claims linear scaling with content-dependent token selection that doesn't itself go quadratic, running 52× faster than dense attention at 1M tokens
- Previous attempts all traded capabilities: Longformer (fixed windows fail on distant info), Mamba (lossy state compression), hybrids (still have O(n²) layers), DeepSeek (selection step is quadratic)
- Caveats: each model run once due to cost, company's model "way smaller than the big labs" per CTO, margin on SWE-bench is "harness as much as model"
- Shipping: API and SubQ Code CLI agent in beta with 12M-token window, 50M-token target Q4 2026, running on neoclouds (hyperscalers too expensive), no open-source weights
- Warning sign: Magic.dev announced 100M-token model August 2024 with 1000× efficiency claim, raised $500M+, but no public evidence of deployment as of early 2026—benchmark-to-production gap remains large
- $29M raised at $500M valuation from Javier Villamizar (ex-SoftBank Vision Fund) and Justin Mateen (Tinder co-founder), 11 Ph.D. researchers, previously called Aldea (pivoted from speech models)
Decoder
- Quadratic attention scaling: In standard transformers, every token must compare with every other token (dense attention), causing O(n²) scaling—doubling the context length quadruples the compute cost. This caps frontier models like GPT-5.5 and Claude around 1-2M tokens. Sparse attention selects which token pairs matter without the selection mechanism itself going quadratic.
Original article
Subquadratic has launched a new AI model with a 12-million-token context window. It outperforms GPT-5.5 on retrieval benchmarks. Attention cost scales quadratically with context length, so doubling the input quadruples the work. Subquadratic claims to have solved the problem. It plans to offer a model with a 50-million-token context window soon.
Explore all 66 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi