DEVOURED
Thinking Machines Releases First Open Model
Thinking Machines has released Inkling, a 975B-parameter mixture-of-experts multimodal model that supports fine-tuning and controllable thinking effort.
What: Inkling is a 975B-parameter MoE model with 41B active parameters, a 1M token context window, and native multimodal support. It was released alongside Inkling-Small, a 12B active parameter version, and is available for fine-tuning via the Tinker platform and various inference backends like vLLM and SGLang.
Why it matters: The model's design focuses on 'controllable thinking effort,' signaling a shift where model efficiency and inference latency are tuned via system prompts rather than static architectural constraints.
Deep dive
- Architecture: Mixture-of-Experts (MoE) with 256 routed experts and 2 shared experts.
- Performance: Reaches performance comparable to larger models like Nemotron 3 Ultra using significantly fewer tokens.
- Multimodality: Encoder-free architecture using dMel spectrograms for audio and 40x40 image patches.
- Epistemics: Trained specifically for calibrated confidence, hedging, and resistance to censorship.
- RL Pipeline: Scaled to 30M+ rollouts, showing emergent compression of chain-of-thought tokens.
- Ecosystem: Supports integration with SGLang, vLLM, TokenSpeed, and llama.cpp.
Decoder
- MoE (Mixture-of-Experts): An architecture where only a subset of parameters (experts) is active for each token, allowing for high total capacity with lower computational cost.
- Controllable thinking effort: The ability to adjust model inference compute dynamically at test time, balancing latency versus output depth.
- HLE (Humanity's Last Exam): A benchmark designed to evaluate reasoning capabilities beyond standard LLM datasets.
Original article
Full article content is not available for inline reading.
Explore all 56 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi