Moonshot Openly Defies The Trump Administration
Chinese AI lab Moonshot is reportedly bypassing US export controls to secure Nvidia Blackwell GPUs for its upcoming 2.8 trillion parameter Kimi K4 model.
Deep dive
- Moonshot claims architectural efficiency gains in Kimi K3, such as reduced KV cache and expert distribution, to offset compute constraints.
- The US administration accuses Moonshot of distilling models from Anthropic's Fable using illicitly acquired hardware.
- Moonshot is actively seeking Nvidia Blackwell GPUs for Kimi K4 despite ongoing threats of a total ban on Chinese open-weight models.
- Industry leaders including Nvidia and Microsoft have signed a letter urging the US to protect the open-weight AI ecosystem.
Decoder
- KV Cache: Key-Value cache used in Transformers to store the results of previous attention layers, reducing the computation needed for each new token generated.
- Open-weight: AI models where the learned parameters (weights) are published, allowing others to run the model locally, as opposed to closed models behind an API.
- Distillation: A process where a smaller model is trained to mimic the output behavior of a larger, more complex teacher model.
Original article
Moonshot Openly Defies The Trump Administration By Seeking Access To Additional NVIDIA GPUs For Training The Next-Gen Kimi K4 Model
Either Moonshot is oblivious to the dangers of the path it is skirting, or egregiously indifferent. After all, why else would it choose to double down on the same furtive strategy that worked for the Kimi K3 model, and likely will for the Kimi K4 as well, but at huge geopolitical costs, risking the utter wrath of the Trump administration in the process?
Moonshot is trying to furtively leverage an even bigger stash of compute resources from NVIDIA to train its upcoming Kimi K4 model
As we explained in a recent post, Moonshot has just unveiled its brand-new, open-source Kimi K3 model, which spans 2.8 trillion parameters and is designed specifically for frontier-scale intelligence.
The model is multi-modal, has a context window of around 1 million tokens, and offers competitive and faster performance than many of its compeers.
What's more, Kimi K3 is quite efficient to run on data center-scale hardware, courtesy of its dramatically reduced KV cache and built-in optimizations for per-token memory use - achieved by distributing its 896 experts across a large number of GPUs.
In fact, Microsoft calculates that it could save as much as $600 million in inference costs by moving Copilot away from OpenAI and Anthropic models and towards Moonshot's Kimi K3.
Of course, Moonshot jumped right into the ongoing multi-dimensional struggle between China and the US recently, when Anthropic and some members of the Trump administration accused Moonshot of distilling its Kimi K3 model from Anthropic's Fable.
The US has long suspected that Chinese engineers often take their model-laden hard drives to US-friendly countries to imbue their models with frontier-level capabilities by using reinforcement learning techniques on cutting-edge NVIDIA GPUs.
Apparently, as per the allegations leveled by a Trump administration official recently, Moonshot not only furtively owns some NVIDIA GB300 servers, but was also able to access additional GB300s via Thailand.
Do note that Fable was unbanned towards the end of June, while Moonshot released its Kimi K3 model around the middle of July. There just wasn't enough time for Moonshot to have distilled a model as massive as Kimi K3, not to mention its unique architectural feats that aren't a part of Fable. Moonshot likely fine-tuned some responses of its model using Fable, though.
This brings us to the core of the matter. The Information has just reported that Moonshot, after the universal success of its Kimi K3 model, is now seeking access to an even larger stash of NVIDIA Blackwell GPUs for the next-gen Kimi K4 model, and appears willing to defy American export controls in the process.
This is not to say that the company isn't using domestic AI chips. After all, Moonshot engineers managed to link multiple eight-chip servers across separate China-based data centers to train the Kimi K3. But, given the fact that Kimi K4 will likely eclipse the 2.8 trillion parameters of its predecessor, Moonshot is desperate for additional compute and appears willing to risk the wrath of the Trump administration in the process.
Meanwhile, the Trump administration is currently mulling whether to ban open-weight models from China altogether. For the benefit of those who might not be aware, weights tell a model how much importance it should accord to any given concept or a string of words. These weights start out as random numbers, and then are changed via iterative updates, where an error metric measures the gap between the prediction and the actual target value after each forward pass. Open-weight models, such as Kimi K3, disclose their weights, de-mystifying their overall architecture, while closed-weight ones, such as Anthropic's Claude, do not.
Here though, the who's who of the AI world, including NVIDIA, Microsoft, Meta, Dell, Perplexity, Palantir, Mistral AI, and more, have now issued a letter, calling on the Trump administration to ensure that the open-weight AI model ecosystem continues to thrive.
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi