ByteDance trains a 10-trillion-parameter AI model, aiming for global leadership
ByteDance is currently training a 10-trillion-parameter AI model, signaling a significant push to compete with top-tier U.S. labs like Anthropic.
Decoder
- Parameters: The internal variables or 'weights' in a neural network that the model learns during training; higher counts generally allow for more nuanced internal representation and larger knowledge storage.
Original article
According to the Financial Times, as Chinese companies continue to close the gap with top U.S. AI labs, ByteDance is training an AI model whose scale may approach Anthropic’s advanced Mythos system. According to three informed sources, the Chinese tech giant is currently in the early stages of training a super-large model with an estimated parameter count reaching the 10-trillion level—approximately three times that of China’s currently largest released model, Kimi K3 under Moonshot AI.
One of the sources said ByteDance’s model is currently undergoing pre-training, a phase that typically takes three to six months; if all goes well, it will then proceed to fine-tuning and eventual release. The exact scale of the model will be finalized later.
Anthropic has not disclosed the parameter counts of its models, but industry estimates suggest its most advanced Mythos 5 model has around 8 trillion parameters, while Fable 5 is estimated at about 5 trillion. It should be noted that while parameter count determines a model’s foundational capacity or memory ceiling, the actual performance of a model also depends on other factors, including data quality and training methodology.
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi