A new study by researchers at the Massachusetts Institute of Technology suggests that the rapid improvement of large language models is driven primarily by access to massive computing power rather than secret proprietary techniques developed by individual AI companies.
The paper, titled Is there “Secret Sauce” in Large Language Model Development?, analyzes 809 large language models released between October 2022 and March 2025 to understand what factors are responsible for improvements in AI capabilities.
The researchers examined benchmark performance and training data for the models and attempted to separate the sources of progress into four components: the amount of training compute used, shared algorithmic progress across the industry, developer-specific techniques, and model-specific design choices. [Read more…] about MIT study suggests computing power – not ‘secret sauce’ – drives most AI model breakthroughs