A group of researchers at the University of California, Berkeley, say they’ve recreated the core technology found in China’s revolutionary DeepSeek AI for just $30.
This extremely cheap DeepSeek recreation is yet another indicator that while models from larger companies have been impressive, there may be much more affordable ways to build them.
What’s particularly impressive is that the entire recreation cost them just $30, Pan claims in a post on Nitter.
The researchers tested multiple model sizes, starting with a 500-million-parameter model that could only guess and stop, regardless of accuracy.
When scaled to 1.5 billion parameters, the DeepSeek recreation began incorporating revision techniques.
For just $30, a team of researchers from the University of California, Berkeley claims to have replicated the essential technology of China’s ground-breaking DeepSeek AI. Despite the impressive models from larger companies, this incredibly low-cost DeepSeek recreation is another indication that there might be much more cost-effective ways to construct them.
Headlined by Ph. A. Candidate Jiayi Pan, the group used a tiny language model with only 3 billion parameters to mimic DeepSeek R1-Zero’s reinforcement learning capabilities. Even though it was small, the AI showed self-verification and search capabilities, which are essential for iteratively improving its own answers.
The Berkeley team used the Countdown game, a numerical puzzle based on the British game show where players must use arithmetic to reach a target number, to test their DeepSeek recreation. Initially generating random guesses, the model evolved self-correction and iterative problem-solving techniques through reinforcement learning.
It eventually learned to make changes to its responses until it found the right one. Additionally, they experimented with multiplication, where the AI used the distributive property to break down equations in a manner similar to how humans might mentally solve complex multiplication problems. This illustrated the model’s capacity to modify its approach in response to the issue.
Pan writes in a post on Nitter that the whole recreation only cost them $30, which is especially amazing. This is a staggeringly small portion of the money that top AI companies spend on extensive training. Beginning with a 500 million parameter model that could only guess and stop, regardless of accuracy, the researchers experimented with various model sizes.
Revision techniques were introduced into the DeepSeek recreation when it was scaled to 1 point 5 billion parameters. According to Pan and the other researchers, there was a notable improvement in the accuracy and number of steps needed to solve problems with models with 3–7 billion parameters.
To put things in perspective, DeepSeek charges a significantly lower price of 0.055 per million tokens, whereas OpenAI charges $15 per million tokens through its API at the time of writing. The Berkeley team’s research indicates that highly effective AI models can be created for a fraction of the price that top AI firms are now spending. You should probably stay away from DeepSeek for a number of reasons, even though it’s inexpensive.
One explanation is the skepticism of certain experts regarding DeepSeek’s purported affordability. Concerns have been raised by AI researcher Nathan Lambert regarding whether DeepSeek’s stated $5 million training cost for its 671-billion-parameter model is an accurate representation of the whole picture. Furthermore, the AI transmits a significant amount of data to China, which is undoubtedly concerning and is already causing DeepSeek to be banned across the United States. A.
According to Lambert, DeepSeek AI’s yearly operating costs could range from $500 million to more than $1 billion, taking into account costs for research staff, energy use, and infrastructure. Additionally, according to OpenAI, there is proof that DeepSeek was trained with ChatGPT, which may help explain some of the lower expenses.
The Berkeley team’s efforts nevertheless demonstrate that advanced reinforcement learning is feasible without the massive funding currently allotted by industry titans like OpenAI, Google, and Microsoft. This study demonstrates what might turn out to be a potentially disruptive change in the field, given that some AI labs spend up to $10 billion a year on model training.