On Tuesday, Microsoft announced a new, freely available lightweight AI language model named Phi-3-mini, which is simpler and less expensive to operate than traditional large language models (LLMs) like OpenAI’s GPT-4 Turbo.
The AI field typically measures AI language model size by parameter count.
Parameters are numerical values in a neural network that determine how the language model processes and generates text.
Some of the largest language models today, like Google’s PaLM 2, have hundreds of billions of parameters.
It’s a follow-up of two previous small language models from Microsoft: Phi-2, released in December, and Phi-1, released in June 2023.
Much has been written about the potential environmental impact of AI models and datacenters themselves, including on Ars.
With new techniques and research, it’s possible that machine learning experts may continue to increase the capability of smaller AI models, replacing the need for larger ones—at least for everyday tasks.
AI models like Phi-3 may be a step toward that future if the benchmark results hold up to scrutiny.
Microsoft unveiled Phi-3-mini, a new lightweight AI language model that is freely available. Compared to more conventional large language models (LLMs) like OpenAI’s GPT-4 Turbo, Phi-3-mini is easier to use and costs less to run. Because of its compact size, it can operate locally on a smartphone and provide an AI model with features comparable to those of ChatGPT’s free version, all without requiring an Internet connection.
Language models for artificial intelligence are usually measured by the number of parameters. The language model’s processing and text generation methods are determined by parameters, which are numerical values in a neural network. They are basically a quantified form of the knowledge that the model has learned through training on large datasets. More parameters typically enable the model to generate more complex and nuanced language, but they also increase the amount of computing power needed for training and operation.
There are billions of parameters in some of the biggest language models available today, such as Google’s PaLM 2. Although it is spread across eight 220-billion parameter models in a mixture-of-experts configuration, OpenAI’s GPT-4 is rumored to have over a trillion parameters. For both models to function correctly, heavy-duty data center GPUs (as well as related systems) are needed.
By contrast, Microsoft went small with Phi-3-mini, which was trained on 3 trillion tokens and has only 3 point 8 billion parameters. Running it on consumer GPU or AI-acceleration hardware, which is available in laptops and smartphones, is therefore perfect. It is an update to Microsoft’s two earlier small language models, Phi-1, which was released in June 2023, and Phi-2, which was released in December.
Microsoft also released “phi-3-mini-128K,” a 128K-token version of Phi-3-mini, in addition to its 4,000-token context window. Microsoft claims that its 7-billion and 14-billion parameter Phi-3 versions are “significantly more capable” than the Phi-3-mini. These versions will be released later.
According to a paper titled “Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone,” Microsoft claims that Phi-3’s overall performance “rivals that of models such as Mixtral 8x7B and GPT-3.5”. “The GPT-3.5 chip powers ChatGPT’s free version, while Mixtral 8x7B from the French AI company Mistral uses a mixture-of-experts model.
In an interview with Ars, AI researcher Simon Willison stated, “[Phi-3] looks like it’s going to be a shockingly good small model if their benchmarks are reflective of what it can actually do.”. Willison texted Ars shortly after supplying that quote, saying, “I got it working, and it’s GOOD.” He had downloaded Phi-3 locally to his Macbook laptop.
Willison claims that “the majority of models that run on a local device still need hefty hardware.”. “Phi-3-mini can generate tokens at a respectable pace even on a standard CPU, and it runs smoothly with less than 8GB of RAM. It runs on a $55 Raspberry Pi and is licensed by MIT. Based on my initial observations, the quality of the results is on par with models four times larger. “.”.
Microsoft’s researchers used well selected, high-quality training data that they first extracted from textbooks to figure out how the company fit a capability potentially comparable to GPT-3.5, which has at least 175 billion parameters, into such a small model. “Our training dataset, which is a resized version of the phi-2 dataset and is made up of both synthetic and heavily filtered web data, is the innovation,” Microsoft claims. Additionally, the model is further aligned with respect to chat format, safety, and robustness. ****.
The possible effects of AI models and datacenters on the environment have been extensively covered in the media, including on Ars. It’s feasible that machine learning specialists will keep improving the power of smaller AI models through new methods and study, negating the need for larger ones—at least for routine tasks. That could theoretically significantly reduce AI’s environmental impact by requiring far less energy overall and saving money over time. If the benchmark results hold up to scrutiny, AI models such as Phi-3 might be a step toward that future.
