Microsoft unveils Phi-3-mini, its smallest AI model yet: How it compares to bigger models

What is Phi-3-mini?

Phi-3-Mini is believed to be first among the three small models that Microsoft is planning to release. It has reportedly outperformed models of the same size and the next size up across a variety of benchmarks, in areas like language, reasoning, coding, and maths.

Essentially, language models are the backbone of AI applications like ChatGPT, Claude, Gemini, etc. These models are trained on existing data to solve common language problems such as text classification, answering questions, text generation, document summarisation, etc.

The ‘Large’ in LLMs has two meanings — the enormous size of training data; and the parameter count. In the field of Machine Learning, where machines are equipped to learn things themselves without being instructed, parameters are the memories and knowledge that a machine has learned during its model training. They define the skill of the model in solving a specific problem.

What’s new in Microsoft’s Phi-3-mini?

The latest model from Microsoft expands the selection of high-quality language models available to customers, offering more practical choices as they build generative AI applications. Phi-3-mini, a 3.8B language model, is available on AI development platforms such as Microsoft Azure AI Studio, HuggingFace, and Ollama.

The amount of conversation that an AI can read and write at any given time is called the context window, and is measured in something called tokens. According to Microsoft, Phi-3-mini is available in two variants, one with 4K context-length, and another with 128K tokens.

With longer context windows, models are more capable of taking in and reasoning over large text content such as documents, web pages, code, and more.

Phi-3-mini is the first model in its class to support a context window of up to 128K tokens, with little impact on quality. The model is instruction-tuned, which means that it is trained to follow the different types of instructions given by users. This also means that the model is ‘ready to use out-of-the-box’.

Microsoft says that in the coming weeks, new models will be added to the Phi-3 family to offer customers more flexibility. Phi-3-small (7B) and Phi-3-Medium will be available in the Azure AI model catalogue and other model libraries shortly.

How is Phi-3-mini different from LLMs?

Phi-3-mini is an SLM. Simply, SLMs are more streamlined versions of large language models. When compared to LLMs, smaller AI models are also cost-effective to develop and operate, and they perform better on smaller devices like laptops and smartphones.

According to Microsoft, SLMs are great for “resource-constrained environments including on-device and offline inference scenarios.” The company claims such models are good for scenarios where fast response times are critical, say for chabots or virtual assistants. Moreover, they are ideal for cost-constrained use cases, particularly with simpler tasks.

While LLMs are trained on massive general data, SLMs stand out with their specialisation. Through fine-tuning, SLMs can be customised for specific tasks and achieve accuracy and efficiency in doing them. Most SLMs undergo targeted training, demanding considerably less computing power and energy compared to LLMs.

SLMs also differ when it comes to inference speed and latency. Their compact size allows for quicker processing. Their cost makes them appealing to smaller organisations and research groups.

How good are the Phi-3 models?

Phi-2 was introduced in December 2023 and reportedly equaled models like Meta’s Llama 2. Microsoft claims that the Phi-3-mini is better than its predecessors and can respond like a model that is 10 times bigger than it.

Based on the performance results shared by Microsoft, Phi-3 models significantly outperformed several models of the same size or even larger ones, including Gemma 7B and Mistral 7B, in key areas.

Microsoft claims that Phi-3-mini demonstrates strong reasoning and logic capabilities. In its blog, Microsoft also said, “ITC, a leading business conglomerate based in India, is leveraging Phi-3 as part of their continued collaboration with Microsoft on the copilot for Krishi Mitra, a farmer-facing app that reaches over a million farmers.”