While the attention is focused on large language AI models, it’s small language AI models that are taking off. Meta seems to be betting big on small language models, as a group of researchers recently published a paper.
To obtain results, large language models like ChatGPT Gemini and Llama can use billions or trillions of parameters. They are too large for mobile devices. So, the Meta scientists noted in their research, there is a growing need for efficient large language models on mobile devices — a need driven by increasing cloud costs and latency concerns.
Scientists explained in their research how they created high quality large language models that had fewer than one billion parameters. This size, they said, is suitable for mobile deployment.
Scientists have achieved results comparable to Meta Llama LLM’s in some areas with their small-language model, contrary to the prevailing belief that data and parameter quantities are the most important factors in determining the quality of a model.
“There’s a prevailing paradigm that ‘bigger is better,’ but this is showing it’s really about how parameters are used,” said Nick DeGiacomo, CEO of BucephalusThe AI-powered supply chain platform is based in New York City.
He told TechNewsWorld that “this opens the door to a more widespread adoption” of AI on devices.
The Crucial Step
Meta’s work is important because it challenges cloud-based AI that often involves data being crunched by far-away data centres, said Darian Shimy. CEO and founder at FutureFundVenture capital firm in San Francisco.
“By bringing AI processing into the device itself, Meta is flipping the script — potentially reducing the carbon footprint associated with data transmission and processing in massive, energy-consuming data centers and making device-based AI a key player in the tech ecosystem,” he told TechNewsWorld.
YashinManraj, CEO at Pvotal TechnologiesEagle Point is the home of an end to end security software developer.
He told TechNewsWorld that it was a first step towards achieving a harmonized SLM-LLM approach, where developers could find the balance between cloud-based data processing and on-device processing. “It is a crucial first step in achieving an SLM-LLM harmonized approach where developers can find the right balance between cloud and on-device data processing,” he told TechNewsWorld.
Meta scientists also took a major step by downsizing the language model. “They propose a model that shrinks by an order-of-magnitude, making it easier to use for wearables such as mobile phones and hearing devices,” said Nishant Neekhra. Skyworks Solutions, a company that manufactures semiconductors in Westlake Village (California).
He told TechNewsWorld, “They are presenting a new set AI applications and new ways that AI can interact in the world.” By shrinking, LLMs are also solving the major growth challenge that plagues LLMs – their inability to be deployed on devices at the edge.
High Impact on Health Care
Medicine is one area where small-scale language models may have a positive impact.
“The research promises unlocking the potential of generative AI applications for mobile devices that are ubiquitous today in health care for remote monitoring and for biometric assessments.” Danielle KelvasTechNewsWorld quoted a physician consultant with IT Medical.
The researchers have opened the door to widespread AI adoption in daily health monitoring, personalized patient care, and everyday health surveillance by showing that SLMs are effective even with less than a million parameters.
Kelvas explained using SLMs ensures that sensitive health information can be processed on a device securely, increasing patient privacy. They can facilitate real-time healthcare monitoring and intervention for patients with chronic illnesses or those who need continuous care.
She added that these models could also reduce technological and financial obstacles to AI deployment in healthcare settings.
Reflecting Industry trends
Meta’s focus is on small AI models that are optimized for mobile devices. This reflects an industry-wide trend to optimize AI for efficiency and access, explained Caridad MuñozProfessor of New Media Technology at City University of New York. She told TechNewsWorld, “This shift addresses not only practical challenges, but is also in line with the growing concerns over the environmental impacts of large-scale AI operation.”
“By championing smaller, more efficient models, Meta is setting a precedent for sustainable and inclusive AI development,” Muñoz added.
The edge computing trend is also a good fit for small language models, as it focuses on bringing AI abilities closer to users. “The large language models from OpenAI, Anthropic, and others are often overkill — ‘when all you have is a hammer, everything looks like a nail,'” DeGiacomo said.
“Specialized and tuned models can be more cost-effective, efficient, and effective for specific tasks,” said he. “Many mobile applications don’t require cutting-edge AI. It’s not necessary to have a supercomputer in order to send a message.
He added that “this approach allows the device focus on routing between what can use the SLM and specialized uses cases, similar to relationship between generalists and specialist doctors.”
Global Connectivity has a profound effect
Shimy said that SLMs have profound implications for global connectivity.
“As the capabilities of on-device AI increase, the requirement for continuous internet connectivity decreases. This could change the tech scene in regions with intermittent or expensive internet,” he said. “This could democratize the access to advanced technology, making cutting edge AI tools available in diverse global markets.”
Manraj pointed out that, while Meta is leading development of SLMs in the AI field, developing countries are closely monitoring the situation. They want to ensure their AI costs remain low. “China, Russia, Iran, and other countries have shown an interest in being able to perform computations on local devices. This seems to be especially important when the latest AI hardware chips cannot be easily accessed or are embargoed.”
He predicted that “this will not be an overnight or dramatic change,” because “complex, multi-language querying will still require cloud based LLMs in order to provide cutting edge value to the end users.” However, this shift towards allowing an on-device ‘last mile’ model can help reduce the burden of the LLMs to handle smaller tasks, reduce feedback loops, and provide local data enrichment.”
“Ultimately,” he said, “the user will clearly be the winner. This would allow them to have a whole new generation of capabilities and an overhaul in front-end apps and how they interact with their world.
He said that while the “usual suspects” are driving innovation, with a potential impact on our everyday lives, “SLMs can also be a Trojan Horse which provides a level of sophistication to the intrusion into our lives. Models capable of harvesting metadata and data at an unprecedented level. We hope, with the appropriate safeguards, that we can channel these efforts into a productive result.”
