FX.co ★ Meta Develops Efficient Language Models For Smartphones

Meta (META) has introduced a new, compact artificial intelligence model called MobileLLM, designed specifically for smartphones and devices with limited computational capabilities.

Developed collaboratively by Meta Reality Labs, Meta AI Research (FAIR), and PyTorch, MobileLLM features fewer than one billion parameters.

Yann LeCun, Meta's Chief AI Scientist, highlighted crucial aspects of the research in a post on X/Twitter, stating, "Our findings indicate that, for smaller models, prioritizing depth over width enhances model performance. Furthermore, by leveraging advanced weight-sharing techniques, including embedding sharing, grouped query attention, and block-wise weight sharing, we achieve significant enhancements in weight utilization within storage-constrained scenarios."

These design innovations have enabled MobileLLM to outperform previous models of similar sizes by 2.7 to 4.3 percent, as evidenced by benchmark tests.

LeCun's post also noted that MobileLLM demonstrates "substantial advancements in zero-shot commonsense reasoning, question answering, and reading comprehension tasks compared to state-of-the-art (SoTA) methods."

Additionally, the researchers showcased the effectiveness of MobileLLM in chat and API call applications, further emphasizing its proficiency in these areas.