Meta is about to launch its biggest Llama model yet — here’s why it’s a big deal

Meta AI logo on a phone
(Image credit: Shutterstock)

In April 2024, Meta released Llama 3, the latest version of its AI-powered large language models based on a dataset that’s at least 7 times larger than Llama 2. 

Initially available in 8B and 70B parameter sizes, Llama 3 already outperformed Llama 2, Google's open-source Gemma, and Anthrophic’s Claude Sonnet at launch. Sonnet has since had an upgrade making it one of the most powerful AI models

But now, leaks suggest that the much-awaited release of the most powerful Llama 3 models which have been trained on more than 400 billion parameters, might also be close at hand. This is just one of a number of new models Meta is working on, utilizing its hundreds of thousands of Nvidia H100 GPUs.

Efficient yet powerful

In early testing, the instruction-tuned Llama 3 400B scored 86.1 on the MMLU benchmark, which already makes it on par with GPT-4's performance with less than half the parameters.

There’s a lot of technical information to unpack here, so let’s talk about why this really matters.

Simply put, large language models with more parameters always tend to perform better on benchmarks and real-world tasks. But the fact that Llama 3 400B can nearly match GPT-4's MMLU score with under 50% of the parameters, suggests that Meta has made enough advancements in model architecture and training to give OpenAI a serious run for its money.

By achieving equal performance with fewer parameters, Llama 3 400B is likely to be much more efficient than OpenAI’s ChatGPT 4 in terms of computational resources, energy consumption, and cost.

Open-source advantage

Meta AI

(Image credit: Future)

Another important reason why people are so excited about Llama 3, is that it has been released under an open license for research and commercial use. Although it isn't yet clear if 400B will be released under that same open license.

If it is released as an open model then these state-of-the-art language capabilities would now be available to researchers and developers for free through multiple cloud platforms and ecosystems, accelerating innovation and enabling more novel applications of the technology.

With the new 400B model packing enough power to rival ChatGPT 4, that puts a lot of power into researcher’s hands. This would allow for more rapid development of advanced language AI applications without relying on expensive proprietary APIs.

What we know so far

Meta AI has been hinting at the release of the 400B model since its original press release about Llama 3 on April 18. “Our largest models are over 400B parameters,” it wrote back then, adding that “over the coming months, we’ll release multiple models with new capabilities including multimodality, the ability to converse in multiple languages, a much longer context window, and stronger overall capabilities.

Since then, the internet has been abuzz with theories and ideas about a possible release date for the 400B models. While the folks at Meta have confirmed that development on Llama 3 400B has already wrapped up, no official release date has been announced as of yet.

Still, WhatsApp Beta users on Android 2.24.14.7 have spotted a new option to try out the Llama 3-405B model for Meta AI. While this option has currently been rolled out to beta users only and with significant limits on usage volume, it’s enough to get people excited about a full release, possibly in late July or August of 2024.

More from Tom's Guide

Category
Arrow
Arrow
Back to MacBook Air
Brand
Arrow
Processor
Arrow
RAM
Arrow
Storage Size
Arrow
Screen Size
Arrow
Colour
Arrow
Storage Type
Arrow
Condition
Arrow
Price
Arrow
Any Price
Showing 10 of 99 deals
Filters
Arrow
Load more deals
Ritoban Mukherjee

Ritoban Mukherjee is a freelance journalist from West Bengal, India whose work on cloud storage, web hosting, and a range of other topics has been published on Tom's Guide, TechRadar, Creative Bloq, IT Pro, Gizmodo, Medium, and Mental Floss.