Meta releases Llama 3 open-source AI model: Better than GPT-4?
What's the story
Meta has launched the latest additions to its Llama series of open-source generative AI models, namely the Llama 3.
The new family comprises two models, Llama 3 8B and Llama 3 70B, containing eight billion and 70 billion parameters, respectively.
This development marks a significant performance enhancement compared to their predecessors, the Llama 2 models, the Llama 2 8B and Llama 2 70B.
Performance
Llama 3 trained on two custom-built GPU clusters
The Llama 3 models are being hailed as some of the best-performing generative AI models currently available, based on their respective parameter counts.
Their training was conducted on two custom-built GPU clusters with a capacity of 24,000 units.
Meta substantiates this claim by referencing the impressive scores achieved by the Llama 3 models on popular AI benchmarks such as MMLU, ARC, and DROP.
These benchmarks measure the model's knowledge, skill acquisition, and reasoning over chunks of text, respectively.
Superiority
Llama 3 models outperform rivals in AI benchmarks
Compared to other open models like Mistral's Mistral 7B and Google's Gemma 7B, both with seven billion parameters, Llama 3 8B surpasses them on at least nine benchmarks.
These include MMLU, ARC, DROP, GPQA, HumanEval, GSM-8K, MATH, AGIEval, and BIG-Bench Hard.
The larger model, Llama 3 70B is competitive with flagship generative AI models like Gemini 1.5 Pro on HumanEval, MMLU, and GSM-8K benchmarks.
It also beats Anthropic's Claude 3 Sonnet, on five benchmarks (GPQA, MMLU, HumanEval, GSM-8K and MATH).
Training & testing
Enhanced training dataset and testing for Llama 3 models
The training dataset for Llama 3 was approximately seven times larger than that of Llama 2, including four times more code.
Synthetic data was also utilized to create longer documents for training purposes.
Meta has created its own test set covering various use cases from—coding and creative writing to reasoning and summarization—where the Llama 3 70B model outperformed several of its competitors.
Features & safety
Llama 3 models offer enhanced features and improved safety measures
New Llama models offer enhanced "steerability," a reduced tendency to refuse to answer questions.
They also boast improved accuracy on trivia questions, queries related to history and STEM fields, and general coding recommendations.
In response to concerns about toxicity and bias in generative AI models, Meta has developed new data-filtering pipelines for its model training data.
It has also upgraded its pair of generative AI safety suites, Llama Guard and CybersecEval.
Accessibility
Llama 3 models are now accessible
The Llama 3 models are now accessible for download and are powering Meta's AI assistant on various platforms including Facebook, Instagram, WhatsApp, and the web.
They will soon be hosted in managed form across a wide range of cloud platforms including Databricks, AWS, Google Cloud, Hugging Face, Kaggle, IBM's WatsonX, Microsoft Azure, NVIDIA's NIM, and Snowflake.
Future plans
Meta imposes restrictions and plans for larger Llama 3 models
Despite being labeled as "open," Meta imposes certain restrictions on the use of its Llama models.
Developers are prohibited from using Llama models to train other generative models.
Additionally, app developers with more than 700 million monthly users must request a special license from Meta to use the new models.
The company is also training larger Llama 3 models with over 400 billion parameters, aiming to make them "converse in multiple languages" and understand images as well as text.