OpenAI vs Meta Using - AI Podcast - Episode 18

In this episode of Using AI, myself, Senior ML Research Scientist Alex Pap and AI Startup Founder and CTO Nitish Mutha continue our series of large language model comparisons with look at Meta vs OpenAI.

Listen to Using AI on Apple Podcasts, Spotify and watch the episode on YouTube, where you can see our awesome llama backgrounds (though I don't think I got the memo for that).

Introducing Llama

Llama, launched by Meta at the end of February 2023, sparked a significant boom in finetuned models. Despite its source code being open, the weights weren't. However, these weights leaked a few days after the launch. The model, interestingly, had no instruction or conversation tuning, and no RLHF. The birth of Llama also led to the emergence of other models such as Alpaca and Vicuna. For more on Llama, listen to Using AI on Apple Podcasts, Spotify, and watch the episode on YouTube.

Llama 2: A Closer Look

Launched in July 2023, Llama 2 was made free for research and commercial use. The model came in various sizes: 7B, 13B, and 70B. But despite its weights being released and the model being downloadable, Llama 2 was not completely open-source. Restrictions were in place to protect Meta's market share.

One of the hindrances included a stipulation that anyone with more than 700 million active users needed a special license. Furthermore, it was prohibited to output results of Llama to improve any other LLM. Hosting Llama 2 was possible through AWS bedrock, Cloudflare, GCP, and Hugging Face.

Llama 3: What's Next?

There's a buzz about Llama 3 being on the horizon. Will it be open-sourced? Rumour has it that Mark Zuckerberg wants to focus more on AI products. However, one cannot overlook the fact that Meta's decision to open-source Llama gives the company some control and authority. It reduces the adoption rate and allows Meta to sell services on top of it. It's a fast customer acquisition channel that also provides a free feedback loop, helping Meta learn and find limitations.

Meta vs OpenAI: The Big Debate

Meta's approach to AI differs significantly from that of OpenAI, Google, Microsoft, and other organisations. This was evident when Meta's chief AI scientist, Yann LeCun, criticised the companies for their “massive corporate lobbying” to write rules that favour their high-performing AI models.

Meta's vast amount of images and user permissions could potentially give them an edge in AI development. Are we heading towards a monopoly? Or does open-sourcing AI have its drawbacks, such as the lack of guardrails and potential irresponsibility?

Links:

• Llama Access Form (meta.com)
• Llama 2 on Hugging face
• Llama 2 7B Chat on Hugging Face
• Meta uses Copyrighted books on AI training despite its lawyers warnings
• Meta + IBM Open Source AI partnership

Listen to Using AI on Apple Podcasts, Spotify and watch the episode on YouTube.