“Llama Killer” Falcon 180B Shows Open-Source AI is Done Playing Catch-Up

This week, the United Arab Emirates (UAE) Technology Innovation Institute (TII) unveiled the largest open-source large language model (LLM) to date, Falcon 180B. It comes with 180 billion parameters and was trained on 3.5 trillion tokens.?

Falcon 180B uses TII’s RefinedWeb dataset, which uses data taken from public web-crawling research papers, legal text, news, literature, and social media conversations.??

This data means that the model can perform well when conducting tasks like reasoning, coding, proficiency, and knowledge tests.?

What is Falcon 180B? A New Challenger for the AI Crown?

Falcon 180B’s release comes just months after Meta launched the pre-trained model Llama 2 in July?and after TII launched Falcon 40B in May 2023. At launch, Llama-2 supported 180-billion parameters and was trained on 2 trillion tokens, which made it the largest open-source LLM at the time of release.?

However, TII’s new LLM is 2.5x bigger than Llama 2 and trained using 4x more computing power. It also outperforms Llama 2 in multi-task language understanding (MMLU) tasks. This is why some are calling Falcon 180B the Llama killer.

These performance advantages make Falcon 180B the largest open-source LLM on the market, and why it’s currently sitting at the top of Hugging Face’s Open LLM Leaderboard.???

Falcon 180B has also shown promising performance against proprietary LLMs, with Hugging Face suggesting that it can rival Google’s PaLM 2, the language model used to power Bard, and highlighting that it outright outperforms GPT-3.5.

That being said, it’s worth noting that the size of the model requires at least 320GB of memory in order to function, which can be a costly investment.?

In any case, while Falcon 180B isn’t on the level of GPT-4, it has demonstrated that the gap between open and closed-source AI is closing rapidly.?

As that gap continues to close, open-source platforms will be situated to carve out a much greater share of the market, particularly if organizations prefer the privacy offered by open-source LLMs.?

Open Source AI’s Privacy Edge vs. Proprietary Models?

Open-source AI models like Falcon 180B offer a distinct advantage over proprietary models in terms of data privacy.?

With an open-source AI model, an organization can train a pre-trained model on its own servers without sending data back to a third-party provider’s centralized model.?

This isn’t the case with most proprietary AI solutions. For instance, OpenAI, Google, and Anthropic all collect data from user conversations with their chatbots. This doesn’t occur with open-source LLM’s.?

However, it’s important to note that OpenAI has attempted to address these privacy concerns by launching ChatGPT Enterprise, which doesn’t collect data from user prompts, so it is likely other proprietary vendors will look to tighten data sharing in the future.?

Democratization and Open-Source AI??

At this level of performance, open-source AI solutions like Falcon 180B have the potential to democratize access to AI so that enterprises can experiment with this technology as part of applications and integrations with complete transparency rather than using products that have been built with an opaque black box approach.??

With open-source AI models, a community of researchers can work together and iterate on code and use cases to drive the development of the technology as a whole forward without being limited by proprietary gatekeepers.?

“We are committed to democratizing access to advanced AI, as our privacy and the potential impact of AI on humanity should not be controlled by a select few,” said Secretary General of the UAE’s Advanced Technology Research Council, H.E. Faisal Al Bannai in the announcement press release.

“While we may not have all the answers, our resolve remains unwavering: to collaborate and contribute to the open source community, ensuring that the benefits of AI are shared by all.”?

In addition, the launch of Falcon 180B will help to challenge the Silicon Valley monopoly of companies like Google, Meta, OpenAI, and Anthropic, which have been dominating AI innovation. It also helps to solidify the Middle East as a key region to watch for the development of these technologies going forward.?

Acceptable Use?

One of the most interesting elements of the release is that Falcon 180B has a more permissive acceptable use policy than other competitors like OpenAI and Anthrophic.?

For instance, Falcon 180B’s policy is broken down into four lines (paraphrased); don’t use it to violate local or international regulations, harm or exploit others, disseminate false information, or defame others.?

In contrast, OpenAI’s usage policy puts forward a list of 14 disallowed uses of its models and further requirements.

Disallowed use includes not only illegal activity and harassment but, more broadly, “activities that have a risk of economic harm,” adult content, political campaigning, non-qualified legal advice or financial advice, high-risk government decision-making, or providing instructions on how to treat health conditions.?

Anthropic’s acceptable use policy also puts forward a large number of prohibited uses and use cases.?

As a result, Falcon 180B has fewer guard rails than these proprietary models, which opens the door to more use cases but could open the door to other challenges around managing controversial content and misinformation.?

A Very Competitive Market?

Above all, Falcon 180B’s release highlights that AI is evolving faster than anyone can anticipate. Organizations can’t afford to assume that one provider will dominate the market forever.?

While OpenAI has enjoyed an early advantage after releasing ChatGPT last year, other competitors are constantly encroaching on that position with more parameters and performance.?

Once proprietary and open-source competitors reach a certain level of performance, it’s going to be more and more difficult for these providers to stand out from the crowd.?