LLama

LLaMA (Large Language Model Meta AI) is a foundational large language model publicly released by Meta to promote open science. Unlike gigantic models which demand substantial resources to train and run, LLaMA is smaller and more efficient, offering a solution to democratize access and study in the expansive field of AI.

Intended to act as a foundation, LLaMA is available in different sizes and is designed for fine-tuning across various tasks. This release aims to provide broader research access to these models, facilitating deeper understanding, advancements, and problem mitigation, like bias and misinformation.

How LLaMA works

1. Training and Tokens: Unlike enormous models, LLaMA demands less computing power, making it easier for researchers to test and validate approaches. LLaMA models, such as the 65B and 33B, are trained on 1.4 trillion tokens, while the 7B variant is trained on one trillion tokens.
2. Functionality: Similar to other large language models, LLaMA predicts subsequent words based on an input sequence of words, effectively generating text. It has been trained on text from the 20 most spoken languages, primarily focusing on those with Latin and Cyrillic scripts.
3. Challenges and Improvements: While LLaMA presents a significant advancement, it still shares some common challenges with other models, including risks of bias, toxicity, and generating false information. However, by open-sourcing LLaMA, Meta encourages researchers to explore and implement solutions to these issues. Accompanying the model, there are evaluations that highlight the model's limitations, particularly concerning bias and toxicity.
4. Access and Licensing: To uphold responsible usage, LLaMA is released under a noncommercial license aimed at research applications. Access is determined on a case-by-case basis for various stakeholders, including academic researchers and industry labs. A dedicated application link for accessing the model is available in Meta's research paper.
5. Collaborative Efforts: Emphasizing the collective responsibility, Meta advocates for the broader AI community—including researchers, policymakers, and the industry—to collaboratively develop guidelines around responsible AI. This release of LLaMA is a step towards that goal, aiming to empower the community to innovate responsibly.