As LLMs gain traction across domains, hallucinations—distortions in LLM output—pose risks of misinformation and exposure of confidential data. Delve into the causes of hallucinations and explore best practices for their mitigation.
Large Language Models (LLMs) are at the forefront of technological discussions, known for their proficiency in processing and generating text that resembles human communication. They are transforming our interactions with technology. However, these models are not without their flaws. One significant issue is their tendency to produce "hallucinations," which affect their reliability.
Hallucinations in LLMs refer to the generation of content that is irrelevant, made-up, or inconsistent with the input data. This problem leads to incorrect information, challenging the trust placed in these models. Hallucinations are a critical obstacle in the development of LLMs, often arising from the training data's quality and the models' interpretative limits.
To use LLMs effectively, it's important to understand these hallucinations. Recognizing their limitations sharpens our insight into both the potential and the challenges of AI technologies.
This article examines the causes of hallucinations, their impact, and ongoing efforts to curb them, aiming to improve the trustworthiness and functionality of LLMs for future applications.
Want to prevent your LLMs from generating misleading or unsafe content? See how Lakera does it in real time.
The Lakera team has accelerated Dropbox’s GenAI journey.
“Dropbox uses Lakera Guard as a security solution to help safeguard our LLM-powered applications, secure and protect user data, and uphold the reliability and trustworthiness of our intelligent features.”
LLM hallucinations can be broken down into specific types, each with its unique characteristics and implications.
A clear classification helps developers and users alike identify, analyze, and address different hallucination scenarios. Such awareness is crucial for enhancing the models' accuracy and trustworthiness.
Hallucinations in Large Language Models (LLMs) are categorized into factuality and faithfulness hallucinations.
This occurs when an LLM generates factually incorrect content. For instance, a model might claim that Charles Lindbergh was the first to walk on the moon, which is a factual error. This type of hallucination arises due to the model's limited contextual understanding and the inherent noise or errors in the training data, leading to responses that are not grounded in reality.
Table 1 categorizes types of factuality hallucinations in LLMs with examples:
These are instances where the model produces unfaithful content or is inconsistent with the provided source content.
For example, in the context of summarization, if an article states that the FDA approved the first Ebola vaccine in 2019, a faithfulness hallucination would include a summary claiming that the FDA rejected it (intrinsic hallucination) or that China started testing a COVID-19 vaccine (extrinsic hallucination), neither of which is mentioned in the original article. (Source)
Table 2 presents examples of faithfulness hallucinations in Large Language Models (LLMs), where the model output deviates from the user's input or the context provided. It categorizes these hallucinations into three types:
The scope of hallucinations in LLMs is broader than task-specific models due to the diverse range of applications and the complex nature of the models.
Intrinsic hallucinations often contradict the original text or external knowledge, while extrinsic hallucinations introduce new, unverifiable information. This phenomenon is observed across various generative tasks, from summarization to dialogue generation and question answering, each posing unique challenges in maintaining accuracy and consistency.
For instance, in open-domain dialogue generation, intrinsic hallucination might involve a chatbot confusing facts or names, while extrinsic hallucination may include the bot making unverifiable claims. Similarly, in generative question answering, intrinsic hallucinations can manifest as responses that don’t align with the source material, and extrinsic hallucinations are answers containing information not found in the original documents.
Mitigating hallucinations in LLMs involves a multifaceted approach, including using scoring systems where human annotators rate the level of hallucination, compare generated content against baselines, and implement various product design strategies.
Red teaming, where human evaluators rigorously test the model, is crucial in identifying and addressing hallucinations. Product-level recommendations like user editability, structured input/output, and user feedback mechanisms also effectively reduce the risk of hallucinations.
Understanding the taxonomy of hallucinations in LLMs and their broader scope is essential for effectively deploying these models in various applications. Continuous efforts in mitigation and refinement are necessary to enhance the reliability and accuracy of LLM outputs. We will cover mitigation strategies in detail later in this article.
**💡 Pro Tip: To gain a deeper understanding of the different types of Large Language Models (LLMs) and their functionalities, explore the comprehensive guide at Large Language Models Guide.**
The causes of hallucinations in Large Language Models (LLMs) are multifaceted and stem from various aspects of their development and deployment.
Let’s dive deep into key causes of hallucinations in LLMs, including issues related to training data, architecture, and inference strategies.
A significant factor contributing to LLM hallucinations is the nature of the training data. LLMs, such as GPT, Falcon, and LlaMa, undergo extensive unsupervised training with large and diverse datasets from multiple origins.
Verifying this data's fairness, unbiasedness, and factual correctness is challenging. As these models learn to generate text, they may also pick up and replicate factual inaccuracies in the training data.
This leads to scenarios where the models cannot distinguish between truth and fiction and may generate outputs that deviate from facts or logical reasoning.
LLMs trained on internet-sourced datasets may include biased or incorrect information. This misinformation can propagate into the model's outputs, as the model doesn't distinguish between accurate and inaccurate data.
For instance, Bard's error regarding the James Webb Space Telescope indicates how reliance on flawed data can lead to confident but incorrect assertions.
Hallucinations can also arise from model architecture flaws or suboptimal training objectives.
For instance, an architecture flaw or a misaligned training objective can lead the model to produce outputs that do not align with the intended use or expected performance.
This misalignment can result in the model generating content that is either nonsensical or factually incorrect.
During the inference stage, several factors can contribute to hallucinations.
These include defective decoding strategies and the inherent randomness in the sampling methods used by the model.
Additionally, issues like insufficient context attention or the softmax bottleneck in decoding can lead to outputs needing to be adequately grounded in the provided context or the training data.
The way prompts are engineered can also influence the occurrence of hallucinations.
The LLM might generate an incorrect or unrelated answer if a prompt lacks adequate context or is ambiguously worded.
Effective prompt engineering requires clarity and specificity to guide the model toward generating relevant and accurate responses.
**💡 Pro Tip: Delve into effective prompt creation techniques to guide LLMs more accurately with the Prompt Engineering Guide.**
When generating text, LLMs use sampling strategies that can introduce randomness into the output.
For example, a high "temperature" setting can increase creativity and the risk of hallucination, as seen with language models generating entirely new plots or ideas.
However, these stochastic methods can sometimes result in unexpected or nonsensical responses, reflecting the probabilistic nature of the model's decision-making process.
LLMs may generate hallucinated content when faced with unclear or imprecise input.
In the absence of explicit information, models can fill gaps with invented data, as evidenced by the instance where ChatGPT created a false accusation against a professor due to an ambiguous prompt.
Sometimes, LLMs are optimized for certain outcomes, such as longer outputs, which can lead to verbose and irrelevant responses.
This over-optimization can cause models to stray from providing concise, accurate information to producing more content that may include hallucinations.
Addressing these factors involves improving data quality, refining model architecture, enhancing decoding strategies, and better prompt engineering to reduce the frequency and impact of hallucinations in LLMs.
Table 3: Summary of Hallucination Causes in Large Language Models Across Data, Training, and Inference Stages (Source)
Table 3 summarizes different types of nuanced causes of hallucinations in LLMs from outstanding research work by Lei Huang and the team. I highly recommend reading this paper as it covers hallucination causes in more detail with model output examples.
LLM Hallucinations can be dangerous and impactful, and we have seen some disastrous outcomes recently. Recently, a huge blowout occurred when a New York attorney used ChatGPT for legal research.
An example of the real-world implications of hallucinations in Large Language Models (LLMs) is the legal case of Mata v. Avianca.
Here, a New York attorney used ChatGPT for legal research, leading to the inclusion of fabricated citations and quotes in a federal case. Steven Schwartz admitted he used ChatGPT to help research the brief in a client's personal injury case against Colombian airline Avianca and unknowingly included the false citations.
The case highlighted the direct consequences of relying on AI-generated content without verification and raised broader ethical and professional concerns within the legal field. (Source)
Such incidents can significantly erode trust in AI technologies.
When LLMs produce hallucinations—outputs that are fabricated or inconsistent with facts—they risk creating misinformation.
The reliance on AI for tasks such as legal research or document review assumes the AI's outputs are reliable and trustworthy. However, when these outputs turn out to be hallucinations, it not only undermines the user's trust in the tool but can also lead to serious professional and legal repercussions, as was the case with the attorneys in Mata v. Avianca, who faced sanctions for their reliance on AI-generated, non-existent case law
The risk extends beyond individual cases to broader societal implications.
Misinformation stemming from AI hallucinations can cascade, influencing decision-making processes and potentially leading to cyberattacks.
In the legal sphere, such misinformation can taint the integrity of judicial proceedings, as judges and other legal professionals rely on accurate and factual case law to make informed decisions.
The Mata v. Avianca case is a cautionary tale of the imperative need for rigorous verification of AI-generated content and the importance of maintaining ethical standards in professional conduct.
The research paper “A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation” addresses the issue of hallucinations and how they affect the reliability of Large Language Models.
The authors propose a novel approach to detect and mitigate these hallucinations during text generation actively.
The approach comprises several steps.
Initially, it involves identifying potential hallucinations by leveraging the model's logit output values. This step is critical because it determines the candidates for hallucination in the generated text.
The next phase involves a validation procedure to check the correctness of the identified hallucinations. If a hallucination is confirmed, the process includes a mitigation strategy to rectify the error without introducing new hallucinations, even in cases of incorrectly detected hallucinations (false positives).
The results of the study are promising. The detection technique achieved a recall of approximately 88%, meaning it could identify a high percentage of actual hallucinations.
The mitigation technique effectively mitigated 57.6% of these correctly detected hallucinations. An important aspect of this mitigation technique is that it does not introduce new hallucinations, a critical factor in maintaining the overall integrity of the model's output.
Furthermore, the approach was tested on GPT-3.5 (text-davinci-003) in an “article generation task” and showed significant effectiveness in reducing the rate of hallucinations from 47.5% to 14.5% on average.
This demonstrates the approach's efficacy in a practical scenario and its potential to enhance the reliability and trustworthiness of large language models, which is vital for their broader adoption in real-world applications.
**💡 Pro Tip: Learn more about enhancing the safety and efficacy of LLM applications with insights from OWASP Top 10 for Large Language Model Applications Guide.**
Exploring different methods to reduce hallucinations in LLMs is crucial.
The study underlines the importance of continued research and development in this area to ensure the factual accuracy of AI-generated content.
The methodology used in this paper, focusing on active detection and mitigation of hallucinations, is a significant contribution to this field.
It sets a precedent for future research to build upon, encouraging the exploration of various approaches to enhance the reliability and effectiveness of these advanced AI systems.
The Knowledge Graph-based Retrofitting (KGR) method is a notable approach to mitigating hallucinations in Large Language Models.
This method, proposed by Xinyan Guan, Yanjiang Liu, Hongyu Lin, Yaojie Lu, Ben He, Xianpei Han, and Le Sun, incorporates LLMs with Knowledge Graphs (KGs).
It effectively addresses factual hallucination during the reasoning process by retrofitting initial draft responses of LLMs based on factual knowledge stored in KGs.
KGR leverages LLMs to autonomously extract, select, validate, and retrofit factual statements in model-generated responses, eliminating manual intervention. This method has significantly improved LLM performance on factual QA benchmarks, particularly in complex reasoning tasks.
The Knowledge Graph-based Retrofitting (KGR) method is a practical and successful approach to addressing hallucinations in Large Language Models (LLMs).
This technique and similar case studies showcase the potential of integrating LLMs with knowledge graphs to enhance accuracy and reliability in complex reasoning tasks.
KGR's autonomously refining LLM responses using factual data from knowledge graphs exemplifies a significant stride in mitigating hallucinations.
These advancements underline the importance and effectiveness of employing innovative strategies to ensure the factual integrity of LLM outputs.
**💡 Pro Tip: For insights into the challenges and solutions related to AI security in the context of LLMs, check out our article about AI security.**
Here is a list of academic papers, technical resources, and real-world case studies on LLMs and AI safety for further reading:
These resources provide different perspectives and insights into hallucinations in LLMs and how to mitigate them.
In the realm of Large Language Models, the phenomenon of generating plausible yet incorrect or nonsensical information, known as "hallucinations," poses a significant threat to the reliability and safety of these AI systems. This is especially concerning in areas where accuracy is paramount, such as healthcare or law.
Efforts to mitigate hallucinations are pivotal for maintaining the credibility and functionality of LLMs. Key methods for identifying and reducing these errors involve a combination of sophisticated metrics and critical human evaluations. These include:
Looking to the future, the development of LLMs is steering towards greater robustness and safety, with a strong emphasis on grounding responses in verified information.
Innovative methods such as SelfCheckGPT detect hallucinations by assessing the consistency of multiple generated answers to the same query. Furthermore, techniques such as chain-of-thought prompting and Retrieval-Augmented Generation (RAG) are explored to fortify the models' ability to provide precise and relevant information.
Consistency and accuracy in responses are evaluated using tools such as BERTScore and Natural Language Inference.
Efforts are continuously made to improve artificial intelligence by advancing both the precision of detection systems and the quality of large language models. This shows a strong dedication in the AI community to develop technology that is advanced, reliable, and trustworthy.
**💡 Pro Tip: Understand the benefits and methodology of integrating external information into LLMs through Retrieval-Augmented Generation.**
Download this guide to delve into the most common LLM security risks and ways to mitigate them.
Get the first-of-its-kind report on how organizations are preparing for GenAI-specific threats.
Compare the EU AI Act and the White House’s AI Bill of Rights.
Get Lakera's AI Security Guide for an overview of threats and protection strategies.
Explore real-world LLM exploits, case studies, and mitigation strategies with Lakera.
Use our checklist to evaluate and select the best LLM security tools for your enterprise.
Discover risks and solutions with the Lakera LLM Security Playbook.
Discover risks and solutions with the Lakera LLM Security Playbook.
Subscribe to our newsletter to get the recent updates on Lakera product and other news in the AI LLM world. Be sure you’re on track!
Lakera Guard protects your LLM applications from cybersecurity risks with a single line of code. Get started in minutes. Become stronger every day.
Several people are typing about AI/ML security. Come join us and 1000+ others in a chat that’s thoroughly SFW.
