Cookie Consent
Hi, this website uses essential cookies to ensure its proper operation and tracking cookies to understand how you interact with it. The latter will be set only after consent.
Read our Privacy Policy
Back

AI Security with Lakera: Aligning with OWASP Top 10 for LLM Applications

Discover how Lakera's security solutions correspond with the OWASP Top 10 to protect Large Language Models, as we detail each vulnerability and Lakera's strategies to combat them.

David Haber
June 11, 2024
Learn how to protect against the most common LLM vulnerabilities

Download this guide to delve into the most common LLM security risks and ways to mitigate them.

In-context learning

As users increasingly rely on Large Language Models (LLMs) to accomplish their daily tasks, their concerns about the potential leakage of private data by these models have surged.

[Provide the input text here]

[Provide the input text here]

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

Lorem ipsum dolor sit amet, Q: I had 10 cookies. I ate 2 of them, and then I gave 5 of them to my friend. My grandma gave me another 2boxes of cookies, with 2 cookies inside each box. How many cookies do I have now?

Title italic

A: At the beginning there was 10 cookies, then 2 of them were eaten, so 8 cookies were left. Then 5 cookieswere given toa friend, so 3 cookies were left. 3 cookies + 2 boxes of 2 cookies (4 cookies) = 7 cookies. Youhave 7 cookies.

English to French Translation:

Q: A bartender had 20 pints. One customer has broken one pint, another has broken 5 pints. A bartender boughtthree boxes, 4 pints in each. How many pints does bartender have now?

Lorem ipsum dolor sit amet, line first
line second
line third

Lorem ipsum dolor sit amet, Q: I had 10 cookies. I ate 2 of them, and then I gave 5 of them to my friend. My grandma gave me another 2boxes of cookies, with 2 cookies inside each box. How many cookies do I have now?

Title italic Title italicTitle italicTitle italicTitle italicTitle italicTitle italic

A: At the beginning there was 10 cookies, then 2 of them were eaten, so 8 cookies were left. Then 5 cookieswere given toa friend, so 3 cookies were left. 3 cookies + 2 boxes of 2 cookies (4 cookies) = 7 cookies. Youhave 7 cookies.

English to French Translation:

Q: A bartender had 20 pints. One customer has broken one pint, another has broken 5 pints. A bartender boughtthree boxes, 4 pints in each. How many pints does bartender have now?

As organizations increasingly integrate Large Language Models (LLMs) into their operations, the Open Web Application Security Project (OWASP) Top 10 has become the go-to reference for understanding and mitigating the primary risks associated with these applications.

Before diving deeper into Lakera’s alignment with OWASP, let’s first better understand the OWASP framework.

Hide table of contents
Show table of contents

Overview of the OWASP LLM Top 10

The rise of LLMs has given birth to new use cases like LLM-based chatbots, RAG applications and intelligent document summarization tools.

By using LLMs, these applications are prone to novel cybersecurity threats that are specific to LLMs and have never been seen before.

The OWASP framework summarizes all the risks an LLM application faces, both LLM specific and traditional ones.

The framework comprises both threats occurring during development of the application as well as at deployment (“runtime”). The risks outlined in the framework are summarized below.

OWASP Top 10 in Detail: Example of a Customer Service Agent

To better understand the OWASP framework, we will now exemplify the threats with the help of a customer service agent.

This customer service agent is deployed by a financial services company with the goal of assisting customers for various banking services.

The agent is based on an LLM and has been fine-tuned with historical customer service interactions.

Plus, it has access to an internal financial database containing sensitive information such as customer account details and transaction histories.

During runtime, such a customer service agent is prone to various forms of risks:

  • LLM01 Prompt Injection: A malicious actor might attempt to manipulate the agent using natural language as a form of attack. By crafting deceptive prompts, the attacker can cause the agent to perform unintended actions like incorrectly referring to the transaction history, insulting the user for their financial behavior or providing malicious phishing links.
  • LLM02 Insecure Output Handling: The output of the agent’s LLM can inadvertently expose sensitive information from the back-end system, emphasizing that threats can arise not only from user inputs but also from LLM outputs. For instance, the output could expose internal system details of the agent which could be exploited by attackers.
  • LLM04 Model Denial of Service: As a form of the traditional Denial of Service (DoS) attacks, this attack aims at overwhelming the agent and model with excessive requests. By flooding the underlying model with a high volume of complex queries, the agent can slow down or become unresponsive, making it effectively unusable. Besides, due to the generally higher operating costs of LLMs, this could also have significant financial implications.
  • LLM06 Sensitive Data/PII: Having access to the internal financial database, the agent may unintentionally disclose confidential data about customer accounts or transactions. This can lead to unauthorized data access, privacy violations and security breaches.
  • LLM08 Excessive Agency: The agent may be manipulated by a jailbreak into gaining excessive functionalities and autonomy. Consequently, this might enable the agent to alter the internal database it has access to, i.e., changing the account details or transaction histories.
  • LLM09 Overreliance: Customers may over rely on the output of the LLM-based agent, by blindly trusting its advice or recommendations. As the output can be incorrect or inappropriate, additional security layers are required.

Meanwhile, vulnerabilities can also arise during the agent’s development and are outlined in the following:

  • LLM03 Training Data Poisoning: When fine-tuning the agent with historical customer service interactions, malicious actors can introduce vulnerabilities into the data or application. These vulnerabilities serve as backdoors and can later be exploited through prompt injections, leading to false recommendations, data leaks or unauthorized access to the underlying account database.
  • LLM05 Supply Chain Vulnerability: Further risks can be introduced into the application if other components or services used during development have been compromised. For example, if the agent utilized third-party libraries or relies on third-party models, vulnerabilities in these components can be exploited by an attacker through injecting malicious code or manipulating the agent’s behavior.
  • LLM07 Insecure Plugin Design: During the development, the agent could have been enhanced by third-party plugins. Similar to the supply chain vulnerabilities, the plugins can introduce significant risks if they are not securely designed. This could lead to leakage of account data or altering of transactions.
  • LLM10 Model Theft: Model Theft is only relevant for applications that are based on self-developed, proprietary models. It describes the risk of malicious actors acquiring insights on the proprietary model and replicating it. This does not apply to the agent in our example, as third-party models are used.

How to Secure an LLM-Based Application

The OWASP LLM Top 10 outline major threats to LLM applications, consisting of both novel risks like LLM01 Prompt Injection and traditional cybersecurity risks like LLM05 Supply Chain Vulnerability. Beyond the threats mentioned by OWASP, LLM applications are targets of further traditional cybersecurity threats such as credential theft or infrastructure attacks.

Holistically securing an LLM application requires addressing both traditional and LLM specific cybersecurity risks. This means complementing existing cybersecurity solutions with innovative, LLM-focused protections. 

Lakera has been at the forefront of building novel AI cybersecurity solutions that complement the traditional security stack and can protect applications against LLM specific threats.

Below, we will outline how Lakera Guard, Lakera’s flagship product, protects LLM applications at runtime.

Lakera Guard: Protecting Against LLM-Specific Runtime Risks

Lakera Guard protects LLM applications against LLM-specific risks at runtime. Acting as a firewall, Lakera Guard uses context-aware, state-of-the-art classifiers to detect prompt injections, data leakage and harmful content in both input and output of your LLM application.

Looking again at our imaginary example of the LLM-based customer service agent, Lakera Guard holistically secure against all runtime risks mentioned above by OWASP:

Prompt Injection Defense

Through its prompt injection classifier, Lakera Guard detects and neutralizes any kind of prompt injections (LLM01 Prompt Injection). Besides, prompt injections and jailbreaks can induce further risks, as they are needed to gain excessive agency or trigger backdoors introduced by training data poisoning.

By neutralizing prompt injections, Lakera Guard helps to protect against these prompt injection-induced threats such as LLM03 Training Data Poisoning and LLM08 Excessive Agency. Lastly, as both inputs and outputs are being filtered, Lakera Guard also mitigates LLM02 Insecure Output Handling.

**💡 Pro Tip: Check out our Prompt Injection Attacks Handbook** 

Data Leakage Protection

Similarly, Lakera Guard’s data leakage classifiers prevent spilling of sensitive data like PII, protecting LLM applications against LLM06 Sensitive Data/PII. This also blocks sharing of sensitive data with the LLM provider in case of PII input.

Content Moderation

As part of Lakera Guard’s holistic protection, application outputs are also scanned for policy violations, harmful content, profanity and hate speech. This provides an additional layer of security needed for LLM09 Overreliance.

Versatile Deployment During Development

While Lakera Guard’s core focus lies on runtime protection, Lakera Guard’s API first approach allows deployment also in earlier stages of the software development lifecycle.

For instance, Lakera Guard can be used to analyze training data for prompt injections and jailbreaks, securing the application already during development against LLM03 Training Data Poisoning. 

What’s Next?

Reach out to us, and together we'll ensure your LLM applications are not only powerful and intelligent but also safe and trusted!

You can get started for free with Lakera Guard or book a demo here.

Lakera LLM Security Playbook
Learn how to protect against the most common LLM vulnerabilities

Download this guide to delve into the most common LLM security risks and ways to mitigate them.

Unlock Free AI Security Guide.

Discover risks and solutions with the Lakera LLM Security Playbook.

Download Free

Explore Prompt Injection Attacks.

Learn LLM security, attack strategies, and protection tools. Includes bonus datasets.

Unlock Free Guide

Learn AI Security Basics.

Join our 10-lesson course on core concepts and issues in AI security.

Enroll Now

Evaluate LLM Security Solutions.

Use our checklist to evaluate and select the best LLM security tools for your enterprise.

Download Free

Uncover LLM Vulnerabilities.

Explore real-world LLM exploits, case studies, and mitigation strategies with Lakera.

Download Free

The CISO's Guide to AI Security

Get Lakera's AI Security Guide for an overview of threats and protection strategies.

Download Free

Explore AI Regulations.

Compare the EU AI Act and the White House’s AI Bill of Rights.

Download Free
David Haber

The CISO's Guide to AI Security

Get Lakera's AI Security Guide for an overview of threats and protection strategies.

Free Download
Read LLM Security Playbook

Learn about the most common LLM threats and how to prevent them.

Download

Explore AI Regulations.

Compare the EU AI Act and the White House’s AI Bill of Rights.

Understand AI Security Basics.

Get Lakera's AI Security Guide for an overview of threats and protection strategies.

Uncover LLM Vulnerabilities.

Explore real-world LLM exploits, case studies, and mitigation strategies with Lakera.

Optimize LLM Security Solutions.

Use our checklist to evaluate and select the best LLM security tools for your enterprise.

Master Prompt Injection Attacks.

Discover risks and solutions with the Lakera LLM Security Playbook.

Unlock Free AI Security Guide.

Discover risks and solutions with the Lakera LLM Security Playbook.

You might be interested
10
min read
AI Security

Chatbot Security Essentials: Safeguarding LLM-Powered Conversations

Discover the security threats facing chatbots and learn strategies to safeguard your conversations and sensitive data.
Emeka Boris Ama
March 26, 2024
10
min read
AI Security

Data Loss Prevention in the Age of Generative AI (with Lakera's Insights)

Learn about data loss prevention in the context of generative AI. Explore some best practices to ensure error-free DLP implementation.
Haziqa Sajid
February 28, 2024
Activate
untouchable mode.
Get started for free.

Lakera Guard protects your LLM applications from cybersecurity risks with a single line of code. Get started in minutes. Become stronger every day.

Join our Slack Community.

Several people are typing about AI/ML security. 
Come join us and 1000+ others in a chat that’s thoroughly SFW.