LangChain offers versatile and user-friendly solutions for those looking to develop applications with Large Language Models (LLMs). But with development comes the essential task of ensuring the model's security.
For starters, the LLM in use shouldn't be easily prone to prompt injections and should avoid producing harmful content or engaging in user interactions that might lead to unintended outputs.
Fortunately for LLM developers, chaining LangChain's LLM component with Lakera Guard — either sequentially or in parallel — provides precisely this level of protection.
A common use case in LangChain is retrieval-augmented generation, namely an LLM-powered Q&A service that answers users’ questions based on a custom database. The following code shows how this can be achieved within LangChain.
Now, let's secure the Q&A service by integrating Lakera Guard into the code mentioned above.
First, we'll need to add the required imports, specify the Lakera API key, and define the function that requests AI security feedback from the Guard’s API—in this case, the detection of prompt injections.
We can chain Lakera Guard and the LLM sequentially. This means that we check if Lakera Guard has identified any AI security risk in the question before the question is sent to the LLM to get an answer.
If you simply want to get the AI security feedback alongside the answer, you can extend the code with the lakera_guard function that adds the feedback to the output.
Instead of invoking QA_chain directly, let us now chain lakera_guard and QA_chain sequentially. This is where the magic happens!
By stopping the question from being answered when an AI security issue is detected, you can save unnecessary calls to the LLM and thereby save money. To achieve that, use the following lakera_guard function instead:
Lakera Guard provides rapid feedback, with a typical response time under 200ms. However, if this latency is excessive for your use case, you can run Lakera Guard's AI security checks in parallel with answering the question. While this approach eliminates any latency, it does have a drawback: the LLM might be triggered even if a question is flagged as an AI security risk and potentially shouldn't be answered in the first place.
By following the above tutorial, you can integrate Lakera Guard into your LangChain application in less than an hour. Lakera Guard acts as a security layer between the user and any LLM. Through the Guard’s API, developers get security feedback on a user’s prompt to the LLM. This includes the detection of prompt injections, toxic content, and data loss. More specifically, Lakera Guard’s backbone consists of text classifiers for prompt injection, jailbreaking, hate speech, inappropriate sexual content, personally identifiable information, and unknown links (spam, phishing).
Download this guide to delve into the most common LLM security risks and ways to mitigate them.
Subscribe to our newsletter to get the recent updates on Lakera product and other news in the AI LLM world. Be sure you’re on track!
Lakera Guard protects your LLM applications from cybersecurity risks with a single line of code. Get started in minutes. Become stronger every day.
Several people are typing about AI/ML security. Come join us and 1000+ others in a chat that’s thoroughly SFW.