Cookie Consent
Hi, this website uses essential cookies to ensure its proper operation and tracking cookies to understand how you interact with it. The latter will be set only after consent.
Read our Privacy Policy
Back

Continuous testing and model selection with Lakera and Voxel51

We are excited to announce the release of our first integration with FiftyOne by Voxel51. This integration makes it possible to benefit from FiftyOne's powerful visualization features to dig into the insights generated by Lakera's MLTest. Read on to learn how you can benefit from this.

Santiago Arias
December 1, 2023
January 6, 2023
Learn how to protect against the most common LLM vulnerabilities

Download this guide to delve into the most common LLM security risks and ways to mitigate them.

In-context learning

As users increasingly rely on Large Language Models (LLMs) to accomplish their daily tasks, their concerns about the potential leakage of private data by these models have surged.

[Provide the input text here]

[Provide the input text here]

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

Lorem ipsum dolor sit amet, Q: I had 10 cookies. I ate 2 of them, and then I gave 5 of them to my friend. My grandma gave me another 2boxes of cookies, with 2 cookies inside each box. How many cookies do I have now?

Title italic

A: At the beginning there was 10 cookies, then 2 of them were eaten, so 8 cookies were left. Then 5 cookieswere given toa friend, so 3 cookies were left. 3 cookies + 2 boxes of 2 cookies (4 cookies) = 7 cookies. Youhave 7 cookies.

English to French Translation:

Q: A bartender had 20 pints. One customer has broken one pint, another has broken 5 pints. A bartender boughtthree boxes, 4 pints in each. How many pints does bartender have now?

Lorem ipsum dolor sit amet, line first
line second
line third

Lorem ipsum dolor sit amet, Q: I had 10 cookies. I ate 2 of them, and then I gave 5 of them to my friend. My grandma gave me another 2boxes of cookies, with 2 cookies inside each box. How many cookies do I have now?

Title italic Title italicTitle italicTitle italicTitle italicTitle italicTitle italic

A: At the beginning there was 10 cookies, then 2 of them were eaten, so 8 cookies were left. Then 5 cookieswere given toa friend, so 3 cookies were left. 3 cookies + 2 boxes of 2 cookies (4 cookies) = 7 cookies. Youhave 7 cookies.

English to French Translation:

Q: A bartender had 20 pints. One customer has broken one pint, another has broken 5 pints. A bartender boughtthree boxes, 4 pints in each. How many pints does bartender have now?

Hide table of contents
Show table of contents

We are excited to announce the release of our first integration with FiftyOne by Voxel51.

Lakera is a leader in evaluating and testing computer vision models and data–to ensure safe, fair, and robust AI. FiftyOne by Voxel51 is a fantastic tool to visualize your data and curating high-quality datasets.

💡 Already a Voxel51 user? You can get started with MLTest in minutes.

Now we've made it possible to benefit from FifyOne's powerful visualization features to dig into the insights generated by Lakera's MLTest. This is particularly convenient if you are already a FiftyOne user, as everything will fit right into your workflow.

What are your benefits?

Easier and more thorough model comparisons.

Every ML engineer has to constantly compare and select the best models for deployment. Either to compare a release candidate to the current production version, to test for regressions when quantizing models, or while doing a hyperparameter search. MLTest provides in-depth functionality to evaluate and compare models that go beyond standard ML metrics and include robustness, fairness, generalizability, and more. It also gives a neat diff-like comparison of one model against another, enabling you to make your decisions quickly.

Model comparison for two computer vision models. Showing a diff similar to code review tools.
MLTests model comparison feature to help select the right model for production.

Our new integration with FiftyOne now enables you to go deeper–for those times when you need to compare models on a prediction-by-prediction basis. To that end, we export all model predictions for individual images to easily load and inspect them with FiftyOne. This makes it easy to quickly compare the predictions for a specific class and filter by false positives/negatives.

Images and predictions from MLTest imported to FiftyOne for closer inspection.

Deep failure analysis/debugging.

MLTest automatically finds and clusters images on which your models don't perform as expected. This happens by analyzing your model's performance and grouping the images based on image embeddings, metadata you provide, and metadata MLTest automatically adds to your images. You can see an example below:

Clusters of images on which the ML model does not perform well. Here showing cluster where precision is lower than expected.
MLTest automatically finds clusters of images on which a model doesn’t perform well.

Getting started with Lakera and FiftyOne is easy. Head over to our docs on the integration. It includes an executable example as a Jupyter Notebook to get you up and running as quickly as possible. We've also previously reviewed the onboarding experience and features of FiftyOne, here.

💡 Learn more about MLTest's benefits, getting started is a matter of minutes.

If you have any questions or would like to continue the discussion around other ways to integrate FiftyOne, please reach out to Santiago at santiago@lakera.ai.

Lakera LLM Security Playbook
Learn how to protect against the most common LLM vulnerabilities

Download this guide to delve into the most common LLM security risks and ways to mitigate them.

Unlock Free AI Security Guide.

Discover risks and solutions with the Lakera LLM Security Playbook.

Download Free

Master Prompt Injection Attacks.

Learn LLM security, attack strategies, and protection tools. Includes bonus datasets.

Unlock Free Guide

Learn AI Security Basics.

Join our 10-lesson course on core concepts and issues in AI security.

Enroll Now

Optimize LLM Security Solutions.

Use our checklist to evaluate and select the best LLM security tools for your enterprise.

Download Free

Uncover LLM Vulnerabilities.

Explore real-world LLM exploits, case studies, and mitigation strategies with Lakera.

Download Free

Understand AI Security Basics.

Get Lakera's AI Security Guide for an overview of threats and protection strategies.

Download Free

Explore AI Regulations.

Compare the EU AI Act and the White House’s AI Bill of Rights.

Download Free
Santiago Arias
Read LLM Security Playbook

Learn about the most common LLM threats and how to prevent them.

Download
You might be interested
No items found.
Activate
untouchable mode.
Get started for free.

Lakera Guard protects your LLM applications from cybersecurity risks with a single line of code. Get started in minutes. Become stronger every day.

Join our Slack Community.

Several people are typing about AI/ML security. 
Come join us and 1000+ others in a chat that’s thoroughly SFW.