Continuous testing and model selection with Lakera and Voxel51

We are excited to announce the release of our first integration with FiftyOne by Voxel51. This integration makes it possible to benefit from FiftyOne's powerful visualization features to dig into the insights generated by Lakera's MLTest. Read on to learn how you can benefit from this.

Santiago Arias
December 1, 2023
January 6, 2023

We are excited to announce the release of our first integration with FiftyOne by Voxel51.

Lakera is a leader in evaluating and testing computer vision models and data–to ensure safe, fair, and robust AI. FiftyOne by Voxel51 is a fantastic tool to visualize your data and curating high-quality datasets.

💡 Already a Voxel51 user? You can get started with MLTest in minutes.

Now we've made it possible to benefit from FifyOne's powerful visualization features to dig into the insights generated by Lakera's MLTest. This is particularly convenient if you are already a FiftyOne user, as everything will fit right into your workflow.

What are your benefits?

Easier and more thorough model comparisons.

Every ML engineer has to constantly compare and select the best models for deployment. Either to compare a release candidate to the current production version, to test for regressions when quantizing models, or while doing a hyperparameter search. MLTest provides in-depth functionality to evaluate and compare models that go beyond standard ML metrics and include robustness, fairness, generalizability, and more. It also gives a neat diff-like comparison of one model against another, enabling you to make your decisions quickly.

Model comparison for two computer vision models. Showing a diff similar to code review tools.
MLTests model comparison feature to help select the right model for production.

Our new integration with FiftyOne now enables you to go deeper–for those times when you need to compare models on a prediction-by-prediction basis. To that end, we export all model predictions for individual images to easily load and inspect them with FiftyOne. This makes it easy to quickly compare the predictions for a specific class and filter by false positives/negatives.

Images and predictions from MLTest imported to FiftyOne for closer inspection.

Deep failure analysis/debugging.

MLTest automatically finds and clusters images on which your models don't perform as expected. This happens by analyzing your model's performance and grouping the images based on image embeddings, metadata you provide, and metadata MLTest automatically adds to your images. You can see an example below:

Clusters of images on which the ML model does not perform well. Here showing cluster where precision is lower than expected.
MLTest automatically finds clusters of images on which a model doesn’t perform well.

Getting started with Lakera and FiftyOne is easy. Head over to our docs on the integration. It includes an executable example as a Jupyter Notebook to get you up and running as quickly as possible. We've also previously reviewed the onboarding experience and features of FiftyOne, here.

💡 Learn more about MLTest's benefits, getting started is a matter of minutes.

If you have any questions or would like to continue the discussion around other ways to integrate FiftyOne, please reach out to Santiago at

Lakera LLM Security Playbook
Learn how to protect against the most common LLM vulnerabilities

Download this guide to delve into the most common LLM security risks and ways to mitigate them.

Santiago Arias
Read LLM Security Playbook
Learn about the most common LLM threats and how to prevent them.
You might be interested
No items found.
untouchable mode.
Get started for free.

Lakera Guard protects your LLM applications from cybersecurity risks with a single line of code. Get started in minutes. Become stronger every day.

Join our Slack Community.

Several people are typing about AI/ML security. 
Come join us and 1000+ others in a chat that’s thoroughly SFW.