The computer vision bias trilogy: Drift and monitoring.
Unforeseen data may be presented to the computer vision system during operation despite careful mitigation of datasets and shortcuts.

Unforeseen data may be presented to the computer vision system during operation despite careful mitigation of datasets and shortcuts.
If the past three years have taught us anything, it is that the world around us can take unexpected turns. The same can be true for your computer vision models.
Unforeseen data may be presented to the computer vision model during operation despite careful mitigation of datasets and shortcuts. One such phenomenon is data drift.
A hospital may change their x-ray machine and keep using the same computer vision model to diagnose, even though the system was not trained with this kind of input data. Similarly, an autonomous car solely built for European streets notable for their twists and turns, may not perform as expected if deployed in an American city.
ML models tend to fail silently and make predictions regardless, albeit erroneous ones. One can mitigate operational bias by adding the right mitigation strategies: the wider ML system should detect in operation if an image looks “suspicious” or “unknown”, and gracefully fail (for example, by asking the doctor for a closer look).
The problem of finding such problematic inputs is called out-of-distribution detection. The challenging problem involves comparing the distribution of high-dimensional objects. If you’re interested in learning more about it, the research in the area is extensive [1], [2], [3]. Note that out-of-distribution detection is a key part of many learning systems.
For example, Generative Adversarial Networks train a discriminator network whose sole task is to detect if a generated image is “suspicious” when judged against a reference dataset. Systems in production should be endowed with an out-of-distribution detector in order to detect problematic samples on the fly. If a problematic image is detected, the system should fail gracefully, thus reducing the risk of silent failures of your computer vision system.
It is essential to keep data drift in mind once your system is in production. Keeping the data and model up-to-date is just a part of any AI’s lifecycle. In the meantime, ensure that mitigation strategies are in place so those suspicious outcomes are detected and looked at by humans in the loop.
Get in touch with mateo@lakera.ai to find out more about what Lakera can do for your team, or get started right away.
Download this guide to delve into the most common LLM security risks and ways to mitigate them.
Get the first-of-its-kind report on how organizations are preparing for GenAI-specific threats.
Compare the EU AI Act and the White House’s AI Bill of Rights.
Get Lakera's AI Security Guide for an overview of threats and protection strategies.
Explore real-world LLM exploits, case studies, and mitigation strategies with Lakera.
Use our checklist to evaluate and select the best LLM security tools for your enterprise.
Discover risks and solutions with the Lakera LLM Security Playbook.
Discover risks and solutions with the Lakera LLM Security Playbook.
Subscribe to our newsletter to get the recent updates on Lakera product and other news in the AI LLM world. Be sure you’re on track!
Lakera Guard protects your LLM applications from cybersecurity risks with a single line of code. Get started in minutes. Become stronger every day.
Several people are typing about AI/ML security. Come join us and 1000+ others in a chat that’s thoroughly SFW.