In contrast to traditional software systems, data is the core ingredient for ML (machine learning). Engineers spend significant amounts of time collecting the “right” datasets. That data drives training and evaluation processes and, ultimately, the quality of the machine learning systems we build. Given the importance of data, how do data bugs arise, and how can one catch them in time?
Before we answer these questions, note that we defined ML bugs as follows [1]:
“A machine learning bug refers to any imperfection in a machine learning item that causes a discordance between the existing and the required conditions.”
Two families of data bugs that often come up in practice can be summarised by these two questions:
- Do I have the wrong data? For example, data points can be duplicated or corrupted.
- Do I have the right data? Even if the data is bug-free, building a system that works at night is impossible if no night images are available in the data.
These families of data bugs are distinct but it’s important to keep track of both as you develop your ml models. Appropriate tests should be written to systematically look for vulnerabilities in both categories.
We will take a closer look at these questions specifically in the context of building a computer vision system. The question of how to check data quality issues during testing and operation is equally important. Read on to dig deeper into whether you have the wrong or right data.
Do I have the wrong data?
Naturally, the first question to ask is whether there are actual errors in the data. Below, we have a brief checklist of ML bugs that often arise in practice.
Do I have the wrong data?
– Missing values: Some values in the data/metadata may be missing for some of the images.
– Incorrect annotations: Inconsistency and noise in the annotations can hinder model performance.
– Data consistency: Some images may have the wrong size, number of channels, or metadata columns — to name a few possibilities. Alternatively, the range of some of the pixels may be off.
– Clear outliers: Some images/metadata may be clearly suspicious and need closer inspection.
– Corrupted data: Some data points may be corrupted.
– Duplicates: Some images may be duplicated. This can bias the machine learning models at training and may also result in the same image being both in the training set and in the validation set.
Adding tests that check for the items in the list above allows us to build a high-coverage test suite that ensures that any issues are caught in time. For example, let’s say you write a test that checks whether the shape of the images in the dataset is correct. In this case, they are all squares. This simple setup would catch rectangle-shaped images if they are added to the dataset in the future.
Do I have the right data?
A more subtle but fundamentally important form of data bugs looks at whether you have all the right data to build the system you envision.
As an example, if you deploy an autonomous driving component, your system needs to do well on roundabouts. If the data used to train your model has no roundabouts, or only very few of them, the system cannot be expected to perform well – even if the system is otherwise bug-free. So it’s essential that your dataset is representative of the intended use case.
Metadata is a powerful tool when testing data representativity. It contains high-level semantic information which can be used to specify relevant tests. In medical imaging, for instance, DICOM (digital imaging and communication in medicine) is the standard format for storing images, which can contain metadata such as the patient’s gender. By explicitly testing that a minimum number of images for each gender is available, we ensure that the system does not suffer from critical imbalances. Similarly, the time of day at which an image was taken can be used to test that images have been taken regularly throughout the day, mitigating issues such as too few images taken in the nighttime.
It is also important to keep track of the potential mismatch between the system you have built and the system you want to build. Testing for data coverage is the first step to ensuring this is avoided. Finding missing aspects in the data can drive data collection, lead to better representativity, and provide better system performance.
Example: Detecting data bugs in Gruyères.
Imagine that we’re building a system for autonomous driving in the city of Gruyères, CH. We need to establish a high-level description of specifications and tests that can be built on data to ensure a better system.
Do I have the wrong data?
Do I have the right data?
Creating data tests requires an upfront investment from engineering teams. We still see too many teams that want to “first focus on building a prototype” and then “worry about data quality” later. While this may work for a few iterations, teams quickly find themselves in situations where data bugs have gone unnoticed and cause substantial delays to their projects. The most mature teams make data quality testing and monitoring a top priority from early on in their projects. They put mechanisms in place to ensure adequate quality levels at all times during development.
Get started with Lakera today.
Get in touch with mateo@lakera.ai to find out more about what Lakera can do for your team, or get started right away.
Learn how to protect against the most common LLM vulnerabilities
Download this guide to delve into the most common LLM security risks and ways to mitigate them.
Don’t miss the updates!
Subscribe to our newsletter to get the recent updates on Lakera product and other news in the AI LLM world. Be sure you’re on track!
You might be interested
Why testing should be at the core of machine learning development.
AI (artificial intelligence) is capable of helping the world scale solutions to our biggest challenges but if you haven’t experienced or heard about AI’s mishaps then you’ve been living under a rock. Coded bias, unreliable hospital systems and dangerous robots have littered headlines over the past few years.
Stress-test your models to avoid bad surprises.
Will my system work if image quality starts to drop significantly? If my system works at a given occlusion level, how much stronger can occlusion get before the system starts to underperform? I have faced such issues repeatedly in the past, all related to an overarching question: How robust is my model and when does it break?
Activate
untouchable mode.
untouchable mode.
Get started for free.
Lakera Guard protects your LLM applications from cybersecurity risks with a single line of code. Get started in minutes. Become stronger every day.
Join our Slack Community.
Several people are typing about AI/ML security. Come join us and 1000+ others in a chat that’s thoroughly SFW.
