Cookie Consent
Hi, this website uses essential cookies to ensure its proper operation and tracking cookies to understand how you interact with it. The latter will be set only after consent.
Read our Privacy Policy

Assessing GenAI Security Solutions in the Wild with PINT

5:00 pm
May 2, 2024
Václav Volhejn
Senior Applied ML Scientist at Lakera
Julia Bazińska
ML Engineer at Lakera
May 2, 2024

Join Václav Volhejn (Senior ML Scientist at Lakera) and Julia Bazińska (ML Engineer at Lakera) for a live webinar discussion exploring the newly released PINT (Prompt Injection Test) benchmark.

Goodhart’s law states that when a measure becomes a target, it ceases to be a good measure. This is an acute issue in the prompt injection defense field where all public benchmarks have been overfit on to the point of becoming uninformative: having a 99% accuracy on a Hugging Face prompt injection dataset does not translate into being an effective defense in practice.

Our new PINT benchmark was designed to combat this issue, and includes a mechanism to create updated versions of the dataset if it too starts being affected by Goodhart’s law.

It was designed to provide a neutral way to evaluate the performance of prompt defense solutions, without relying on known public datasets. It is a critical piece to ensure that prompt defense solutions can be evaluated objectively and translate to operational attacks (rather than being optimized purely to perform on public benchmarks).

This webinar is a must-attend for anyone looking to learn more about how to benchmark AI security solutions and how to contribute to a more secure GenAI future.

  • Learn the origin story of PINT: Discover why we developed a new benchmark for evaluating prompt injection detection systems and why it's open-source.
  • Understand what PINT is & how to use it: Gain insight into its design, methodology, and how to effectively use it for evaluating prompt injection detection systems.
  • Explore PINT use cases: See the examples of models evaluated using PINT.
  • Learn how you can contribute to PINT: Find out how you can play a role in the ongoing development of PINT and contribute to a more secure future for Generative AI.
Václav Volhejn
Senior Applied ML Scientist at Lakera

Václav Volhejn is a Senior Applied ML Scientist at Lakera. In 2023 he designed the initial version of Gandalf and now works on improving Lakera's prompt injection detector.

Read more
Julia Bazińska
ML Engineer at Lakera

Julia Bazińska is a Machine Learning Engineer at Lakera AI, developing their core product - Lakera Guard.

Read more


untouchable mode.
Get started for free.

Lakera Guard protects your LLM applications from cybersecurity risks with a single line of code. Get started in minutes. Become stronger every day.

Join our Slack Community.

Several people are typing about AI/ML security. 
Come join us and 1000+ others in a chat that’s thoroughly SFW.