Adversarial Machine Learning: Lessons I Learned over 20 Years and Challenges I See Ahead

Modern AI systems can achieve remarkable levels of accuracy, yet high accuracy does not necessarily imply robustness. The discovery of adversarial examples made this limitation strikingly visible: small, carefully crafted perturbations can lead highly accurate machine-learning models to fail in unexpected ways, challenging assumptions that are commonly made in statistical machine learning. In this talk, I will revisit some basic ideas of adversarial machine learning and use them as a starting point to reflect on lessons I have learned over more than twenty years of research in the field. Rather than providing a tutorial or a comprehensive survey, I will focus on a few principles that I believe remain central to understanding the security and robustness of modern AI systems, and on some challenges that I see ahead. One such challenge is performance regression in model updates: a new model may improve average accuracy or robustness while performing worse than its predecessor on specific inputs. I will discuss both accuracy and robustness regression and some approaches we have developed to mitigate these effects. I will also briefly outline a broader, still open challenge: turning AI security and trustworthiness requirements into technically grounded and repeatable verification procedures and evidence, including evidence that may support conformity assessment against forthcoming harmonised standards under the European AI Act. This is one of the research directions pursued within InfoAIcert, our flagship project on conformity assessment of AI systems.

FABIO ROLI

In collaboration with:

Joining us in person offers the opportunity for direct networking and interaction with both speakers and participants.
Please complete the form below to reserve your seat.