Breaking into a Deep Learning box
Presenter
April 10, 2021
Abstract
Recent decade brought explosive progress in the applications of Machine Learning and data-driven Artificial Intelligence (AI) to real-life problems across sectors. Autonomous cars and automated passport control are examples of the new reality. Deep Learning models, or more generally, models with multiple learnable processing stages constitute a large class of models to which a significant part of the recent successes has been apportioned. Notwithstanding these successes, there are emerging challenges too.
In this talk we will discuss a set of vulnerabilities which may typically arise in large Deep Learning models. These vulnerabilities are extreme sensitivities of the models to data or structure perturbations. We will present a formal theoretical framework for assessing and analysing two classes of such vulnerabilities. The first class is linked with adversarial examples. Vulnerabilities of the second class are linked with purposeful malicious structure perturbations which may be, with high probability, undetectable through input-output validation. We name these perturbations “stealth attacks”. We will show how to construct stealth attacks on Deep Learning models that are hard to spot unless the validation set is made exponentially large.
For both classes of attacks, the high dimensionality of the AI’s decision-making space appears to be a major contributor to the AI’s vulnerability. We conclude with recommendations of how robustness to malicious perturbations of data and structure can be mitigated by ensuring that the data dimensionality at relevant processing stages in Deep Learning models is kept sufficiently small.