The Reliability of Deep Learning for Signal and Image Processing: Interpretability, Robustness, and Accuracy

Abstract

This thesis investigates several aspects of using data-driven methods for image and signal processing tasks, particularly those aspects related to the reliability of approaches based on deep learning. It is organized in two parts. The first part studies the interpretability of predictions made by neural network classifiers. A key component for achieving interpretable classifications is the identification of relevant input features for the predictions. While several heuristic approaches towards this goal have been proposed, there is yet no generally agreed-upon definition of relevance. Instead, these heuristics typically rely on individual (often not explicitly stated) notions of interpretability, making comparisons of results difficult. The contribution of the first part of this thesis is the introduction of an explicit definition of relevance of input features for a classifier prediction and an analysis thereof. The formulation is based on a rate-distortion trade-off and derived from the observation and identification of common questions that practitioners would like to answer with relevance attribution methods. It turns out that answering these questions is extremely challenging: A computational complexity analysis reveals the hardness of determining the most relevant input features (even approximately) for Boolean classifiers as well as for neural network classifiers. This hardness in principle justifies the adoption of heuristic strategies and the explicit rate-distortion formulation inspires a novel approach that specifically aims at answering the identified questions of interest. Furthermore, it allows for a quantitative evaluation of relevance attribution methods, revealing that the newly proposed heuristic performs best in identifying the relevant input features compared to previous methods. The second part studies the accuracy and robustness of deep learning methods for the reconstruction of signals from undersampled indirect measurements. Such inverse problems arise for example in medical imaging, geophysics, communication, or astronomy. While widely used classical variational solution methods come with reconstruction guarantees (under suitable assumptions), the underlying mechanisms of data-driven methods are mostly not well understood from a mathematical perspective. Nevertheless, they show promising results and frequently empirically outperform classical methods in terms of reconstruction quality and speed. However, several doubts remain regarding their reliability, in particular questions concerning their robustness to perturbations. Indeed, for classification tasks it is well known that neural networks are vulnerable to adversarial perturbations, i.e., tiny modifications that are visually imperceptible but mislead the neural network to make a wrong prediction. This raises the question if similar effects also occur in the context of signal recovery. The contribution of the second part of this thesis is an extensive numerical study of the robustness of a representative selection of end-to-end neural networks for solving inverse problems. It is demonstrated that for such regression problems (in contrast to classification) neural networks can be remarkably robust to adversarial and statistical perturbations. Furthermore, they show state-of-the-art performance resulting in highly accurate reconstructions: In the idealistic scenario of synthetic and perturbation-free data neural networks have the potential to achieve near-perfect reconstructions, i.e., their reconstruction error is close to numerical precision.

Jan Macdonald

My research is at the interface of applied and computational mathematics and scientific machine learning. I am interested in inverse problems, signal- and image recovery, and robust and interpretable deep learning.