Videos

Reinhard Heckel - The role of data and models for deep-learning based image reconstruction

Presenter
December 1, 2022
Abstract
Recorded 01 December 2022. Reinhard Heckel of the Technical University of Munich presents "The role of data and models for deep-learning based image reconstruction" at IPAM's Multi-Modal Imaging with Deep Learning and Modeling Workshop. Abstract: Deep-learning methods give state-of-the-art performance for a variety of imaging tasks, including accelerated magnetic resonance imaging. In this talk we discuss whether improved models and algorithms or training data are the most promising way forward. First, we ask whether increasing the model size and the training data improves performance in a similar fashion as it has in domains such as language modeling. We find that scaling beyond relatively few examples yields only marginal performance gains. Second, we discuss the robustness of deep learning based image reconstruction methods. Perhaps surprisingly, we find no evidence for neural networks being any less robust than classical reconstruction methods (such as l1 minimization). However, we find that both classical and deep learning based approaches perform significantly worse under distribution shifts, i.e., when trained (or tuned) and tested on slightly different data. Finally, we show that the out-of-distribution performance can be improved through more diverse training data, or through an algorithmic intervention called test-time-training. Learn more online at: http://www.ipam.ucla.edu/programs/workshops/workshop-iv-multi-modal-imaging-with-deep-learning-and-modeling/?tab=overview