Deep Probabilistic Priors for Solving Inverse Problems

by Paweł Pierzchlewicz
Doctoral thesis
Date of Examination:2024-11-27
Date of issue:2025-03-20
Advisor:Fabian H. Sinz
Referee:Bernhard Schmitzer
Referee:Alexander Ecker
Referee:Prof. Dr. Bela Gipp
Referee:Lisa Beinborn
Referee:Constantin Pape
Persistent Address: http://dx.doi.org/10.53846/goediss-11055

 

 

Files in this item

Name:PhD_Thesis_Pierzchlewicz.pdf
Size:42.4Mb
Format:PDF

The following license files are associated with this item:

Abstract

English

Inverse problems are problems where causal sources are inferred from observed effects. They are ubiquitous in science and engineering. Examples include image reconstruction from neural responses, feature visualizations of a deep neural network and 3D pose estimation from 2D images. These problems are inherently ill-posed, often having multiple solutions for a given set of observations. The standard approach to solve inverse problems is to use simplified hand-crafted priors, which struggle with high-dimensional data that lies on very constrained manifolds such as images or human poses. This thesis studies the application of deep probabilistic priors to solve inverse problems more effectively. I explore the idea that deep probabilistic priors, such as Normalizing Flows and Diffusion Models, could improve the quality and robustness of solutions compared to traditional methods. I tested this hypothesis in two domains: vision neuroscience and human pose estimation. In the domain of neuroscience, I showed that using a diffusion model helps instill a better natural image prior, thus improving architecture generalizability in the tasks of synthesizing most exciting inputs for individual neurons and reconstructing images from neural responses. I compared the deep probabilistic prior to previous state-of-the-art handcrafted regularization methods on each of these tasks. In the domain of human pose estimation, I first showed that existing multi-hypothesis pose estimation methods are miscalibrated. By using a deep probabilistic prior in the form of a normalizing flow and then improving upon it with a diffusion model, I showed that the issue of calibration can be mitigated when optimizing for likelihood based objectives rather than sample based metrics. Furthermore, I explored the zero-shot capabilities of this approach, showing that with a deep probabilistic prior we can zero-shot infer new human motions. Through a series of experiments and case studies, I demonstrate that deep probabilistic priors for inverse problems outperform hand-crafted regularization techniques in terms of accuracy, robustness, and calibration. My findings suggest that deep probabilistic priors offer a promising avenue for tackling complex inverse problems across various domains.
Keywords: Pose Estimation; Deep Probabilistic Priors; Diffusion Models; Vision Neuroscience; Inverse Problems
 
Statistik