Advancing information theory for distributed computation - with applications to biological networks and Artificial Intelligence

by David Alexander Ehrlich
Cumulative thesis
Date of Examination:2025-08-25
Date of issue:2025-10-15
Advisor:Prof. Dr. Michael Wibral
Referee:Prof. Dr. Michael Wibral
Referee:Prof. Dr. Fred Wolf
Persistent Address: http://dx.doi.org/10.53846/goediss-11562

 

 

Files in this item

Name:PhD_Thesis_David_Alexander_Ehrlich_corrected...pdf
Size:18.8Mb
Format:PDF

The following license files are associated with this item:

Abstract

English

Biological brains and artificial neural networks alike derive their impressive information processing abilities from the complex interplay of individual neurons. While the function of an individual neuron is comparatively simple to understand, how they organize into distributed computation systems capable of solving difficult tasks remains mostly elusive. Understanding this organization requires an interpretable and implementation-independent framework for information processing. Such a framework can be found in Information Theory, which quantifies how well one variable can be predicted from another. In distributed computation, however, the more relevant question is how one variable can be predicted from multiple sources, e.g., how information from individual neurons is integrated in a neuron’s activity. Disentangling the different ways in which these sources can contribute to the overall prediction in unique, redundant and synergistic ways is the subject of Partial Information Decomposition (PID). Through this decomposition, PID provides a powerful tool to understand how information is represented and transformed in distributed computation systems. This thesis advances information theory and PID for analyzing and constructing distributed computation systems. Key contributions include the definition of a novel analytical definition and estimator for a continuous PID, computationally efficient and interpretable PID-based summary structures revealing that representations in deep neural networks become more accessible and robust with training and throughout the network architecture, and the introduction of novel locally-learning neurons optimizing an abstract PID-based goal function. Furthermore, the thesis introduces a two-step framework for identifying spatial information flows through biological tissue, marking a first step towards understanding spatial computation in living tissue. Collectively, these efforts provide novel concepts, tools and insights for understanding and constructing distributed computation systems.
Keywords: information theory; partial information decomposition; distributed computation; machine learning; artificial intelligence; synergy; redundancy
 
Statistik