Data Analysis of Musical Timeseries
by Corentin Nelias
Date of Examination:2022-10-24
Date of issue:2024-01-08
Advisor:Prof. Dr. Theo Geisel
Referee:Prof. Dr. Stefan Klumpp
Referee:Prof. Dr. Viola Priesemann
Referee:Prof. Dr. Florentin Wörgötter
Referee:Prof. Dr. Ulrich Parlitz
Referee:Prof. Dr. Michael Wilczek
Files in this item
Name:PhD_thesis_corrected.pdf
Size:62.4Mb
Format:PDF
Abstract
English
Music, despite its ubiquitous familiarity, is surprisingly hard to define. Should one consider its acoustic features, the mechanics by which it is produced and performed, as well as the cultural norms attached to them, or the various cognitive responses it elicits among listeners? These, and many other interrogations are part of ongoing discussions among authorities. The present work deals with the different dynamical processes that embody musical compositions and performances such as the changes of pitches in time, the loudness fluctuations or the variations of rhythm and timing. In other words, it asks a question about a seldom studied aspect: the stochastic properties of music. What are the statistical characteristics of musical sequences, and what can we learn from them? This thesis is organized around three main projects. The first one studies the existence and effects of minute rhythmic deviations in jazz performances, the second one is concerned with the spectral properties of musical sequences and the last one considers the categorical nature of key variables from such sequences. The first project arose with the realization that, notwithstanding a clear consensus concerning their existence, the effects of microtiming deviations on listeners were still unagreed upon. Microtiming deviations represent tiny deviations with respect to the intended timing of a note. These timing fluctuations are an inherent part of human performances. Yet, despite having been known for more than 30 years, the literature almost evenly split on whether or not these deviations positively impact music listening. By analyzing a large database of accurate jazz solos transcriptions, we were able to uncover a new type of systematic microtiming deviation we call downbeat delays. In a second part of the project, we conducted an online survey to investigate the effects of such downbeat delays on listeners. We were able to show that, under a certain synchronicity condition, such delays positively impact the feeling of swing among listeners. The results of this project have direct implications for music production, and can serve as a basis for further investigations: what other kinds of systematic microtiming deviations are key features of musical performances? In the second project, we look at the correlation structures present in musical pieces by analyzing pitch and loudness sequences. For this task, the power-spectral density represents a very useful tool that has, in fact, already been used in the past to this very aim. Indeed, several articles reported long-range correlations in musical ii sequences. However, the exact nature and shape of these correlations remained unclear as the existing reports provide conflicting results. Moreover, the universality of the aforementioned correlations has not been established since past studies were mostly constrained to classical music scores. In the present work, we carried out an analysis of a large corpus of written as well as improvised music, and were able to reveal a characteristic behavior in the power-spectrum of pitch time-series that we describe as power-law + plateau. The description of this structure allows to understand and unify the differences in the existing literature into a coherent picture. This shape does really seem to be an underlying characteristic of musical pitch sequences, as we observe it for all composers, epoch and sub-genres that we analyzed. The last project was motivated by the fact that the vast majority of studies on musical sequences (including the ones presented in the two previous projects) do not consider the categorical nature of pitches or intervals. The exact same note played over two different harmonic backgrounds can sound completely different. For this reason, a description in terms of note functionality would seem more appropriate, but the mapping to said functionality can be ambiguous and subject to interpretation. In order to find optimal mappings and perform analysis accounting for the function of pitches and intervals, I applied tools such as the information bottleneck or the spectral envelope. I show that the use of such categorical timeseries analysis tools can help with motif identification and interesting symmetries in the compositions and improvisations of different artists. Finally, in appendix I show the early results of a project seeking to estimate the entropy profile -amount of information contained in each note as a function of time- with the use of artificial neural networks. The aim is to help provide an answer to the established hypothesis that the emotional response to music is linked to expectation and surprise. The network is able to generate accurate predictions and identify "surprising" features, or "expected" structures. It could be used in an appropriately designed experiment comparing participants’ responses to the generated entropy profile to learn more about the link between expectations and emotions.
Keywords: complex systems; deep learning; music; timeseries analysis