FWF Translational Research TRP 109-N23 - Department of Computational Perception

Department of

Computational Perception

Home – Mission – Teaching – People – Research – Media – Impressum

COMPUTATIONAL MUSIC PERFORMANCE RESEARCH, APPLIED

Project Title: Computational Music Performance Research, Applied

Sponsor: Austrian National Science Fund (Fonds zur Förderung der wissenschaftlichen Forschung, FWF)

Project Number: TRP 109-N23

Duration: 36 months (July 2010 - June 2013)

Abstract

This project builds on many years of basic research where we investigated various aspects of expressive music performance with novel computational methods, and developed predictive computer models of performance dimensions such as expressive timing, dynamics, and articulation. In the process, we not only made interesting discoveries about this complex art, but also developed a lot of new computational music processing technology (e.g., algorithms for beat and tempo tracking, audio-to-audio synchronisation, performance visualisation, or performance prediction, to name but a few). The goal of the present project is to build on these methods and develop them to the point where they may become efficient and robust enough for real-world applications.

The research to be performed in this project is motivated by three visionary application scenarios:
1. Multimedia Arts: Real-time visualisation of music-dramatic works on stage (as realised, in particular, by the Ars Electronica Center / AEC Futurelab, Linz).
2. Synthetic Musical ‘Expression’: Naturally expressive synthetic instruments and virtual orchestras.
3. Automatic Feedback on Expressive Performance: Interactive tools for music performance teaching and analysis.

From an analysis of the requirements that such applications would pose, we derive the following three main research goals:

1. Expression Extraction: computer methods for automatically and precisely extracting details of expressive performance (i.e., tempo, timing, dynamics, etc.) from audio recordings, possibly in real time; this also involves methods for the precise alignment of audio recordings to musical scores.

2. Expression Tracking: algorithms for following live performance (audio streams) and aligning them to given scores or reference audio recordings in real time; these methods should also work with incomplete scores or recordings and be extremely robust and reactive in the face of expressive deviations, unforeseen disruptions, errors, etc. A sub-problem here is the development of methods for inducing robust predictive tempo models on-line.

3. Expression Rendering: computational models of specific dimensions of expressive performance that permit the automatic generation of synthetic music performances that sound reasonably musical and ‘natural’; possibly also methods for interactively modifying or controlling such performances.

A number of potential application partners – e.g., the Ars Electronica Center and the Vienna Symphonic Library – are supporting this work (e.g., by supplying test data and audio materials).

Publications

Flossmann, S. (2012)
Expressive Performance Rendering with Probabilistic Models - Creating, Analyzing, and Using the Magaloff Corpus, Dissertation.

Niedermayer, B. (2012)
Accurate Audio-to-Score Alignment - Data Acquisition in the Context of Computational Musicology, Dissertation.

Niedermayer, B., Böck, S., and Widmer, G. (2011).
On the Importance of "Real" Audio Data for MIR Algorithm Evaluation at the Note-Leve - A comparative Study. In Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR 2011), Miami, Florida, USA.

Flossmann, S., and Widmer, G. (2011).
Toward a Multilevel Model of Expressive Piano Performance. In Proceedings of the International Symposium on Performance Science (ISPS 2011), Toronto, Canada.

Flossmann, S., and Widmer, G. (2011).
Toward a Model of Performance Errors: A qualitative Review of Magaloff´s Chopin. In Proceedings of the International Symposium on Performance Science (ISPS 2011), Toronto, Canada.

Grachten, M. and Widmer, G. (2011).
A Method to Determine the Contribution of Annotated Performance Directives in Music Performances. In Proceedings of the International Symposium on Performance Science (ISPS 2011), Toronto, Canada.

Grachten, M. and Widmer G. (2011).
Explaining Expressive Dynamics As a Mixture of Basis Functions. In Proceedings of the Eighth Sound and Music Computing Conference (SMC 2011), Padua, Italy.

Niedermayer, B., Widmer, G., and C. Reuter (2011).
Version Detection for Historical Musical Automata. In Proceedings of the 8th Sound and Music Computing Conference (SMC 2011), Padova, Italy.

Frostel, H. Arzt, A. and Widmer, G. (2011).
The Vowel Worm: Real-Time Mapping and Visualisation of Sung Vowels in Music. In Proceedings of the 8th Sound and Music Computing Conference (SMC 2011), Padova, Italy.

Flossmann, S., Grachten, M. and Widmer, G. (2011).
Expressive Performance with Bayesian Networks and Linear Basis Models. Extended abstract, Rencon Workshop 2011: Musical Performance Rendering competition for Computer Systems.

Ritter, A., Grachten, M. and Widmer, G. (2011).
Macht Musizieren Gesund? Zur Herzrate und Dessen Variabilität Während Mozarts Klavierkonzert Nr. 14. In 27. Jahrestagung der Deutschen Gesellschaft für Musikpsychologie),

Flossmann, S., Goebl, W., Grachten, M., Niedermayer, B. and Widmer, G. (2010).
The Magaloff Project: An Interim Report. Journal of New Music Research, 39 (4), 363-377

Goebl, W., Flossmann, S., and Widmer, G. (2010).
Investigations into between-hand synchronization in Magaloff’s Chopin. Computer Music Journal (in press)

Arzt, A. and Widmer, G. (2010).
Towards Effective 'Any-Time' Music Tracking. In Proceedings of the Starting AI Researchers' Symposium (STAIRS 2010), Lisbon,Portugal

Arzt, A. and Widmer, G. (2010).
Robust Real-time Music Tracking. In Proceedings of the 2nd Vienna Talk on Musical Acoustics (VITA 2010), Vienna, Austria

Arzt, A. and Widmer, G.(2010).
Simple Tempo Models for Real-time Music Tracking. In Proceedings of the 7th Sound and Music Computing Conference (SMC 2010), Barcelona, Spain

Niedermayer, B. and Widmer, G. (2010).
Strategies towards the Automatic Annotation of Classical Piano Music. In Proceedings of the 7th Sound and Music Computing Conference (SMC 2010), Barcelona, Spain

Molina, M., Grachten, M. and Widmer, G. (2010).
Evidence for Pianist-specific Rubato Style in Chopin Nocturnes. In Proceedings of the 11th International Society for Music Information Retrieval Conference (ISMIR 2010), Utrecht, The Netherlands.

Niedermayer, B. and Widmer, G. (2010).
A Multi-Pass Algorithm for Accurate Audio-to-Score Alignment. In Proceedings of the 11th International Society for Music Information Retrieval Conference (ISMIR 2010), Utrecht, The Netherlands.

Flossmann, S. Goebl, W. and Widmer, G.(2010).
The Magaloff Corpus: An Empirical Error Study. In Proceedings of the 11th International Conference on Music Perception and Cognition (ICMPC), Seattle, WA, USA

last edited by sf / bn on 2012-04-26