FWF Translational Research TRP 109-N23 - Department of Computational Perception

Department of

Computational Perception

Home – Mission – Teaching – People – Research – Media – Impressum

COMPUTATIONAL MUSIC PERFORMANCE RESEARCH, APPLIED

Project Title: Computational Music Performance Research, Applied

Sponsor: Austrian National Science Fund (Fonds zur Förderung der wissenschaftlichen Forschung, FWF)

Project Number: TRP 109-N23

Duration: 36 months (July 2010 - June 2013)

Abstract

This project builds on many years of basic research where we investigated various aspects of expressive music performance with novel computational methods, and developed predictive computer models of performance dimensions such as expressive timing, dynamics, and articulation. In the process, we not only made interesting discoveries about this complex art, but also developed a lot of new computational music processing technology (e.g., algorithms for beat and tempo tracking, audio-to-audio synchronisation, performance visualisation, or performance prediction, to name but a few). The goal of the present project is to build on these methods and develop them to the point where they may become efficient and robust enough for real-world applications.

The research to be performed in this project is motivated by three visionary application scenarios:
1. Multimedia Arts: Real-time visualisation of musical works on stage
2. Synthetic Musical ‘Expression’: Naturally expressive synthetic instruments and virtual orchestras
3. Automatic Feedback on Expressive Performance: Interactive tools for music performance analysis and teaching.

From an analysis of the requirements that such applications would pose, we derive the following three main research goals:

1. Expression Extraction: computer methods for automatically and precisely extracting details of expressive performance (i.e., tempo, timing, dynamics, etc.) from audio recordings, possibly in real time; this also involves methods for the precise alignment of audio recordings to musical scores.

2. Expression Tracking: algorithms for following live performance (audio streams) and aligning them to given scores or reference audio recordings in real time; these methods should also work with incomplete scores or recordings and be extremely robust and reactive in the face of expressive deviations, unforeseen disruptions, errors, etc. A sub-problem here is the development of methods for inducing robust predictive tempo models on-line.

3. Expression Rendering: computational models of specific dimensions of expressive performance that permit the automatic generation of synthetic music performances that sound reasonably musical and ‘natural’; possibly also methods for interactively modifying or controlling such performances.

A number of potential application partners – e.g., the Ars Electronica Center and the Vienna Symphonic Library – are supporting this work (e.g., by supplying test data and audio materials).

Publications

Grachten, M. and Krebs. F. (2014)
An Assessment of Learned Score Features for Modeling Expressive Dynamics in Music. IEEE Transactions on Multimedia.
Pre-print.

Collins, T., Böck, S., Krebs, F., and Widmer, G. (2014).
Bridging the Audio-Symbolic Gap: The Discovery of Repeated Note Content Directly from Polyphonic Music Audio. In Proceedings of the 53rd AES Conference on Semantic Audio, Audio Engineering Society, London, Jan. 2014.
Best Paper Award.

Arzt, A., Böck, S., Flossmann, S., Frostel, H., Gasser, M. and Widmer, G. (2014).
The Complete Classical Music Companion v0.9. Demo paper, 53rd AES Conference on Semantic Audio, Audio Engineering Society, London, Jan. 2014.
Best Demonstration Award.

Widmer, G. (2014).
What Really Moves Us in Music: Expressivity as a Challenge to Semantic Audio Research. Proceedings of the 53rd AES Conference on Semantic Audio, Audio Engineering Society, London, Jan. 2014.
Pre-print.

Collins, T., Arzt, A., Flossmann, S., and Widmer, G.(2013).
SIARCT-CFP: Improving Precision and the Discovery of Inexact Musical Patterns in Point-Set Representations. In Proceedings of the 14th International Society for Music Information Retrieval Conference (ISMIR 2013), pp. 549-554, Curitiba, Brazil.

Korzeniowski, F., Krebs, F., Arzt, A. and Widmer, G. (2013).
Tracking Rests and Tempo Changes: Improved Score Following with Particle Filters. In Proceedings of the International Computer Music Conference (ICMC), Perth, Australia.

Grachten, M. and Widmer, G. (2012).
Linear Basis Models For Prediction And Analysis of Musical Expression. Journal of New Music Research Vol. 41 (4), pp.311-322.
Pre-print

Arzt, A., Widmer, G. and Dixon, S. (2012).
Adaptive Distance Normalization for Real-Time Music Tracking. In Proceedings of the 20th European Signal Processing Conference (Eusipco 2012), Bucharest, Romania.

Arzt, A., Widmer, G. Böck, S., Sonnleitner, R. and Frostel, H. (2012).
Towards a Complete Classical Music Companion. In Proceedings of the 20th European Conference on Artificial Intelligence (ECAI 2012), Montpellier, France.

Krebs, F. and Grachten, M. (2012).
Combining Score and Filter Based Models to Predict Tempo Fluctuations in Expressive Music Performances. In Proceedings of the 9th Sound and Music Computing Conference (SMC 2012), Copenhagen, Denmark.

Flossmann, S., Grachten, M. and Widmer, G. (2012)
Expressive Performance Rendering with Probabilistic Models, In Guide to Computing for Expressive Music Performance, Springer, 2012.
Pre-print

Flossmann, S. (2012)
Expressive Performance Rendering with Probabilistic Models - Creating, Analyzing, and Using the Magaloff Corpus. PhD Thesis, Dept. of Computational Perception, Johannes Kepler University Linz.

Niedermayer, B. (2012)
Accurate Audio-to-Score Alignment - Data Acquisition in the Context of Computational Musicology. PhD Thesis, Dept. of Computational Perception, Johannes Kepler University Linz.

Niedermayer, B., Böck, S., and Widmer, G. (2011).
On the Importance of "Real" Audio Data for MIR Algorithm Evaluation at the Note-Leve - A comparative Study. In Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR 2011), Miami, Florida, USA.

Flossmann, S., and Widmer, G. (2011).
Toward a Multilevel Model of Expressive Piano Performance. In Proceedings of the International Symposium on Performance Science (ISPS 2011), Toronto, Canada.

Flossmann, S., and Widmer, G. (2011).
Toward a Model of Performance Errors: A qualitative Review of Magaloff´s Chopin. In Proceedings of the International Symposium on Performance Science (ISPS 2011), Toronto, Canada.

Grachten, M. and Widmer, G. (2011).
A Method to Determine the Contribution of Annotated Performance Directives in Music Performances. In Proceedings of the International Symposium on Performance Science (ISPS 2011), Toronto, Canada.

Grachten, M. and Widmer G. (2011).
Explaining Expressive Dynamics As a Mixture of Basis Functions. In Proceedings of the Eighth Sound and Music Computing Conference (SMC 2011), Padua, Italy.

Niedermayer, B., Widmer, G., and C. Reuter (2011).
Version Detection for Historical Musical Automata. In Proceedings of the 8th Sound and Music Computing Conference (SMC 2011), Padova, Italy.

Frostel, H. Arzt, A. and Widmer, G. (2011).
The Vowel Worm: Real-Time Mapping and Visualisation of Sung Vowels in Music. In Proceedings of the 8th Sound and Music Computing Conference (SMC 2011), Padova, Italy.

Flossmann, S., Grachten, M. and Widmer, G. (2011).
Expressive Performance with Bayesian Networks and Linear Basis Models. Extended abstract, Rencon Workshop 2011: Musical Performance Rendering competition for Computer Systems.

Ritter, A., Grachten, M. and Widmer, G. (2011).
Macht Musizieren Gesund? Zur Herzrate und deren Variabilität während Mozarts Klavierkonzert Nr. 14. In 27. Jahrestagung der Deutschen Gesellschaft für Musikpsychologie),

Korzeniowski, F. (2011).
Real-time Capable Singer Tracking Using Pitch and Lyrics Information. Master Thesis, Dept. of Computational Perception, Johannes Kepler University Linz.

Arzt, A. and Widmer, G. (2010).
Towards Effective 'Any-Time' Music Tracking. In Proceedings of the Starting AI Researchers' Symposium (STAIRS 2010), Lisbon,Portugal

Arzt, A. and Widmer, G. (2010).
Robust Real-time Music Tracking. In Proceedings of the 2nd Vienna Talk on Musical Acoustics (VITA 2010), Vienna, Austria

Arzt, A. and Widmer, G.(2010).
Simple Tempo Models for Real-time Music Tracking. In Proceedings of the 7th Sound and Music Computing Conference (SMC 2010), Barcelona, Spain

Niedermayer, B. and Widmer, G. (2010).
Strategies towards the Automatic Annotation of Classical Piano Music. In Proceedings of the 7th Sound and Music Computing Conference (SMC 2010), Barcelona, Spain

Niedermayer, B. and Widmer, G. (2010).
A Multi-Pass Algorithm for Accurate Audio-to-Score Alignment. In Proceedings of the 11th International Society for Music Information Retrieval Conference (ISMIR 2010), Utrecht, The Netherlands.

last edited by gw on 2014-04-10