Department of Computational Perception

Performances as Highly Enriched aNd Interactive Concert eXperiences (PHENICX)

Sponsor: European Commission

Call identifier: FP7-ICT-2011-9

Project Number: 601166

Duration: 36 months (Feb 2013 – Jan 2016)

Abstract

Modern digital multimedia and internet technology have radically changed the ways people find entertainment and discover new interests online, seemingly without any physical or social barriers. Such new access paradigms are in sharp contrast with the traditional means of entertainment. An illustrative example of this is live music concert performances that are largely being attended by dedicated audiences only. The PHENICX project aims at bridging the gap between the online and offline entertainment worlds. It will make use of the state-of-the-art digital multimedia and internet technology to make the traditional concert experiences rich and universally accessible: concerts will become multimodal, multi-perspective and multilayer digital artefacts that can be easily explored, customized, personalized, (re)enjoyed and shared among the users. The main goal is twofold: (a) to make live concerts appealing to potential new audience and (b) to maximize the quality of concert experience for everyone. Scientific objectives of PHENICX are (i) to generate a reliable and effective set of techniques for multimodal enrichment of live music concert recordings suitable for implementation and deployment in real-world situations, and (ii) finding ways to offer the resulting multi-faceted digital artefacts as engaging digital experiences to a wide range of users. The project will establish a methodological framework to map these scientific objectives onto a solid implementation platform that will be developed incrementally and tested in real-life use settings. PHENICX will mainly focus on classical music - a European heritage asset that suffers heavily from an image of inaccessibility to outsider audiences. However, findings from PHENICX will be relevant to live concert situations in any genre. With an innovative technology partner, as well as two authoritative professional music stakeholders in the consortium, the project has strong immediate impact and dissemination potential.

Partners

• Royal Concertgebouw Orchestra Amsterdam, the Netherlands
• Universitat Pompeu Fabra, Barcelona, Spain
• Technische Universiteit Delft, the Netherlands
• Johannes Kepler Universität Linz, Austria
• Austrian Research Institute for Artificial Intelligence, Vienna, Austria
• Video Dock b.v., Amsterdam, Netherlands
• Fundacio privada per a l'escola superior de musica de catalunya, Barcelona, Spain

Publications

Journals

Music Information Retrieval: Recent Developments and Applications Schedl, M., Gómez, E., and Urbano, J. Foundations and Trends in Information Retrieval, volume 8, number 2-3, pp. 127-261, 2014.

The Neglected User in Music Information Retrieval Research Schedl, M., Flexer, A., and Urbano, J. Journal of Intelligent Information Systems, 2013.

Harvesting Microblogs for Contextual Music Similarity Estimation - A Co-occurrence-based Framework Schedl, M., Hauger, D., and Urbano J. Multimedia Systems, 2013.

Peer-Reviewed Conference and Workshop Proceedings

Using Instagram Picture Features to Predict Users' Personality Ferwerda, B., Schedl, M., and Tkalčič, M. Proceedings of the 22nd International Conference on MultiMedia Modeling (MMM 2016), Miami, USA, January 2016.

Towards Personalizing Classical Music Recommendations Schedl, M. Proceedings of the 2nd International Workshop on Social Media Retrieval and Analysis (SoMeRA 2015), Atlantic City, USA, November 2015.

Improving Music Recommendations with a Weighted Factorization of the Tagging Activity Vall, A., Skowron, M., Knees, P., and Schedl, M. Proceedings of the 16th International Society for Music Information Retrieval Conference (ISMIR 2015), Malaga, Spain, October 2015.

I-Vectors for Timbre-Based Music Similarity and Classification Eghbal-zadeh, H., Lehner, B., Schedl, M., and Widmer, G. Proceedings of the 16th International Society for Music Information Retrieval Conference (ISMIR 2015), Malaga, Spain, October 2015.

Listener-aware Music Search and Recommendation Schedl, M. Proceedings of the 6th International Workshop on Mining Ubiquitous and Social Environments (MUSE 2015), Porto, Portugal, September 2015.

Listener-aware Music Recommendation from Sensor and Social Media Data Schedl, M. Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2015), Porto, Portugal, September 2015.

Timbral Modeling for Music Artist Recognition Using I-vectors Eghbal-zadeh, H., Schedl, M., and Widmer, G. Proceedings of the 23rd European Signal Processing Conference (EUSIPCO 2015), Nice, France, August-September 2015.

PHENICX: Innovating the Classical Music Experience Liem, C.C.S., Gómez, E., and Schedl, M. Proceedings of the IEEE International Conference on Multimedia and Expo (ICME 2015), Torino, Italy, June-July 2015.

Tailoring Music Recommendations to Users by Considering Diversity, Mainstreaminess, and Novelty Schedl, M., and Hauger, D. Proceedings of the 38th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2015), Santiago, Chile, August 2015.

Personality & Emotional States: Understanding Users' Music Listening Needs Ferwerda, B., Schedl, M., and Tkalčič, M. Extended Proceedings of the 22nd International Conference on User Modeling, Adaptation and Personalization (UMAP 2015), Dublin, Ireland, June-July 2015.

Personality Correlates for Digital Concert Program Notes Tkalčič, M., Ferwerda, B., Hauger, D., and Schedl, M. Proceedings of the 22nd International Conference on User Modeling, Adaptation and Personalization (UMAP 2015), Dublin, Ireland, June-July 2015.

Personality Traits Predict Music Taxonomy Preferences Ferwerda, B., Yang, E., Schedl, M., and Tkalčič, M. ACM CHI '15 Extended Abstracts on Human Factors in Computing Systems (CHI 2015), Seoul, Republic of Korea, April 2015.

On the Influence of User Characteristics on Music Recommendation Schedl, M., Hauger, D., Farrahi, K., and Tkalčič, M. Proceedings of the Proceedings of the 37th European Conference on Information Retrieval (ECIR 2015), Vienna, Austria, March-April 2015.

Iron Maiden while jogging, Debussy for dinner? - An analysis of music listening behavior in context Gillhofer, M. and Schedl, M. Proceedings of the 21th International Conference on MultiMedia Modeling (MMM 2015), Sydney, Australia, January 2015.

Genre-based Analysis of Social Media Data on Music Listening Behavior Schedl, M. and Tkalčič, M. Proceedings of the 1st ACM International Workshop on Internet-Scale Multimedia Management (ISMM 2014), Orlando, USA, Novermber 2014.

PatternViewer: An Application for Exploring Repetitive and Tonal Structure Nikrang, A., Collins, T., and Widmer, G. Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR 2014), Taipei, Taiwan, October 2014.

Impact of Listening Behavior on Music Recommendation Farrahi, K., Schedl, M., Vall, A., Hauger, D., and Tkalčič, M. Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR 2014), Taipei, Taiwan, October 2014.

Enhancing Music Recommender Systems with Personality Information and Emotional States: A Proposal Ferwerda, B. and Schedl, M. Proceedings of the 2nd Workshop on Emotions and Personality in Personalized Services (EMPIRE 2014), Aalborg, Denmark, July 2014.

Using Social Media Mining for Estimating Theory of Planned Behaviour Parameters Tkalčič, M., Ferwerda, B., Schedl, M., Liem, C. Melenhorst, M., Odić, A., and Košir, A. Proceedings of the 2nd Workshop on Emotions and Personality in Personalized Services (EMPIRE 2014), Aalborg, Denmark, July 2014.

Social Media and Classical Music? - A first analysis within the PHENICX project: “Performances as Highly Enriched aNd Interactive Concert eXperiences” Schedl, M. SoMeRA'14: Proceedings of the First International Workshop on Social Media Retrieval and Analysis

To Post or Not to Post: The Effects of Persuasive Cues and Group Targeting Mechanisms on Posting Behavior Ferwerda, B., Schedl, M., and Tkalčič, M. Proceedings of the 6th ASE International Conference on Social Computing (SocialCom 2014), Stanford, USA, May 2014.

SoMeRA 2014: Social Media Retrieval and Analysis Workshop Schedl, M., Knees, P., and Shen, J. Proceedings of the 37th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2014), Gold Coast, Australia, July 2014.

User Geospatial Context for Music Recommendation in Microblogs Schedl, M., Vall, A., and Farrahi, K. Proceedings of the 37th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2014), Gold Coast, Australia, July 2014.

Mobile Music Genius: Reggae at the Beach, Metal on a Friday Night? Schedl, M., Breitschopf, G., and Ionescu, B. Proceedings of the ACM International Conference on Multimedia Retrieval (ICMR 2014), Glasgow, Scotland, April 2014.

Improved musical onset detection with convolutional neural networks Schlüter, J. and Böck, S. Proceedings of the 39th International Conference on Acoustics, Speech and Signal Processing (ICASSP 2014), Florence, Italy, May 2014.

Location-Aware Music Artist Recommendation Schedl, M. and Schnitzer, D. Proceedings of the 20th International Conference on MultiMedia Modeling (MMM 2014), Dublin, Ireland, January 2014.

Bridging the Audio-Symbolic Gap: The Discovery of Repeated Note Content Directly from Polyphonic Music Audio Collins, T., Böck, S., Krebs, F., and Widmer, G. Proceedings of the 53rd AES Conference on Semantic Audio, Audio Engineering Society, London, January 2014.

Local Group Delay Aided Vibrato And Tremolo Suppression For Onset Detection Böck, S. and Widmer, G. Proceedings of the 14th International Society for Music Information Retrieval Conference (ISMIR), Curitiba, Brazil.

Rhythmic Pattern Modeling for Beat- and Downbeat Tracking in Musical Audio Krebs, F., Böck, S. and Widmer, G. Proceedings of the 14th International Society for Music Information Retrieval Conference (ISMIR), Curitiba, Brazil.

Automatic Alignment of Music Performances with Structural Differences Grachten, M., Gasser, M., Arzt, A., and Widmer, G. Proceedings of the 14th International Society for Music Information Retrieval Conference (ISMIR), Curitiba, Brazil.

Refined Spectral Template Models for Score Following Korzeniowski, F. and Widmer, G. Proceedings of the Sound and Music Computing Conference (SMC), Stockholm, Sweden.

Tracking Rests and Tempo Changes: Improved Score Following with Particle Filters Korzeniowski, F., Krebs, F., Arzt, A. and Widmer, G. Proceedings of the International Computer Music Conference (ICMC), Perth, Australia.

Enhanced Peak Picking for Onset Detection with Recurrent Neural Networks Böck, S., Schlüter, J., and Widmer, G. Proceedings of the 6th International Workshop on Machine Learning and Music, European Conference on Machine Learning (ECML 2013), Prague, Czech Republic.

Maximum Filter Vibrato Suppression for Onset Detection Böck, S. and Widmer, G. Proceedings of the 16th International Conference on Digital Audio Effects (DAFx-13), Maynooth, Ireland.

Hybrid Retrieval Approaches to Geospatial Music Recommendation Schedl, M. and Schnitzer, D. Proceedings of the 35th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2013), Dublin, Ireland, July-August 2013.

Personality and Social Context: Impact on Emotion Induction from Movies Odic, A., Tkalcic, M., Tasic, J.F., and Kosir, A. Proceedings of the 1st Workshop on Emotions and Personality in Personalized Services (EMPIRE 2013), Rome, Italy, June 2013.