cp.jku - Music Interfaces & Visualization

Department of

Computational Perception

Home – Mission – Teaching – People – Research – Media – Awards – Impressum

MUSIC INTERFACES & VISUALIZATION

<< back to overview

Personal music collection keep on growing and virtually all music is available to anyone – anywhere and anytime. Thus, the way music is used changes rapidly. However, the way music collections are organized and made accessible has basically remained the same, i.e., according to the hierarchical scheme (genre –) artist – album – track. Our goal is to overcome this limited view on music by building new (graphical) interfaces to visualize and to navigate in large collections. In case you are interested in visualization of or interfaces to other types of data, we are open to your ideas.

Automatic Playlist Generation and Personal Radio Stations

Consuming music is most often a lean-back experience, requiring minimum (or even no) interaction by the user. The aim is therefore simply that the "right music" should be played back at any time, preferrably in a meaningful sequence without explicit actions and constant adjustment. In contrast to traditional radio broadcasts that target a broad audience and a most agreeable, generic music taste, such personalized radio stations should address each user's individual need. Many factors can play a role in what is considered the currently right music including the user's general music taste, the current emotional state, activity, situation, surrounding, or social context. However, while implicilty captured in listening histories of users to some extent, extracting such factors remains the central challenge.

Examples for projects:

Latent context modelling of playlists (e.g., using language models)
Predicting mixtape labels, e.g., the purpose of a hand-crafted playlist
User taste profiling and modeling
Learning to predict user feedback (e.g., ratings) or user generated content (e.g., tags) from the music content
Context-aware and behaviour-aware services
Modelling and predicting sequences of music consumption

References:

Zheleva et al. Statistical models of music-listening sessions in social media. In: Proc 19th WWW, 2010.
McFee and Lanckriet. The Natural Language of Playlists. In: Proc 12th ISMIR, 2011.
Aizenberg et al. Build your own music recommender by modeling internet radio streams. In: Proc 21st WWW, 2012.
Hariri et al. Context-aware music recommendation based on latent-topic sequential patterns. In: Proc 6th RecSys, 2012.
Sen and Larson. From Sensors to Songs: A Learning-Free Novel Music Recommendation System using Contextual Sensor Data. In: Proc 1st RecSys Workshop on Location-Aware Recommendations, 2015.

Contact: Peter Knees, Markus Schedl

credit: Brian Sawyer licensed under CC BY-SA 2.0

Intelligent Music Interfaces

The general philosophy is that music collections should be structured (automatically, by the computer) and presented according to intuitive musical criteria. Objectives are to develop innovative, creative, appealing, user-centered, and playful applications to access music and thus to enable new ways of discovering hidden treasures in large collections. One example is nepTune, an interactive, landscape-like interface that permits and even encourages the exploration of music repositories.

Examples for projects:

Exploration of music collections in augmented reality
Planet nepTune: a Google Earth-like 3D user interface for browsing large music collections
nepTune as a second-screen interface on iPad and AppleTV
Visual playlist generation
User-adaptive interfaces to music collections
Active music listening interfaces
Gesture and interaction-based interfaces

Contact: Peter Knees, Markus Schedl

Mobile Music Interfaces & Processing

For mobile devices the challenge is to provide powerful and efficient algorithms and strategies to deal with the limited resources available on such devices (e.g., processing power, screen resolution, interaction capabilities). Music processing on such devices includes developing optimized feature extractors and efficient usage of Web services. Intelligent interfaces aim at elaborating novel and easy-to-use paradigms to improve the user experience when sifting through music and multimedia collections stored on the device or streamed from the Web.

Examples for projects (can be carried out on Android and iOS platforms alike):

Efficient implementation of acoustic and musical features
3D user interfaces to multimedia collections
User-aware interfaces
Accelerometer-based activity detection

Contact: Markus Schedl, Peter Knees

Visualizations

We are also interested in and are actively developing various visualization approaches for different application scenarios related to information visualization and visual analytics.

Examples for projects:

Visualizing "Popularity Flows" around the world using Twitter data, cf. MusicTweetMap
Real-Time Music Visualisation
Investigating the wealth of dimensionality reduction/data projection techniques (e.g., spectral, graph-based, probabilistic methods, neural networks) for representing music and multimedia collections

Contact: Markus Schedl

last edited by pk on Sep 29, 2015