|
Department of
Computational
Perception
|
|
TOPICS IN MUSIC UNDERSTANDING & TRACKING
<< back to Music Understanding
& Tracking
Automatic Music
Accompaniment
Problem Description:
The task is to develop an "artificial orchestra" that follows a soloist
(e.g., a pianist playing the solo part of a Mozart piano concerto) and
plays
the orchestra part along with the soloist, perfectly synchronising the
orchestra to the soloist's timing and tempo changes.
The challenge for the orchestra is not only to track and follow tempo
changes, but also to predict possible changes in tempo and
anticipate when the soloist will continue after a pause. To that
end,
the computer will have to learn a predictive tempo model during the
live
performance (possibly guided by tempo model learned, off-line, from
previous
performances or rehearsals).
What you can build on:
- robust real-time music tracker
- a software framework for real-time audio applications (C++)
- audio recordings of solo and orchestra part for experiments
This little video shows an application of our real-time music tracking
algorithm (and Prof. Widmer as amateur pianist): an
automatic page turning device for musicians that listens and knows when
it is time to turn the page.
Our automatic reactive page turner
What you will have to do / develop:
- look into audio time-stretching libraries and setups for
experiments (we will help you with that)
- build working prototype: connect piano tracking and
time-stretched replay of orchestra accompaniment
- develop predictive tempo models and synchonisation strategies
that feel "natural" to the solo performer (this is a very hard
task, and we do not expect perfect results!)
Required skills:
- an interest in (classical) music
- programming skills: C++
Some literature for a start:
Note on financial compensation:
If this works out extremely well (which we will be able to tell around
half way
into the thesis working period), we will also be able to pay some
financial compensation!
Contact: Andreas Arzt,
Gerhard Widmer
last edited by pk on Oct 1, 2015