Department of Computational Perception
Department of
Computational Perception
 Johannes Kepler Universität Linz

Home  –  Mission  –  Teaching  –  People  –  Research  –  Media  –  Impressum

sound/tracks

When travelling on a train, many people enjoy looking out of the window and watching the landscape passing by. The fleeting impressions of the moving scenery and the composition of the passing objects generate a piece of visual music with its own tempo and rhythm, its own colours and harmonies.The project sound/tracks aims at capturing these visual impressions and translates them into a musical composition in real-time – producing an immediate soundtrack to the train journey based on the passing scenery. The view out of the window is captured with a camera and translated into instantaneously played back piano music. This yields a reflection of the visual impression, adding a synaesthetic sound dimension to the visual experience and deepening the state of contemplation. The passing scenery can be considered the score of a musical composition which is going to be interpreted based on outside conditions such as weather and lighting, the speed of the train, and the quality of the camera. Thus, every journey will produce a unique composition. In addition to intensifying the experience of a train journey, sound/tracks permits to persistently capture and archive the fleeting impressions of journey and composition and allows for re-experiencing the trip both visually and acoustically at a later point.

Demonstration

Here you can find a selection of sound/track videos
(800x450, mp4, The following selection of videos is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. Creative Commons License)

Japan

Glimpsing at Tokyo Tower before arriving at Shimbashi (2:40, 118MB)

Mt. Fuji sneaks in while approaching Shin-Fuji station (2:38, 87MB)

Heading south from Minami-Miyazaki (3:04, 240MB)

Rhythmic impressions around Ōdōtsu (4:11, 224MB)

sound/waves on Ashinoko  (2:29, 64MB)

From Tokyo to Kyoto (2:00, 76MB)

Austria

Another Bruck and a Wall (6:27, 204MB)

Chromatic Stream (3:17, 165MB)

Dusk in Lower Austria (4:45, 161MB)   [see also rocavaco's remix "Dusk in LA" in the Remixes section]

Impressions from Vienna West (5:27, 218MB)

Motorail in Knittelfeld (2:51, 94MB)

Nightly St. Pölten (4:10, 80MB)

Refinery Schwechat at Night (5:19, 100MB)

Snowy Trains in the Alps (2:26, 59MB)

The Green Heart of Austria (4:53, 192MB)

Tunnel near Mallnitz-Obervellach (3:49, 141MB)

Netherlands

Venlo, Netherlands (1:25, 90MB)


For better comprehension, you are invited to watch a short video, explaining the main idea and the historical, as well as the technical background and giving some examples:

sound/tracks in 4 minutes

[© 2012,  Department of Computational Perception, JKU Linz]


If you like the automatically composed piano music of sound/tracks, you can also listen to a compilation of selected audio tracks, all performed by the computer on a Bösendorfer CEUS Computer-controlled Imperial Grand Piano.
sound/tracks - The Music (Cover) sound/tracks - The Music (mp3, 192 kBit/s)

Track Listing:
3. Scenic Styria (2:43)
4. St. Pölten in a Different Light (based on different color-sound mapping) (1:24)

The journey Wels – Linz is also available as sonified video (Wels – Linz).


Finally, have a look at some panoramic Score Images, i.e. scans of passing landscapes (click to enlarge):

Bruck a.d. Mur

Autotransportzug, Knittelfeld

Lichtwechsel

Wels


Persons Involved (in alphabetical order)

Awards, Exhibitions, Performances

References

Knees, P., Pohle, T., and Widmer, G. (2012).
sound/tracks: artistic real-time sonification of train journeys.
Journal on Multimodal User Interfaces (in press, online first), Springer.

Pohle, T., Knees, P., and Widmer, G. (2008).
sound/tracks: Real-Time Synaesthetic Sonification and Visualisation of Passing Landscapes. In Proceedings of the ACM Multimedia 2008 - Interactive Arts Program, Vancouver, BC, Canada.

Knees, P., Pohle, T., and Widmer, G. (2008).
sound/tracks: Real-Time Synaesthetic Sonification of Train Journeys. In Proceedings of the ACM Multimedia 2008 - Interactive Arts Exhibition, Vancouver, BC, Canada.

Pohle, T. and Knees, P. (2008).
Real-Time Synaesthetic Sonification of Traveling Landscapes.
In Proceedings of the 5th International Mobile Music Workshop (MMW 2008), Vienna, Austria.

Download

There are two versions of sound/tracks to immediately create a soundtrack for any train journey:

Remixes



Acknowledgments

Bösendorfer logo Special Thanks are due to L. Bösendorfer Klavierfabrik GmbH, Vienna for giving us the opportunity to record selected tracks on a CEUS Computer-controlled Imperial Grand Piano and for supporting the performace at the ERC Gala in April 2012.
We also gratefully acknowledge support by Yamaha and the Cultural Department of the Austrian Embassy in Tokyo during the Japan Media Arts Festival in February 2011 and the Media Arts Festival in Miyazaki in December 2012.

Research underlying this work is supported by the Austrian Fonds zur Förderung der Wissenschaftlichen Forschung (FWF) under project numbers L112-N04, L511-N15, TRP 109-N23, and Z159 (Wittgenstein Award).


last edited by pk on May 2, 2012