sound/tracks - Department of Computational Perception

When travelling on a train, many people enjoy looking out of the window and watching the landscape passing by. The fleeting impressions of the moving scenery and the composition of the passing objects generate a piece of visual music with its own tempo and rhythm, its own colours and harmonies.The project sound/tracks aims at capturing these visual impressions and translates them into a musical composition in real-time – producing an immediate soundtrack to the train journey based on the passing scenery. The view out of the window is captured with a camera and translated into instantaneously played back piano music. This yields a reflection of the visual impression, adding a synaesthetic sound dimension to the visual experience and deepening the state of contemplation. The passing scenery can be considered the score of a musical composition which is going to be interpreted based on outside conditions such as weather and lighting, the speed of the train, and the quality of the camera. Thus, every journey will produce a unique composition. In addition to intensifying the experience of a train journey, sound/tracks permits to persistently capture and archive the fleeting impressions of journey and composition and allows for re-experiencing the trip both visually and acoustically at a later point.

Demonstration

Another Bruck and a Wall (6:27, 204MB)

Chromatic Stream (3:17, 165MB)

Dusk in Lower Austria (4:45, 161MB)

Impressions from Vienna West (5:27, 218MB)

Motorail in Knittelfeld (2:51, 94MB)

Nightly St. Pölten (4:10, 80MB)

Refinery Schwechat at Night (5:19, 100MB)

Snowy Trains in the Alps (2:26, 59MB)

The Green Heart of Austria (4:53, 192MB)

Tunnel near Mallnitz-Obervellach (3:49, 141MB)

For better comprehension, you are invited to watch a promotion video of the project, explaining the main idea and the historical, as well as the technical background and giving some examples:

sound/tracks - The Video (352x288, mov, H.264/AAC codec)

If you like the automatically composed piano music of sound/tracks, you can also listen to a compilation of selected audio tracks, all performed by the computer on a Bösendorfer CEUS Computer-controlled Imperial Grand Piano.

sound/tracks - The Music (mp3, 192 kBit/s)

Track Listing:

1. Leaving Villach Central Station (3:08)

2. Swing in St. Veit / Glan (3:39)

3. Scenic Styria (2:43)

4. St. Pölten in a Different Light (based on different color-sound mapping) (1:24)

The journey Wels – Linz is also available as sonified video (Wels – Linz).

Finally, have a look at some panoramic Score Images, i.e. scans of passing landscapes (click to enlarge):