MSMD Real Performances

In order to explore the generalization capabilities of our reinforcement learning agents, we asked two pianists to record 16 pieces (8 per pianist) from the MSMD test split. The pianists were asked to perform the pieces without going to extremes. Still, the performances include local tempo deviations, as well as additional ornaments like trills (especially in the Bach and Mozart pieces). The actual recordings took place at our institute in a regular office environment (room dimensions c. 5 × 6 meters). The instrument was a Yamaha AvantGrand N2 hybrid piano (A4 = 440 Hz). From the piano, we simultaneously recorded the MIDI signal (performance MIDI), the direct stereo output (dry), and the acoustic room signal by placing an omnidirectional microphone about 2 meters away from the piano. To establish a ground truth, we used the performance MIDI files, the corresponding score MIDI file, and a rendered version of the score, where the latter two were taken from the MSMD dataset (i.e., the score images were generated with Lilypond). Using the alignment technique described by Dorfer et al. (2018a), we established the needed connection between note head positions in the scores and the corresponding notes in the performance MIDI file and audio recordings, respectively. Note that this alignment is automatically generated and not perfect for all of the notes.

Pieces Overview


Composer Piece name Dur. (sec.)
Bach, Johann Sebastian Polonaise in F major, BWV Anh. 117a 47.32
Bach, Johann Sebastian Sinfonia in G minor, BWV 797 99.69
Bach, Johann Sebastian French Suite No. 6 in E major, Menuet, BWV 817 37.21
Bach, Johann Sebastian Partita in E minor, Allemande, BWV 830-2 86.73
Bach, Johann Sebastian Prelude in C major, BWV 924a 40.43
Bach, Johann Sebastian Minuet in F major, BWV Anh. 113 40.49
Bach, Johann Sebastian Minuet in G major, BWV Anh. 116 51.56
Bach, Johann Sebastian Minuet in A minor, BWV Anh. 120 31.32
Chopin, Frédéric François Nocturne in B♭ minor, Op. 9, No. 1 328.92
Mozart, Wolfgang Amadeus Piano Sonata No. 11 in A major, 1st Movt, Variation 1, KV331 56.33
Mussorgsky, Modest Petrovich Pictures at an Exhibition, Promenade III 27.17
Schumann, Robert Album für die Jugend, Op. 68, 1. Melodie 45.5
Schumann, Robert Album für die Jugend, Op. 68, 6. Armes Waisenkind 73.52
Schumann, Robert Album für die Jugend, Op. 68, 8. Wilder Reiter 24.88
Schumann, Robert Album für die Jugend, Op. 68, 16. Erster Verlust 55.83
Schumann, Robert Album für die Jugend, Op. 68, 26. Untitled 74.4

Acknowledgments

We would like to thank Jan Hajič jr. and Carlos Eduardo Cancino Chacón - both wonderful colleagues and fabulous piano players - for helping to record the piano performances.