Musical Gestures: Sound, Movement, and Meaning

One of the main topics of the Musical Gestures book is that of different gesture types. We may think of two main categories of music-related gestures: sound-producing gestures, meaning gestures that produce sound on an instrument, and sound-accompanying gestures, meaning gestures that listeners make to music. However, the distinction between these two main categories may not always be so clear cut; we may often see musicians make a number of different body movements when performing, and it is one of the main topics of musical gestures research to try to understand the different functions of these body movements.

Here we can see an example of this in some excerpts of Guqin performance that is discussed in chapter 7:

That there is always a continuous movement in sound-producing gestures can also be clearly seen in this slow-motion video recording of a pianist's finger, hand, and arm movements:

With present-day technology, we also see the development of new musical instruments where the performers' gestures control the sound generation, but where the sound itself is generated by a computer. The great challenge in developing such new musical instruments is that of giving both the performer and the audience some sense of connection between the control gestures made by the performer and the features of the resultant sound. Here we see an example of such gestural control in the form of a "music balls", a series of new musical instruments containing various sensors that send control data to a computer generating the sound:

As can be seen in chapter 11, there is also ongoing work in controlling the performance of more traditional Western music by way of conducting gestures. In the following example we see how a game controller can be used to control the tempo in the playback of an orchestral score:

The second main category of music-related gestures, that of sound-accompanying gestures, can be seen almost everywhere, at concerts, in dance, and in innumerable everyday listening situations. Common to most such sound-accompanying gestures is that they tend to lock onto some salient feature in the music such as its beat and various metrical patterns, various melodic and textural shapes, or more diffusely the overall sense of effort and motion of the music. In the following example, we can see the slow, protracted gestures of a dancer to the correspondingly slow, protracted musical sound. The video excerpt has been processed so as to more clearly show the trajectories of the dancer's gestures:

As a contrast to this slow and calm kind of sound-accompanying gestures, we can see an example of fast and more abrupt kinds of gestures in the following example:  

There are a number of other interesting video examples available on the Internet and from the various groups around the world that study musical gestures.

Related Software

• EyesWeb by the University of Genova
• MoCap Toolbox for Matlab by the University of Jyväskylä
• fourMs software by the University of Oslo

Involved Groups/Labs

• Finnish Centre of Excellence in Interdisciplinary Music Research, University of Jyväskylä
• FourMs, University of Oslo
• IDMIL, McGill University
• InfoMus Lab, University of Genova
• IPEM and EmcoMetecca, University of Ghent
• ISSCM International Cooperative in Systematic and Comparative Musicology

Related Projects

• ConGAS (Gesture controlled audio systems)
• Enactive Network
• HUMAINE emotion research project
• Human Supervision and Control in Engineering and Music
• Interactive Music Network
• Musical Gestures Project, University of Oslo
• S2S^2 - Sound to Sense, Sense to Sound
• Sonic Interaction Design (COST IC0601 Action SID)
• Sounding Object Project
• TAI-CHI Tangible Acoustic Interfaces for Computer-Human Interaction

Contents

Foreword
Editors' preface

Part I: Gestures in Music

1. Why study musical gestures?
Marc Leman and Rolf Inge Godøy

2. Musical Gestures: concepts and methods in research
Alexander Refsum Jensenius, Marcelo M. Wanderley, Rolf Inge Godøy, and Marc Leman

3. Gestures in performance
Sofia Dahl, Frédéric Bevilacqua, Roberto Bresin, Martin Clayton, Laura Leante, Isabella Poggi, and Nicolas Rasamimanana

4. Music and Gestures: a historical introduction and survey of earlier research
Albrecht Schneider

Part II: Gestural Significations

5. Gestural affordances of musical sound
Rolf Inge Godøy

6. Music, Gesture, And The Formation Of Embodied Meaning
Marc Leman

7. The functional role and bio-kinetics of basic and expressive gestures in activation and sonification
Leon van Noorden

Part III: Gesture Generation And Control

8. Gesture and timbre
Tor Halmrast, Knut Guettler, Rolf Bader, and Rolf Inge Godøy

9. Sensorimotor control of sound-producing gestures
Sylvie Gibet

10. Visual gesture recognition: from motion tracking to expressive gesture
Antonio Camurri and Thomas B. Moeslund

11. Conductors’ gestures and their mapping to sound synthesis
Gunnar Johannsen and Teresa Marrin Nakra

Contributors
Index