Constraint-based Spatialization

SemanticHIFITechnologiesRendering > Constraint-based Spatialization <

General Description

The MusicSpace application is a control interface for sound spatialization that describes a sound scene (sound sources and the listener's avatar) from a top view. The listener can at any time move the sound sources or its avatar (in order for example to get closer to an instrument he/she likes) and perceive the changes in the auditory scene in real time.

Moreover, a constraint solver algorithm and authoring tool is included to the system in order to specify relations between sound sources and/or the avatar, so that a naïve user (e.g. music amateur that has no particular knowledge in audio engineering) can perform actions on the sound scene while being guaranteed that the resulting 3D audio mix will remain consistent.

This project was originally a MIDI-based application (e.g. could only play midifiles and use very simple panning techniques to perform spatialization). A more advanced version allowed audio sources but was restricted to the use of the Microsoft DirectX mixing capabilities.

This demonstration was the opportunity to create a bridge between MusicSpace, developed at Sony-CSL, and the IRCAM Spatialisateur in order to take advantage of both control possibilities with constraints brought by MusicSpace and high quality spatial audio rendering brought by the IRCAM Spatialisateur.

Responsible partner/team: IRCAM Room Acoustics team

Overview of the MusicSpace graphical user interface
Corresponding implementation of the rendering engine in the Max/MSP environment