Shaping sound with audio beamforming

The idea of using an array of speakers to shape sound using beamforming has been around for many years, but up to now it has been difficult to put into practice.

Using MATLAB and Signal Processing Toolbox, Philips Research engineers have developed a technology that enables two people in the same room to hear the same audio output at different volumes. Philips was granted a patent for its audio beamforming technology in 2012.

Beamforming relies on different speakers responding to the same input signal in different ways - for example, by slightly delaying the signal, playing it at different volumes or using cancellation effects. The different speaker settings allow the system to control the size, shape and direction of the acoustic wave. Because of the large range of sound wavelengths, there are conflicting requirements for ensuring good performance at both low frequencies (requiring a relatively large array size) and high frequencies (requiring a small distance between speakers). Fulfilling both requirements typically means that the array needs to consist of a relatively large number of speakers that have to be controlled individually. As a result, dynamically shaping the acoustic wave requires powerful real-time signal processing that until recently has been too expensive for consumer applications. With the falling cost of signal processing chips, this technology has become cheap enough to be applied in consumer products.

Philips’ audio beamforming system consists of a loudspeaker array with each speaker in the array controlled by a digital signal processor (DSP). The DSP uses an FIR filter and other signal processing algorithms to control the loudspeaker output.

Enhancing the model

Werner de Bruijn and his colleagues began their research with a proof-of-concept model consisting of a simple loudspeaker frequency response in the transfer function. As the project matured, they included more sophisticated and realistic speaker frequency responses. In cases where they knew which speaker they were going to be using, they included the measured loudspeaker frequency responses. If this data wasn’t available (perhaps because the speaker hadn’t yet been built), they used a theoretical response, derived from a MATLAB model.

Audio beamforming applied to an audiovisual device on display in a shopping mall.

Testing the system behaviour

Using the MATLAB audio toolbox, the engineers tested the performance of the loudspeaker parameters using sound propagation simulation. They entered the room characteristics (length, width, height and loudspeaker locations) into their models and converted the frequency-domain output data to a time-domain model. They configured the output of the time-domain model as a series of movie frames, which they then combined to show an animation of the propagation of the sound wave through the room using the MATLAB movie function. The animation clearly revealed individual beams and their frequency dependence. At low frequencies, there was little beam formation, which is acceptable because the type of content the engineers wanted to beam (eg, TV commentary) typically had relatively little energy at very low frequencies anyway. At higher frequencies, they saw additional unwanted beams forming. They examined the results to find out whether these extra beams would cause effects audible to users.

Because the output of the modelling work includes a complex transfer function for every point in the room, the engineers ran one further check on the behaviour of the system. They took the response at two points spaced about 20 cm apart (the width of a human head) and convolved the response with a real audio signal, such as music or a television commentary. They then output the results of their MATLAB calculations as an audio stream and listened to the result. These simulations enabled the researchers to hear what the real audio would sound like at any point in the room, giving them insight into the system’s sound quality and intensity.

As well as varying the sound level at different points in the room, the same technology can be used to feed different audio signals to different parts of the room. One person can listen to classical music while another listens to a TV program. The different sound channels can be placed anywhere in the room.

Ongoing audio research

The combination of MATLAB simulation and low-cost digital signal processing is making it possible to create brand new products using audio beamforming. A recent project applies audio beamforming to an audiovisual device on display in a shopping mall. With audio beamforming technology,  the sound can be directed to a person who is interacting with the display or even have a separate audio stream for each individual interacting with the system (for example, one stream in English and one in Dutch). Individuals passing by the system, perhaps only two metres or so away, won’t hear a thing.