We show run-time computations using acoustic ray tracing with diffraction rays for localizing the 3D position of the sound source in the environment.
Inputs to our run-time algorithm are the audio stream collected by the microphone array, the mesh reconstructed in the precomputation, and the robot position localized by the SLAM algorithm.
Based on those input, we perform acoustic ray tracing supporting direct, reflection, and diffraction effects by generating various acoustic rays.
The diffraction-aware acoustic ray tracing is highlighted in blue and our main contribution in this paper.
The source position estimation is performed by identifying ray convergence.
This figure shows the diffraction condition proposed in order to support various cases that arise in real environments.
When an incident ray passes close to an edge of a wedge, we consider the ray to be generated by the edge diffraction.
We measure the angle Theta_D between ray and its ideal generated ray that hits the edge exactly, for checking our diffraction condition.
This figure illustrates our acoustic ray tracing method for handling the diffraction effect.
Suppose that we have an acoustic ray satisfying the diffraction condition.
We generate N_d diffraction rays covering the possible incoming direction (especially, in the shadow region) that cause the diffraction.
Please refer to the paper for details.
To demonstrate the benefits of our diffraction-aware method, we test our approach in two different testing scenarios in a classroom environment.
Left figure: The evaluation environment for the static sound source. Direct paths from the sound source to the listener are blocked by the obstacle.
Right figure: A Non-Line-of-Sight (NLOS) moving source scene around an obstacle.
Our method can localize the position of the source using acoustic sensors and our diffraction-aware ray tracing.
Please refer to the paper and the working video for the results.
This video shows the operations of our algorithm, Diffraction-Aware Sound Localization for a Non-Line-of-Sight Source, in various environments.
First, it shows the main contributions and benefits of this paper.
Second, it shows examples of generating reflection and diffraction acoustic rays.
Third, it shows the operation of our algorithm in the environments with a stationary non-line-of-sight (NLOS) sound source of clapping.
Fourth, it shows the operation of our algorithm in the environments with a dynamic NLOS sound source of clapping.
Fifth, it shows the operation of our algorithm in the environments with a stationary non-line-of-sight (NLOS) sound source of the male voice
This video shows that our algorithm works well in many scenarios containing a dynamic and NLOS source and many kinds of sounds.
The description video.
Abstract
We present a novel sound localization algorithm for a non-line-of-sight (NLOS) sound source in indoor environments. Our approach exploits the diffraction properties of sound waves as they bend around a barrier or an obstacle in the scene. We combine a ray tracing based sound propagation algorithm with a Uniform Theory of Diffraction (UTD) model, which simulate bending effects by placing a virtual sound source on a wedge in the environment. We precompute the wedges of a reconstructed mesh of an indoor scene and use them to generate diffraction acoustic rays to localize the 3D position of the source. Our method identifies the convergence region of those generated acoustic rays as the estimated source position based on a particle filter. We have evaluated our algorithm in multiple scenarios consisting of a static and dynamic NLOS sound source. In our tested cases, our approach can localize a source position with an average accuracy error, 0.7m, measured by the L2 distance between estimated and actual source locations in a 7m*7m*3m room. Furthermore, we observe 37% to 130% improvement in accuracy over a state-of-the-art localization method that does not model diffraction effects, especially when a sound source is not visible to the robot.
