Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Rir Data Model
Numerical Analysis
Experimental Analysis
Conclusions
References
All Topics
Computer Science
Artificial Intelligence

Deep Prior Approach for Room Impulse Response Reconstruction

Profile image of Mirco PezzoliMirco Pezzoli

Sensors

https://doi.org/10.3390/S22072710
visibility

…

description

15 pages

link

1 file

Abstract

In this paper, we propose a data-driven approach for the reconstruction of unknown room impulse responses (RIRs) based on the deep prior paradigm. We formulate RIR reconstruction as an inverse problem. More specifically, a convolutional neural network (CNN) is employed prior, in order to obtain a regularized solution to the RIR reconstruction problem for uniform linear arrays. This approach allows us to avoid assumptions on sound wave propagation, acoustic environment, or measuring setting made in state-of-the-art RIR reconstruction algorithms. Moreover, differently from classical deep learning solutions in the literature, the deep prior approach employs a per-element training. Therefore, the proposed method does not require training data sets, and it can be applied to RIRs independently from available data or environments. Results on simulated data demonstrate that the proposed technique is able to provide accurate results in a wide range of scenarios, including variable direction ...

Related papers

MIRaGe: Multichannel Database of Room Impulse Responses Measured on High-Resolution Cube-Shaped Grid
Sharon Gannot

2020 28th European Signal Processing Conference (EUSIPCO), 2021

We introduce a database of multi-channel recordings performed in an acoustic lab with adjustable reverberation time. The recordings provide detailed information about room acoustics for positions of a source within a confined area. In particular, the main positions correspond to 4104 vertices of a cube-shaped dense grid within a 46 × 36 × 32 cm volume. The database can serve for simulations of a real-world situations and as a tool for detailed analyses of beampatterns of spatial processing methods. It could be used also for training and testing of mathematical models of the acoustic field.

View PDFchevron_right
Towards Improved Room Impulse Response Estimation for Speech Recognition
Anton Ratnarajah

arXiv (Cornell University), 2022

We propose to characterize and improve the performance of blind room impulse response (RIR) estimation systems in the context of a downstream application scenario, far-field automatic speech recognition (ASR). We first draw the connection between improved RIR estimation and improved ASR performance, as a means of evaluating neural RIR estimators. We then propose a GAN-based architecture that encodes RIR features from reverberant speech and constructs an RIR from the encoded features, and uses a novel energy decay relief loss to optimize for capturing energy-based properties of the input reverberant speech. We show that our model outperforms the state-of-the-art baselines on acoustic benchmarks (by 72% on the energy decay relief and 22% on an early-reflection energy metric), as well as in an ASR evaluation task (by 6.9% in word error rate).

View PDFchevron_right
Blind estimation of room acoustic parameters from speech signals based on extended model of room impulse response
Suradej Duangpummet

arXiv (Cornell University), 2022

The speech transmission index (STI) and room acoustic parameters (RAPs), which are derived from a room impulse response (RIR), such as reverberation time and early decay time, are essential to assess speech transmission and to predict the listening difficulty in a sound field. Since it is difficult to measure RIR in daily occupied spaces, simultaneous blind estimation of STI and RAPs must be resolved as it is an imperative and challenging issue. This paper proposes a deterministic method for blindly estimating STI and five RAPs on the basis of an RIR stochastic model that approximates an unknown RIR. The proposed method formulates a temporal power envelope of a reverberant speech signal to obtain the optimal parameters for the RIR model. Simulations were conducted to evaluate STI and RAPs from observed reverberant speech signals. The root-mean-square errors between the estimated and ground-truth results were used to comparatively evaluate the proposed method with the previous method. The results showed that the proposed method can estimate STI and RAPs effectively without any training.

View PDFchevron_right
Refinement and Implementation of a Robust Directional Room Impulse Response Denoising Process, Including Applications to Highly Varied Measurement Databases
Pierre Massé

2019

Recent developments in the measurement of directional room impulse responses (DRIR) by spherical microphone arrays (SMA) have led to their extensive use in sound spatialisation. Room reverberation effects can be reproduced in three-dimensional surround sound systems (e.g. Higher-Order Ambisonics) through multi-channel DRIR convolution. However, such measured impulse responses inevitably present a non-negligible noise floor, leading to a perceptible ‘infinite reverberation effect’. Further, individual sensor noise and non-stationary measurement artefacts may additionally corrupt the deconvolved impulse response. This paper presents recent work regarding the implementation of state of the art DRIR analysis and denoising techniques and their application to extensive DRIR databases measured across a highly varied collection of spaces. We first review the basic energy decay relief (EDR) analysis and reverberation tail re-synthesis process, before presenting several novel refinements deve...

View PDFchevron_right
AN INVESTIGATION OF ACOUSTIC IMPULSE RESPONSE MEASUREMENT AND MODELING FOR SMALL ROOMS
young kim
View PDFchevron_right
Blind Estimation of Room Acoustic Parameters and Speech Transmission Index using MTF-based CNNs
Suradej Duangpummet

2021 29th European Signal Processing Conference (EUSIPCO), 2021

Room acoustic parameters, such as reverberation time (T60) and clarity (C80), as well as an objective index, the speech transmission index (STI), are essential in acoustics. Such parameters and STI are, however, difficult to obtain in everyday places with people. Blind estimation without measuring room impulse response (RIR) is necessary and challenging. This paper proposes a method based on the modulation transfer function (MTF) and Schroeder's RIR model for estimating T60s in sevenoctave bands. The estimated T60s are used to approximate the MTF and RIR. The STI and five room-acoustic parameters, including T60, early decay time (EDT), C80, Deutlichkeit (D50), and center time (Ts), can therefore be estimated. Convolutional neural networks (CNNs) were used for mapping temporal amplitude envelopes of a reverberant speech signal to T60s for the sub-bands. Simulations were carried out by estimating the five parameters and STI from unseen reverberant speech signals. The root-mean-square errors between ground-truths and estimated parameters suggest that the accuracy of estimated T60 and STIs can be improved by about 40% and 25% compared with previous methods, respectively. The other parameters were also correctly estimated, and they are comparable with those obtained from standard measurements.

View PDFchevron_right
Measurement, Analysis, and Denoising of Directional Room Impulse Responses in Complex Spaces
Pierre Massé

2020

The use of directional room impulse responses (DRIR) measured with spherical microphone arrays (SMA) has become widespread in the reproduction of spatial reverberation effects on surround-sound systems through multichannel convolution. However, the measurement of such DRIRs in "real-world" conditions is inevitably subject to several risk factors, including the presence of a nondecaying noise floor that can produce an "infinite reverberation" effect when convolved with an input sound. Recent work has focused on model-based techniques for removing this noise floor by replacing it with a re-synthesized prolongation of the measured reverberation tail, which has concurrently led to the development of a framework for the spatial analysis of reverberation properties. We present here a comprehensive evaluation of the proposed techniques through their application to DRIRs measured in particularly complex spaces, including spatially anisotropic late reverberation tails as ...

View PDFchevron_right
Adaptive inverse filtering of room acoustics
Andy Khong

2008 42nd Asilomar Conference on Signals, Systems and Computers, 2008

Equalization techniques for high order, multichannel, FIR systems are important for dereverberation of speech observed in reverberation using multiple microphones. In this case the multichannel system represents the room impulse responses (RIRs). The existence of near-common zeros in multichannel RIRs can slow down the convergence rate of adaptive inverse filtering algorithms. In this paper, the effect of common and near-common zeros on both the closed-form and the adaptive inverse filtering algorithms is studied. An adaptive shortening algorithm of room acoustics is presented based on this study.

View PDFchevron_right
Single-Microphone LP Residual Skewness-Based Inverse Filtering of the Room Impulse Response
saeed sheikhzadeh

IEEE Transactions on Audio, Speech & Language Processing, 2012

This paper presents a method based on higher order statistics (HOS), namely the normalized third-order moment (skewness), for blind estimation of the inverse filter of the room impulse response (RIR). Skewness is used as a measure of asymmetry, and a comprehensive comparison with the commonly used metric (kurtosis) is presented. It is shown that a sufficiently long linear predictive (LP) residual of the speech signal has an asymmetric pdf with sufficient skewness to be used as a score function for the HOS-based approach. The proposed algorithm is optimized for the inverse filter estimation problem. This optimization includes an efficient initialization for high reverberation intensities, enabling the method to be employed in highly reverberant rooms. The direct-to-reverberation ratio (DRR) as well as the equalized impulse response clearly show that our method can estimate the inverse filter even in highly reverberant environments. In addition, performance results using recorded background noise and in time-varying environments illustrate that our approach is applicable in real world situations. The proposed method is shown to be superior to the method by Wu and Wang, particularly in terms of reducing the coloration effect. Experiments under different acoustic conditions confirm the effectiveness of the proposed method for time delay estimation (TDE). Finally, the proposed algorithm is used as the first-stage of monaural segregation, and it is shown to improve the performance under different conditions.

View PDFchevron_right
Database of binaural room impulse responses of an apartment-like environment
Sascha Spors

HAL (Le Centre pour la Communication Scientifique Directe), 2016

We present a database of binaural room impulse responses (BRIRs) measured in an apartment-like environment. The BRIRs were captured at four different sound source positions, each combined with four listener positions. A head and torso simulator (HATS) with varying head-orientation in the range of ±78 • with 2 • resolution was used. Additionally, BRIRs of 20 listener positions along a trajectory connecting two of the four positions were measured, each with a fixed head-orientation. The data is provided in the Spatially Oriented Format for Acoustics (SOFA) and it is freely available under the Creative Commons (CC-BY-4.0) license. It can be used to simulate complex acoustic scenes in order to study the process of auditory scene analysis for humans and machines.

View PDFchevron_right

References (58)

  1. Tohyama, M.; Koike, T. (Eds.) Transfer Function and Frequency Response Function. In Fundamentals of Acoustic Signal Processing; Academic Press: London, UK, 1998; pp. 75-110. [CrossRef]
  2. Nelson, P.A.; Elliott, S.J. Active Control of Sound; Academic Press: New York, NY, USA, 1991.
  3. Cobos, M.; Antonacci, F.; Alexandridis, A.; Mouchtaris, A.; Lee, B. A survey of sound source localization methods in wireless acoustic sensor networks. Wirel. Commun. Mob. Comput. 2017, 2017, 3956282. [CrossRef]
  4. Gannot, S.; Vincent, E.; Markovich-Golan, S.; Ozerov, A. A consolidated perspective on multimicrophone speech enhancement and source separation. IEEE/ACM Trans. Audio Speech Lang. Process. 2017, 25, 692-730. [CrossRef]
  5. Pezzoli, M.; Carabias-Orti, J.J.; Cobos, M.; Antonacci, F.; Sarti, A. Ray-Space-Based Multichannel Nonnegative Matrix Factorization for Audio Source Separation. IEEE Signal Process. Lett. 2021, 28, 369-373. [CrossRef]
  6. Tylka, J.G.; Choueiri, E.Y. Fundamentals of a parametric method for virtual navigation within an array of ambisonics microphones. J. Audio Eng. Soc. 2020, 68, 120-137. [CrossRef]
  7. Rife, D.D.; Vanderkooy, J. Transfer-function measurement with maximum-length sequences. J. Audio Eng. Soc. 1989, 37, 419-444.
  8. Farina, A. Advancements in Impulse Response Measurements by Sine Sweeps. In Audio Engineering Society Convention 122; Audio Engineering Society: Vienna, Austria, 2007. Available online: http://www.aes.org/e-lib/browse.cfm?elib=14106 (accessed on 28 March 2022).
  9. Stan, G.B.; Embrechts, J.J.; Archambeau, D. Comparison of different impulse response measurement techniques. J. Audio Eng. Soc. 2002, 50, 249-262.
  10. Ajdler, T.; Sbaiz, L.; Vetterli, M. Dynamic measurement of room impulse responses using a moving microphone. J. Acoust. Soc. Am. 2007, 122, 1636-1645. [CrossRef]
  11. Thiergart, O.; Del Galdo, G.; Taseska, M.; Habets, E.A.P. Geometry-based spatial sound acquisition using distributed microphone arrays. IEEE Trans. Audio Speech Lang. Process. 2013, 21, 2583-2594. [CrossRef]
  12. Pezzoli, M.; Borra, F.; Antonacci, F.; Sarti, A.; Tubaro, S. Estimation of the Sound Field at Arbitrary Positions in Distributed Microphone Networks Based on Distributed Ray Space Transform. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada, 15-20 April 2018; pp. 186-190.
  13. Pezzoli, M.; Borra, F.; Antonacci, F.; Sarti, A.; Tubaro, S. Reconstruction of the Virtual Microphone Signal Based on the Distributed Ray Space Transform. In Proceedings of the 26th European Signal Processing Conference (EUSIPCO), Rome, Italy, 3-7 September 2018; pp. 1537-1541.
  14. Pezzoli, M.; Borra, F.; Antonacci, F.; Tubaro, S.; Sarti, A. A parametric approach to virtual miking for sources of arbitrary directivity. IEEE/ACM Trans. Audio Speech Lang. Process. 2020, 28, 2333-2348. [CrossRef]
  15. Pulkki, V.; Delikaris-Manias, S.; Politis, A. Parametric Time-Frequency Domain Spatial Audio; Wiley Online Library: Hoboken, NJ, USA, 2018.
  16. Das, O.; Calamia, P.; Gari, S.V.A. Room Impulse Response Interpolation from a Sparse Set of Measurements Using a Modal Architecture. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada, 6-11 June 2021; pp. 960-964.
  17. Haneda, Y.; Kaneda, Y.; Kitawaki, N. Common-acoustical-pole and residue model and its application to spatial interpolation and extrapolation of a room transfer function. IEEE Trans. Speech Audio Process. 1999, 7, 709-717. [CrossRef]
  18. Koyama, S.; Daudet, L. Sparse Representation of a Spatial Sound Field in a Reverberant Environment. IEEE J. Sel. Top. Signal Process. 2019, 13, 172-184. [CrossRef]
  19. Damiano, S.; Borra, F.; Bernardini, A.; Antonacci, F.; Sarti, A. Soundfield reconstruction in reverberant rooms based on compressive sensing and image-source models of early reflections. In Proceedings of the 2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA, 17-20 October 2021.
  20. Zea, E. Compressed sensing of impulse responses in rooms of unknown properties and contents. J. Sound Vib. 2019, 459, 114871. [CrossRef]
  21. Antonello, N.; De Sena, E.; Moonen, M.; Naylor, P.A.; Van Waterschoot, T. Room impulse response interpolation using a sparse spatio-temporal representation of the sound field. IEEE/ACM Trans. Audio Speech Lang. Process. 2017, 25, 1929-1941. [CrossRef]
  22. Borra, F.; Gebru, I.D.; Markovic, D. Soundfield reconstruction in reverberant environments using higher-order microphones and impulse response measurements. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 12-17 May 2019; pp. 281-285.
  23. Borra, F.; Krenn, S.; Gebru, I.D.; Marković, D. 1st-order microphone array system for large area sound field recording and reconstruction: Discussion and preliminary results. In Proceedings of the 2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA, 20-23 October 2019; pp. 378-382.
  24. Birnie, L.; Abhayapala, T.; Tourbabin, V.; Samarasinghe, P. Mixed Source Sound Field Translation for Virtual Binaural Application With Perceptual Validation. IEEE/ACM Trans. Audio Speech Lang. Process. 2021, 29, 1188-1203. [CrossRef]
  25. Mignot, R.; Chardon, G.; Daudet, L. Low frequency interpolation of room impulse responses using compressed sensing. IEEE/ACM Trans. Audio Speech Lang. Process. 2013, 22, 205-216. [CrossRef]
  26. Jin, W.; Kleijn, W.B. Theory and design of multizone soundfield reproduction using sparse methods. IEEE/ACM Trans. Audio Speech Lang. Process. 2015, 23, 2343-2355.
  27. Williams, E.G. Fourier Acoustics; Academic Press: London, UK, 1999.
  28. Fahim, A.; Samarasinghe, P.N.; Abhayapala, T.D. Sound field separation in a mixed acoustic environment using a sparse array of higher order spherical microphones. In Proceedings of the 2017 Hands-Free Speech Communications and Microphone Arrays (HSCMA), San Francisco, CA, USA, 1-3 March 2017; pp. 151-155.
  29. Pezzoli, M.; Cobos, M.; Antonacci, F.; Sarti, A. Sparsity-Based Sound Field Separation in The Spherical Harmonics Domain. In Proceedings of the Accepted for IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, 22-27 May 2022.
  30. Lee, S. Review: The Use of Equivalent Source Method in Computational Acoustics. J. Comput. Acoust. 2016, 25, 1630001. [CrossRef]
  31. Donoho, D.L. Compressed sensing. IEEE Trans. Inf. Theory 2006, 52, 1289-1306. [CrossRef]
  32. Herrmann, F.J.; Hennenfent, G. Non-parametric seismic data recovery with curvelet frames. Geophys. J. Int. 2008, 173, 233-248.
  33. Labate, D.; Lim, W.Q.; Kutyniok, G.; Weiss, G. Labate, D.; Lim, W.Q.; Kutyniok, G.; Weiss, G. Sparse Multidimensional Representation Using Shearlets. In Proceedings of the Wavelets XI, International Society for Optics and Photonics, San Diego, CA, USA, 31 July-4 August 2005; Volume 5914, p. 59140U.
  34. Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. Available online: http: //www.deeplearningbook.org (accessed on 28 March 2022).
  35. Olivieri, M.; Pezzoli, M.; Antonacci, F.; Sarti, A. A Physics-Informed Neural Network Approach for Nearfield Acoustic Holography. Sensors 2021, 21, 7834. [CrossRef]
  36. Bianco, M.J.; Gerstoft, P.; Traer, J.; Ozanich, E.; Roch, M.A.; Gannot, S.; Deledalle, C.A. Machine learning in acoustics: Theory and applications. J. Acoust. Soc. Am. (JASA) 2019, 146, 3590-3628. [CrossRef] [PubMed]
  37. Olivieri, M.; Malvermi, R.; Pezzoli, M.; Zanoni, M.; Gonzalez, S.; Antonacci, F.; Sarti, A. Audio Information Retrieval and Musical Acoustics. IEEE Instrum. Meas. Mag. 2021, 24, 10-20. [CrossRef]
  38. Olivieri, M.; Pezzoli, M.; Malvermi, R.; Antonacci, F.; Sarti, A. Near-field Acoustic Holography analysis with Convolutional Neural Networks. In Proceedings of the INTER-NOISE and NOISE-CON Congress and Conference Proceedings, Seoul, Korea, 23-26 August 2020; Volume 261, pp. 5607-5618.
  39. Campagnoli, C.; Pezzoli, M.; Antonacci, F.; Sarti, A. Vibrational modal shape interpolation through convolutional auto encoder. In Proceedings of the INTER-NOISE and NOISE-CON Congress and Conference Proceedings, Seoul, Korea, 23-26 August 2020; Volume 261, pp. 5619-5626.
  40. Lluís, F.; Martínez-Nuevo, P.; Bo Møller, M.; Ewan Shepstone, S. Sound field reconstruction in rooms: Inpainting meets super-resolution. J. Acoust. Soc. Am. 2020, 148, 649-659. [CrossRef] [PubMed]
  41. Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention; Springer: Berlin/Heidelberg, Germany, 2015; pp. 234-241.
  42. Bertalmio, M.; Sapiro, G.; Caselles, V.; Ballester, C. Image inpainting. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA, USA, 23-28 July 2000; pp. 417-424.
  43. Lim, B.; Son, S.; Kim, H.; Nah, S.; Lee, K.M. Enhanced Deep Residual Networks for Single Image Super-Resolution. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA, 21-26 July 2017; pp. 1132-1140.
  44. Ulyanov, D.; Vedaldi, A.; Lempitsky, V. Deep image prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18-23 June 2018; pp. 9446-9454.
  45. Dittmer, S.; Kluth, T.; Maass, P.; Baguer, D.O. Regularization by architecture: A deep prior approach for inverse problems. J. Math. Imaging Vis. 2020, 62, 456-470. [CrossRef]
  46. Kong, F.; Lipari, V.; Picetti, F.; Bestagini, P.; Tubaro, S. A deep prior convolutional autoencoder for seismic data interpolation. In Proceedings of the EAGE 2020 Annual Conference & Exhibition Online, European Association of Geoscientists & Engineers, Online, 8-11 December 2020; pp. 1-5.
  47. Picetti, F.; Lipari, V.; Bestagini, P.; Tubaro, S. Anti-Aliasing Add-On for Deep Prior Seismic Data Interpolation. arXiv 2021, arXiv:2101.11361.
  48. Kong, F.; Picetti, F.; Lipari, V.; Bestagini, P.; Tang, X.; Tubaro, S. Deep Prior-Based Unsupervised Reconstruction of Irregularly Sampled Seismic Data. IEEE Geosci. Remote Sens. Lett. 2020, 19, 7501305. [CrossRef]
  49. Malvermi, R.; Antonacci, F.; Sarti, A.; Corradi, R. Prediction of Missing Frequency Response Functions through Deep Image Prior. In Proceedings of 2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA, 17-20 October 2021.
  50. Michelashvili, M.; Wolf, L. Audio denoising with deep network priors. arXiv 2019, arXiv:1904.07612
  51. Ibtehaz, N.; Rahman, M.S. MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw. 2020, 121, 74-87. [CrossRef]
  52. Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7-12 June 2015; pp. 1-9. [CrossRef]
  53. Maas, A.L.; Hannun, A.Y.; Ng, A.Y. Rectifier nonlinearities improve neural network acoustic models. Proc. ICML Citeseer 2013, 30, 3.
  54. Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32; Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., Garnett, R., Eds.; Curran Associates, Inc.: Red Hook, NY, USA, 2019; pp. 8024-8035.
  55. Kingma, D.; Ba, J. Adam: A Method for Stochastic Optimization, 3rd International Conference on Learning Representations. arXiv 2014, arXiv:1412.6980.
  56. Pezzoli, M.; Comanducci, L.; Waltz, J.; Agnello, A.; Bondi, L.; Canclini, A.; Sarti, A. A Dante Powered Modular Microphone Array System. In Proceedings of the Audio Engineering Society Convention 145, Audio Engineering Society, New York, NY, USA, 17-20 October 2018. Available online: http://www.aes.org/e-lib/browse.cfm?elib=19743 (accessed on 28 March 2022).
  57. Gunda, R.; Vijayakar, S.; Singh, R. Method of images for the harmonic response of beams and rectangular plates. J. Sound Vib. 1995, 185, 791-808. [CrossRef]
  58. Scheibler, R.; Bezzam, E.; Dokmanić, I. Pyroomacoustics: A python package for audio room simulation and array processing algorithms. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada, 15-20 April 2018; pp. 351-355.

Related papers

Implicit neural representation with physics-informed neural networks for the reconstruction of the early part of room impulse responses
Mirco Pezzoli

arXiv (Cornell University), 2023

Recently deep learning and machine learning approaches have been widely employed for various applications in acoustics. Nonetheless, in the area of sound field processing and reconstruction classic methods based on the solutions of wave equation are still widespread. Recently, physics-informed neural networks have been proposed as a deep learning paradigm for solving partial differential equations which govern physical phenomena, bridging the gap between purely data-driven and model based methods. Here, we exploit physics-informed neural networks to reconstruct the early part of missing room impulse responses in an uniform linear array. This methodology allows us to exploit the underlying law of acoustics, i.e., the wave equation, forcing the neural network to generate physically meaningful solutions given only a limited number of data points. The results on real measurements show that the proposed model achieves accurate reconstruction and performance in line with respect to state-of-the-art deep-learning and compress sensing techniques while maintaining a lightweight architecture.

View PDFchevron_right
Room Impulse Response Reconstruction Based on Spatio-Temporal-Spectral Features Learned from a Spherical Microphone Array Measurement
Thushara Abhayapala

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Large-scale Room Impulse Response (RIR) measurements are required to accurately determine a room's acoustic response to different source-listener configurations. RIR reconstruction methods are often used to reduce these measurement costs. Prior knowledge of room acoustic parameters can ensure reliable and robust RIR reconstruction. This paper proposes a method to reconstruct RIRs based on reflection source locations and time-frequency-directiondependent reflection magnitude response estimated from a single spherical microphone array measurement. These input parameters are learned by applying the eigenbeam spatial correlation method and von Mises-Fisher (vMF)-based directivity modeling. According to the performance evaluation, the composition of the learned features in the RIR reconstruction formulation successfully preserves the objective characteristics of the real room.

View PDFchevron_right
dEchorate: a calibrated room impulse response dataset for echo-aware signal processing
Sharon Gannot

EURASIP Journal on Audio, Speech, and Music Processing, 2021

This paper presents a new dataset of measured multichannel room impulse responses (RIRs) named dEchorate. It includes annotations of early echo timings and 3D positions of microphones, real sources, and image sources under different wall configurations in a cuboid room. These data provide a tool for benchmarking recent methods in echo-aware speech enhancement, room geometry estimation, RIR estimation, acoustic echo retrieval, microphone calibration, echo labeling, and reflector position estimation. The dataset is provided with software utilities to easily access, manipulate, and visualize the data as well as baseline methods for echo-related tasks.

View PDFchevron_right
AV-RIR: Audio-Visual Room Impulse Response Estimation
Anton Ratnarajah

arXiv (Cornell University), 2023

Accurate estimation of Room Impulse Response (RIR), which captures an environment's acoustic properties, is important for speech processing and AR/VR applications. We propose AV-RIR, a novel multi-modal multi-task learning approach to accurately estimate the RIR from a given reverberant speech signal and the visual cues of its corresponding environment. AV-RIR builds on a novel neural codec-based architecture that effectively captures environment geometry and materials properties and solves speech dereverberation as an auxiliary task by using multi-task learning. We also propose Geo-Mat features that augment material information into visual cues and CRIP that improves late reverberation components in the estimated RIR via image-to-RIR retrieval by 86%. Empirical results show that AV-RIR quantitatively outperforms previous audio-only and visual-only approaches by achieving 36% -63% improvement across various acoustic metrics in RIR estimation. Additionally, it also achieves higher preference scores in human evaluation. As an auxiliary benefit, dereverbed speech from AV-RIR shows competitive performance with the state-of-the-art in various spoken language processing tasks and outperforms reverberation time error score in the real-world AVSpeech dataset. Qualitative examples of both synthesized reverberant speech and enhanced speech are available online 1 .

View PDFchevron_right
dEchorate: a Calibrated Room Impulse Response Database for Echo-aware Signal Processing
Sharon Gannot

arXiv (Cornell University), 2021

This paper presents dEchorate: a new database of measured multichannel Room Impulse Responses (RIRs) including annotations of early echo timings and 3D positions of microphones, real sources and image sources under different wall configurations in a cuboid room. These data provide a tool for benchmarking recent methods in echo-aware speech enhancement, room geometry estimation, RIR estimation, acoustic echo retrieval, microphone calibration, echo labeling and reflectors estimation. The database is accompanied with software utilities to easily access, manipulate and visualize the data as well as baseline methods for echo-related tasks.

View PDFchevron_right

Related topics

  • Computer Science
  • Artificial Intelligence
  • Analytical Chemistry
  • Medicine
  • Robustness (evolution)
  • Electrical and Electronic Engine...

    • Cited by

    Grad-CAM-Inspired Interpretation of Nearfield Acoustic Holography using Physics-Informed Explainable Neural Network
    Sharon Gannot

    ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    The interpretation and explanation of decision-making processes of neural networks are becoming a key factor in the deep learning field. Although several approaches have been presented for classification problems, the application to regression models needs to be further investigated. In this manuscript we propose a Grad-CAM-inspired approach for the visual explanation of neural network architecture for regression problems. We apply this methodology to a recent physics-informed approach for Nearfield Acoustic Holography, called Kirchhoff-Helmholtz-based Convolutional Neural Network (KHCNN) architecture. We focus on the interpretation of KHCNN using vibrating rectangular plates with different boundary conditions and violin top plates with complex shapes. Results highlight the more informative regions of the input that the network exploits to correctly predict the desired output. The devised approach has been validated in terms of NCC and NMSE using the original input and the filtered one coming from the algorithm.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved