Academia.eduAcademia.edu

Outline

Title
Abstract
FAQs
Figures
Pitch Detection (F 0 ) by Time Domain Methods
Frequency Domain Analysis
Analysis via Mel-Cepstrum
Auditory Model for Sound Analysis
Auditory Analysis: Seneff's Model
Case Study
Cues from Midi Data
Perception-Based Music Analysis
Standard Spectrum Method
Analysis of the RMS
Introduction
MPEG-7 Objectives
Scope of the Standard
Making Audio-Visual Material as Searchable as Text
References

From audio to content

Profile image of Giovanni De PoliGiovanni De Poli

2014

Abstract

Models for the representation the information from sound are necessary for the description of the in-formation, from the perceptive and operative point of view. Beyond the models, analysis methods are needed to discover the parameters which allow sound description, possibly lossless from the physical and perceptual properties description. When aiming to the extraction of information for sound, we need to discard every feature which is non relevant. This process of feature extraction consists of var-ious steps, starting from pre-processing the sound, then windowing, extraction, and post-processing procedures. An audio signal classification system can be generally represented as represented in Figure 4.1. Pre-processing: The pre-processing consists of noise reduction, equalization, low-pass filtering. In speech processing (voice has a low-pass behavior) a pre-emphasis is applied by high-pass fil-tering the signal to smooth the spectrum, to achieve an uniform energy distribution spect...

FAQs

sparkles

AI

What factors influence the choice of window functions in short-time signal analysis?add

The choice of window functions is influenced by the need for short stationarity assumption, noise reduction, and complete parameter coverage based on frame rate, indicating a balance between duration and smoothness.

How does frame rate affect the temporal accuracy in audio signal processing?add

The frame rate, defined as the number of frames computed per second, directly impacts temporal accuracy by determining the hopsize H and thus the resolution of the time-frequency analysis.

What methods are used for pitch detection in audio signals?add

Pitch detection methods utilize the Short-Time Autocorrelation Function and Short-Time Average Magnitude Difference Function (AMDF), identifying the first maximum or minimum after k=0 to accurately estimate the fundamental frequency F0.

How does Short-Time Average Energy differ from Average Magnitude in signal analysis?add

Short-Time Average Energy is sensitive to large signal events, while Average Magnitude addresses this issue by focusing on amplitude, providing a more stable measure in energy-affected scenarios.

What insights does the Zero-Crossing Rate (ZCR) provide regarding audio signals?add

ZCR indicates the fundamental frequency of narrow-band signals; it counts the number of times the signal crosses zero, linking to low computational cost and efficient frequency estimation.

Figures (42)
Figure 4.1: Scheme for supervised classification.
Figure 4.1: Scheme for supervised classification.
Figure 4.2: Various windows on the left; on the right, three windows over the signals(n), shifted from origin by 2N, 3N e 4N samples
Figure 4.2: Various windows on the left; on the right, three windows over the signals(n), shifted from origin by 2N, 3N e 4N samples
Figure 4.3: On the top: a short excerpt from violin performance of Handel.... Other diagrams rep- resent normalized Short-Time Average Energy and Short-Time Average Magnitude , computed using rectangular windows with N=100 samples and frame rate equal to signal sampling rate (8kHz). Write two MATLAB functions to compute Short-Time Average Energy e Magnitude.
Figure 4.3: On the top: a short excerpt from violin performance of Handel.... Other diagrams rep- resent normalized Short-Time Average Energy and Short-Time Average Magnitude , computed using rectangular windows with N=100 samples and frame rate equal to signal sampling rate (8kHz). Write two MATLAB functions to compute Short-Time Average Energy e Magnitude.
Figure 4.4: On the top: a short excerpt from violin performance of Handel.... Other diagrams represent Short-Time Average Energy computed with different Hamming windows.
Figure 4.4: On the top: a short excerpt from violin performance of Handel.... Other diagrams represent Short-Time Average Energy computed with different Hamming windows.
Cepstrum method allows the separation of a signal y(n) = x(n) * h(n), (source-filter model), where the source x(n) passes through a filtered described by impulse response h(n). Signal spectrum y(n) results Y(k) = X(k) - H(k), which is the product of two spectrums (k is the discrete-frequencies index). The former is related to the source spectrum, and the latter to the filter spectrum. It’s pretty difficult to separate these two spectrums, thus what is usually done is to extract the envelope (real) of the filter, and making the phase related to the source only. Cesptrum idea is based on the properties of logarithms : log(a - b) = log(a) + log(b). Taking into account the logarithm of the absolute value of spectrum Y (k), we get:
Cepstrum method allows the separation of a signal y(n) = x(n) * h(n), (source-filter model), where the source x(n) passes through a filtered described by impulse response h(n). Signal spectrum y(n) results Y(k) = X(k) - H(k), which is the product of two spectrums (k is the discrete-frequencies index). The former is related to the source spectrum, and the latter to the filter spectrum. It’s pretty difficult to separate these two spectrums, thus what is usually done is to extract the envelope (real) of the filter, and making the phase related to the source only. Cesptrum idea is based on the properties of logarithms : log(a - b) = log(a) + log(b). Taking into account the logarithm of the absolute value of spectrum Y (k), we get:
Figure 4.6: Automatically formant estimation from cepstrally smooted log Spectra [from Schaefer Rabiner].
Figure 4.6: Automatically formant estimation from cepstrally smooted log Spectra [from Schaefer Rabiner].
Figure 4.7: (a) Trasformation from Hz to mel. (b) Mel-scale filterbank. below 1000 Hz is 200 Hz; then it will raise exponentially. Mel-cesptrum aim to estimate the spectral envelope of this filterbank output. When Y, is the logarithm of energy exiting from channel n, we can use the discrete time cosine transform DCT to obtain the mel-cepstral coefficients MFCC (mel frequency cepstral coefficients) by means of following equation:
Figure 4.7: (a) Trasformation from Hz to mel. (b) Mel-scale filterbank. below 1000 Hz is 200 Hz; then it will raise exponentially. Mel-cesptrum aim to estimate the spectral envelope of this filterbank output. When Y, is the logarithm of energy exiting from channel n, we can use the discrete time cosine transform DCT to obtain the mel-cepstral coefficients MFCC (mel frequency cepstral coefficients) by means of following equation:
Figure 4.8: Example of mel-cesptrum analysis of a clarinet tone: tone spectrum (high left); spectral envelope estimated with mel cepstrum (high right); spectral envelope estimated with LPC (low Ikeft); spectral envelope reconstructed with first 6 mel cepstral coefficients (low right).
Figure 4.8: Example of mel-cesptrum analysis of a clarinet tone: tone spectrum (high left); spectral envelope estimated with mel cepstrum (high right); spectral envelope estimated with LPC (low Ikeft); spectral envelope reconstructed with first 6 mel cepstral coefficients (low right).
Figure 4.9: Block diagram of the joint Synchrony/Mean-Rate model of Auditory Speech Processing.
Figure 4.9: Block diagram of the joint Synchrony/Mean-Rate model of Auditory Speech Processing.
Figure 4.10: Block diagram of the 40-channel critical-band linear filter bank
Figure 4.10: Block diagram of the 40-channel critical-band linear filter bank
Figure 4.11: Frequency responses of the 40-channel critical-band linear filter bank.
Figure 4.11: Frequency responses of the 40-channel critical-band linear filter bank.
Figure 4.12: Mathematical framework of the joint Synchrony/Mean-Rate model of Auditory Speech Processing.
Figure 4.12: Mathematical framework of the joint Synchrony/Mean-Rate model of Auditory Speech Processing.
Figure 4.13: Result of the application of the four modules implementing the hair-cell synapse model to a simple 1000 Hz sinusoid. Left and right plots refer to the global 60 ms stimulus and to its corresponding first 10 ms window, in different positions along the model.
Figure 4.13: Result of the application of the four modules implementing the hair-cell synapse model to a simple 1000 Hz sinusoid. Left and right plots refer to the global 60 ms stimulus and to its corresponding first 10 ms window, in different positions along the model.
Figure 4.14: Block diagram of the Generalized Synchrony Detector (GSD) module.
Figure 4.14: Block diagram of the Generalized Synchrony Detector (GSD) module.
Figure 4.15: Output of the model, as applied to a clean B Clarinet sound: (a) envelope, (b) synchrony.
Figure 4.15: Output of the model, as applied to a clean B Clarinet sound: (a) envelope, (b) synchrony.
Figure 4.16: Synchrony parameter output of the analysis of the same B Clarinet of Fig 7b, superin- posed with a gaussian random noise at a level of 5 db S/N ratio.
Figure 4.16: Synchrony parameter output of the analysis of the same B Clarinet of Fig 7b, superin- posed with a gaussian random noise at a level of 5 db S/N ratio.
Zero-crossing rate is a measure of times the signal crosses the zero axe (see Sec. 4.1.1.4). Noisy sounds tend to have high values of ZCR.
Zero-crossing rate is a measure of times the signal crosses the zero axe (see Sec. 4.1.1.4). Noisy sounds tend to have high values of ZCR.
This book is available under the CreativeCommons Attribution-NonC ommercial- ShareAlike licence, www.creativecommons.org. ©2006 by the authors where a, is the amplitude of 7-th partial; Total amplitude of the residual component, resulting by sum of absolute values of the residual within the frame:
This book is available under the CreativeCommons Attribution-NonC ommercial- ShareAlike licence, www.creativecommons.org. ©2006 by the authors where a, is the amplitude of 7-th partial; Total amplitude of the residual component, resulting by sum of absolute values of the residual within the frame:
Figure 4.18: Features and extraction processes as proposed by Cuidado Project for MPEG-7 sound descriptors. Weight of i-th harmonic with reference to the total sinusoidal component:
Figure 4.18: Features and extraction processes as proposed by Cuidado Project for MPEG-7 sound descriptors. Weight of i-th harmonic with reference to the total sinusoidal component:
This book is available under the CreativeC ommons Attribution-NonC ommercial- ShareAlike licence, www.creativecommons.org. ©2006 by the authors Spectral slope represents the amount of decreasing of the spectral amplitude. It is computed by linear regression of the spectral amplitude. Spectral skewness gives a measure of the asymmetry of a distribution around its mean value. is the spread of the spectrum around its mean value, i.e. the variance of the spectral distribution
This book is available under the CreativeC ommons Attribution-NonC ommercial- ShareAlike licence, www.creativecommons.org. ©2006 by the authors Spectral slope represents the amount of decreasing of the spectral amplitude. It is computed by linear regression of the spectral amplitude. Spectral skewness gives a measure of the asymmetry of a distribution around its mean value. is the spread of the spectrum around its mean value, i.e. the variance of the spectral distribution
Figure 4.19: Example of onset detector based on local energy: time-domain audio signal (a), 40ms windowed RMS smoothing with 75% overlap (b), peaks in slope of envelope (c).
Figure 4.19: Example of onset detector based on local energy: time-domain audio signal (a), 40ms windowed RMS smoothing with 75% overlap (b), peaks in slope of envelope (c).
Figure 4.20: fo temporal variation (with a window and one step with the values above considered) calculated by standard spectrum for the 5 first seconds of a considered signal. In this type of study it is verified the variation of the spectral shape in respect to the first maximum of the FFT’.
Figure 4.20: fo temporal variation (with a window and one step with the values above considered) calculated by standard spectrum for the 5 first seconds of a considered signal. In this type of study it is verified the variation of the spectral shape in respect to the first maximum of the FFT’.
Figure 4.21: RMS of a5 second window of signal.The blue line represents the R/S temporal variation, while that the red line represents the dynamic threshold that follows the RMS of the signal. The crosses on signal R/S represent the detected onsets. The circles represent the zones of effective onsets.
Figure 4.21: RMS of a5 second window of signal.The blue line represents the R/S temporal variation, while that the red line represents the dynamic threshold that follows the RMS of the signal. The crosses on signal R/S represent the detected onsets. The circles represent the zones of effective onsets.
Figure 4.23: Detection of onsets for a five seconds window: the last figure represents the onset detec- tion, where red lines indicate the instants for each detected onset.
Figure 4.23: Detection of onsets for a five seconds window: the last figure represents the onset detec- tion, where red lines indicate the instants for each detected onset.
Figure 4.24: Mpeg-4 high level system architecture.
Figure 4.24: Mpeg-4 high level system architecture.
Figure 4.25: An example of an MPEG-4 scene.
Figure 4.25: An example of an MPEG-4 scene.
Figure 4.26: Objects in a scene (a) and the corresponding BInary Format for Scene (BIFS) represen- tation (b). the nodes and graph leaves that require streaming support to retrieve media contents or ancillary data (video stream, audio stream, facial animation parameters) are logically connected to the decoding pipelines.
Figure 4.26: Objects in a scene (a) and the corresponding BInary Format for Scene (BIFS) represen- tation (b). the nodes and graph leaves that require streaming support to retrieve media contents or ancillary data (video stream, audio stream, facial animation parameters) are logically connected to the decoding pipelines.
Figure 4.27: Major components of an MPEG-4 terminal (receiver side).
Figure 4.27: Major components of an MPEG-4 terminal (receiver side).
Figure 4.28: Major components of an MPEG-4 terminal (receiver side).
Figure 4.28: Major components of an MPEG-4 terminal (receiver side).
Figure 4.29: In a traditional audio coder, the source model and perception model are defined outside of the transmission (for example, in a standards document). The codec designers do the best job they can at designing these models, but then they are fixed for all content (a). In Structured Audio, the source model is part of the content. It is transmitted in the bitstream and used to give different semantics to the signal representation for each piece of content. There can be a different source model, or multiple source models, for each different piece of content (b). Figure 4.29: In a traditional audio coder, the source model and perception model are defined outside of
Figure 4.29: In a traditional audio coder, the source model and perception model are defined outside of the transmission (for example, in a standards document). The codec designers do the best job they can at designing these models, but then they are fixed for all content (a). In Structured Audio, the source model is part of the content. It is transmitted in the bitstream and used to give different semantics to the signal representation for each piece of content. There can be a different source model, or multiple source models, for each different piece of content (b). Figure 4.29: In a traditional audio coder, the source model and perception model are defined outside of
refer to text in addition to its audio-visual information. MPEG-7 therefore has standardized different Description Tools for textual annotation and controlled vocabularies, taking into account existing standards and practices.
refer to text in addition to its audio-visual information. MPEG-7 therefore has standardized different Description Tools for textual annotation and controlled vocabularies, taking into account existing standards and practices.
Figure 4.34: Abstract representation of possible applications using MPEG-7.
Figure 4.34: Abstract representation of possible applications using MPEG-7.
Figure 4.35: Abstract representation of possible relation between Descriptors and Description Schemes.
Figure 4.35: Abstract representation of possible relation between Descriptors and Description Schemes.
Figure 4.36: Abstract representation of possible applications using MPEG-7.
Figure 4.36: Abstract representation of possible applications using MPEG-7.
Figure 4.38: AudioSpectrumEnvelope description of a pop song. The required data storage is NM values where NV is the number of spectrum bins and / is the number of time points.
Figure 4.38: AudioSpectrumEnvelope description of a pop song. The required data storage is NM values where NV is the number of spectrum bins and / is the number of time points.
Figure 4.39: A 10-basis component reconstruction showing most of the detail of the original spectro- gram including guitar, bass guitar, hi-hat and organ notes. The left vectors are an AudioSpectrumBasis Descriptor and the top vectors are the corresponding AudioSpectrumProjection Descriptor. The re- quired data storage is 10(M + N) values. high-level tools that exchange some degree of generality for descriptive richness. The five sets of audio Description Tools that roughly correspond to application areas are integrated in the standard: audio signature, musical instrument timbre, melody description, general sound recognition and indexing, and spoken content. The latter two are excellent examples of how the Audio Framework and MDS Description Tools may be integrated to support other applications.
Figure 4.39: A 10-basis component reconstruction showing most of the detail of the original spectro- gram including guitar, bass guitar, hi-hat and organ notes. The left vectors are an AudioSpectrumBasis Descriptor and the top vectors are the corresponding AudioSpectrumProjection Descriptor. The re- quired data storage is 10(M + N) values. high-level tools that exchange some degree of generality for descriptive richness. The five sets of audio Description Tools that roughly correspond to application areas are integrated in the standard: audio signature, musical instrument timbre, melody description, general sound recognition and indexing, and spoken content. The latter two are excellent examples of how the Audio Framework and MDS Description Tools may be integrated to support other applications.

Related papers

Automatic Sound Classification Inspired by Auditory Scene Analysis
Stefan Launer

2000

A sound classification system for the automatic recognition of the acoustic environment in a hearing instrument is dis- cussed. The system distinguishes the four sound classes 'clean speech', 'speech in noise', 'noise', and 'music' and is based on auditory features and hidden Markov models. The em- ployed features describe level fluctuations, the spectral form and harmonicity. Sounds from a large

View PDFchevron_right
Audio Signal Feature Extraction and Classification Using Local Discriminant Bases
shivani sharma

—Audio feature extraction plays an important role in analyzing and characterizing audio content. Auditory scene analysis , content-based retrieval, indexing, and fingerprinting of audio are few of the applications that require efficient feature extraction. The key to extract strong features that characterize the complex nature of audio signals is to identify their discriminatory sub-spaces. In this paper, we propose an audio feature extraction and a multigroup classification scheme that focuses on identifying discriminatory time-frequency subspaces using the local discriminant bases (LDB) technique. Two dissimilarity measures were used in the process of selecting the LDB nodes and extracting features from them. The extracted features were then fed to a linear discriminant analysis-based classifier for a three-level hierarchical classification of audio signals into ten classes. In the first level, the audio signals were grouped into artificial and natural sounds. Each of the first level groups were subdivided to form the second level groups viz. instrumental, automobile, human, and nonhuman sounds. The third level was formed by subdividing the four groups of the second level into the final ten groups (drums, flute, piano, aircraft, helicopter , male, female, animals, birds and insects). A database of 213 audio signals were used in this study and an average classification accuracy of 83% for the first level (113 artificial and 100 natural sounds), 92% for the second level (73 instrumental and 40 automobile sounds; 40 human and 60 nonhuman sounds), and 89% for the third level (27 drums, 15 flute, and 31 piano sounds; 23 aircraft and 17 helicopter sounds; 20 male and 20 female speech; 20 animals , 20 birds and 20 insects sounds) were achieved. In addition to the above, a separate classification was also performed combining the LDB features with the mel-frequency cepstral coefficients. The average classification accuracies achieved using the combined features were 91% for the first level, 99% for the second level, and 95% for the third level. Index Terms—Audio classification, dissimilarity measures, feature extraction, linear discriminant analysis (LDA), local discrim-inant bases (LDB), wavelet packets.

View PDFchevron_right
Feature Analysis for Audio Classification
Daniel Acevedo

Lecture Notes in Computer Science, 2014

In this work we analyze and implement several audio features. We emphasize our analysis on the ZCR feature and propose a modification making it more robust when signals are near zero. They are all used to discriminate the following audio classes: music, speech, environmental sound. An SVM classifier is used as a classification tool, which has proven to be efficient for audio classification. By means of a selection heuristic we draw conclusions of how they may be combined for fast classification.

View PDFchevron_right
A proposal for the description of audio in the context of MPEG-7
Perfecto Herrera

Proceedings of the CBMI'99 European …, 1999

Sound content description is one of the aims of the MPEG-7 initiative. Although MPEG-7 focuses on indexing and retrieval of audio, there are other sound content-based processing applications waiting to be developed once we have a robust set of descriptors and structures for putting them into relation, and for expressing semantic concerns about sound. Spectral Modeling techniques provide one usable framework for extracting and organizing sound content descriptions. In this paper we will introduce one particular approach to spectral modeling, then we will present some sound descriptors that can be derived from them in order to develop sound descriptions, and we will discuss the features of a structure for organizing the information that can be derived from them (a so called "Description Scheme"). All of our current descriptors can be considered low-or mid-level, thus we will not cover the high level description of music (musical forms and styles, roles of characters in a movie, etc.) which is also relevant in MPEG-7 indeed. The descriptors proposed are the result of a sound analysis based on a spectral modeling technique, and for all of them we have devised automatic extraction procedures. The Description Scheme we present is intended to be a generic one that, based on a hierarchical (and recursive in some places) structure, can describe sound at multiple levels of detail, addressing both syntactic (structural) and semantic (content) ways for describing sound.

View PDFchevron_right
Review on Design and Implementation of Audio Signal Classification System to classify the Media in Speech/Music
IRJET Journal

Over the last few years exceeding efforts have been made to develop methods for extracting information from audiovisual media, mandate that they may be stored and retrieved in databases automatically. Audio classification serves as the fundamental step towards the quickly growth in audio data volume. Automatic audio classification is very useful in content based audio retrieval and online audio distribution. The accuracy of the classification relies on the efficacy of the features and classification scheme. In this work both, time domain and frequency domain features are extracted from the input signal. Time domain feature is Root Mean Square (RMS). Frequency domain feature is spectral flux. After feature extraction, classification will be. The selection of the important features is explained as well as the classifiers used for classification are compared.

View PDFchevron_right
Sound recognition: a connectionist approach
Hadi Harb

2003

This paper presents a general audio classification approach inspired by our modest knowledge about the human perception of sound. Simple psychoacoustic experiments show that the relation between short term spectral features has a great impact on the human audio classification performance. For instance, short term spectral features extracted from speech sound can be perceived as non-speech sounds if organized in a special way in time. We have developed the idea of incorporating several consecutive spectral features when modelling the audio signal in relatively long term time windows. The modelling scheme that we propose, piecewise Gaussian modelling (PGM), was combined with a neural network to develop a general audio classifier. The classifier was evaluated on the problems of speech/music classification. male/female classification and special events detection in sports videos. The good classification accuracy obtained by the classifier suggests us to continue the research in order to improve the model and to closely combine it to some well-known psychoacoustic experimental results.

View PDFchevron_right
Audio Signal Processing
IJRASET Publication

International Journal for Research in Applied Science & Engineering Technology (IJRASET), 2022

Audio Signal Processing is also known as Digital Analog Conversion (DAC). Sound waves are the most common example of longitudinal waves. The speed of sound waves is a particular medium depends on the properties of that temperature and the medium. Sound waves travel through air when the air elements vibrate to produce changes in pressure and density along the direction of the wave's motion. It transforms the Analog Signal into Digital Signals, and then converted Digital Signals is sent to the Devices. Which can be used in Various things., Such as audio signal, RADAR, speed processing, voice recognition, entertainment industry, and to find defected in machines using audio signals or frequencies. The signals pay important role in our day-today communication, perception of environment, and entertainment. A joint time-frequency (TF) approach would be better choice to effectively process this signal. The theory of signal processing and its application to audio was largely developed at Bell Labs in the mid-20 th century. Claude Shannon and Harry Nyquist's early work on communication theory and pulse-code modulation (PCM) laid the foundations for the field.

View PDFchevron_right
Features Based on Auditory Physiology and Perception
Richard Stern

Techniques for Noise Robustness in Automatic Speech Recognition, 2012

It is well known that human speech processing capabilities far surpass the capabilities of current automatic speech recognition and related technologies, despite very intensive research in automated speech technologies in recent decades. Indeed, since the early 1980's, this observation has motivated the development of speech recognition feature extraction approaches that are inspired by auditory processing and perception, but it is only relatively recently that these approaches have become effective in their application to computer speech processing. The goal of this chapter is to review some of the major ways in which feature extraction schemes based on auditory processing have facilitated greater speech recognition accuracy in recent years, as well as to provide some insight into the nature of current trends and future directions in this area. We begin this chapter with a brief review of some of the major physiological and perceptual phenomena that have motivated feature extraction algorithms based on auditory processing. We continue with a review and discussion of three seminal 'classical' auditory models of the 1980s that have had a major impact on the approaches taken by more recent contributors to this field. Finally, we turn our attention to selected more recent topics of interest in auditory feature analysis, along with some of the feature extraction approaches that have been based on them. We conclude with a discussion of the attributes of auditory models that appear to be most effective in improving speech recognition accuracy in difficult acoustic environments. 8.2 Some Attributes of Auditory Physiology and Perception In this section we very briefly review and discuss a selected set of attributes of auditory physiology that historically or currently have been the object of attention by developers of Techniques for Noise Robustness in Automatic Speech Recognition Virtanen, Singh, and Raj (eds)

View PDFchevron_right
Some Clues to Build a Sound Analysis Relevant to Hearing
Laurent Millot

Journal of The Audio Engineering Society, 2004

Analysis tools used in research laboratories, for sound synthesis, by musicians or sound engineers can be rather different. Discussion of the assumptions and of the limitations of these tools permits to propose a first tool as relevant and versatile as possible for all the sound actors with a major aim: one must be able to listen to each element of the analysis because hearing is the final reference tool. This tool should also be used, in the future, to reinvestigate the definition of sound (or Acoustics) on the basis of some recent works on musical instrument modeling, speech production and loudspeakers design. Audio illustrations will be given.

View PDFchevron_right
Sound Analysis and Processing with AudioSculpt 2
Axel Roebel

AudioSculpt is an application for the musical analysis and processing of sound files. The program allows very detailed study of a sound's spectrum, waveform, fundamental frequency and partial contents. Multiple algorithms provide automatic segmentation of sounds. All analyses can be edited, stored and used to guide processing within the application, such as spectral filtering, time-stretching and noise removal, or serve as input for compositional environments. The current version is a complete revision, introducing many new features, a real-time mode, new analysis, processing and segmentation algorithms plus a significantly enhanced user interface. The introduction of SDIF as data format for analysis data and the ability to use long and multichannel sound files, take the application to a new level of usability.

View PDFchevron_right

References (68)

  1. P. Bello, L. Daudet, S. Abdallah, C. Duxbury, M. Davies, and M. B. Sandler. A tutorial on onset detection in music signals. IEEE Transactions on Speech and Audio Processing, 13(5):1035-1047, 2005.
  2. L. Van Immerseel and J. Martens. Pitch and voiced/unvoiced determination with an auditory model. The Journal of the Acoustical Society of America, 91, 1992.
  3. A. Klapuri. Sound onset detection by applying psychoacoustic knowledge. In Proceedings of the IEEE Int. Conf. Acoust. (ICASSP), volume 6, pages 3089-3092, Phoenix, AR (USA), 1999.
  4. M. Leman. Visualization and calculation of roughness of acoustical musical signals using the synchronization index model (sim). In Proceedings of the of the COST G-6 Conference on Digital Audio Effects (DAFX-00), pages 125-130, Verona, Italy, 2000.
  5. M. Leman, M. Lesaffre, and K. Tanghe. A toolbox for perception-based music analyses. IPEM -Dept. Of Musicology, Ghent University, ??
  6. M. Leman, M. Lesaffre, and K. Tanghe. Introduction to the ipem toolbox. In Proceeding of the XIII Meeting of the FWO Research Society on Foundations of Music Research, 2001a.
  7. M. Leman, M. Lesaffre, and K. Tanghe. An introduction to the ipem toolbox for perception-based music analysis. Mikropolyphonie-The Online Contemporary Music Journal, 7, 2001b.
  8. G. Li and A. Khokar. Content-based indexing and retrieval of audio data using wavelets. In International Conference on Multimedia and Expo (II), page 885888. IEEE, 2000.
  9. L. Rabiner and B. H. Juang. Fundamentals of Speech Recognition. Prentice-Hall, 1993.
  10. E. Scheirer. Music-Listening Systems. PhD thesis, MIT, 2000.
  11. W. Schloss. On the Automatic Transcription of Percussive Music: From Acoustic Signal to High Level Analysis. PhD thesis, Stanford University, 1985.
  12. M. Slaney and R. Lyon. A perceptual pitch detector. In Proc. International Conference on Acoustics, Speech and Signal Processing (ICASSP), page 357360, Albuquerque, NM, 1990. IEEE.
  13. M. Slaney and R. Lyon. On the importance of time-a temporal representation of sound. In M. Cooke, B. Beet, and M. Crawford, editors, Visual Representations of Speech Signals, page 95116. John Wiley & Sons Ltd, 1993.
  14. G. Tzanetakis and P. Cook. Sound analysis using mpeg compressed audio. In International Conference on Acoustics, Speech and Signal Processing (ICASSP), Istanbul, 2000.
  15. G. Tzanetakis, G. Essl, and P. Cook. Audio analysis using the discrete wavelet transform. In Proceedings Conference in Acoustics and Music Theory Applications, 2001.
  16. 1 Sound Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 4.1.1 Time domain: Short-time analysis . . . . . . . . . . . . . . . . . . . . . . . 4.1
  17. 1.1.1 Windowing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2
  18. 1.1.2 Short-Time Average Energy and Magnitude . . . . . . . . . . . . 4.3
  19. 1.1.3 Temporal envelope estimation . . . . . . . . . . . . . . . . . . . . 4.5
  20. 1.1.4 Short-Time Average Zero-Crossing Rate . . . . . . . . . . . . . . 4.5
  21. 1.1.5 Short-Time Autocorrelation Function . . . . . . . . . . . . . . . . 4.6
  22. 1.1.6 Short-Time Average Magnitude Difference Function . . . . . . . . 4.7
  23. 1.1.7 Pitch detection (f 0 ) by time domain methods . . . . . . . . . . . . 4.8
  24. 1.2 Frequency domain analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.9 4.1.2.1 Spectral Envelope estimation . . . . . . . . . . . . . . . . . . . . 4.9
  25. 1.2.2 Spectral Envelope and Pitch estimation via Cepstrum . . . . . . . 4.9
  26. 1.2.3 Analysis via mel-cepstrum . . . . . . . . . . . . . . . . . . . . . 4.11
  27. 1.3 Auditory model for sound analysis . . . . . . . . . . . . . . . . . . . . . . . 4.13
  28. 1.3.1 Auditory modeling motivations . . . . . . . . . . . . . . . . . . . 4.13
  29. 1.3.2 Auditory analysis: Seneff's model . . . . . . . . . . . . . . . . . 4.15
  30. 2 Audio features for sound description . . . . . . . . . . . . . . . . . . . . . . . . . . 4.18 4.2.1 Temporal features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.20
  31. 2.1.1 ADSR envelop modelling . . . . . . . . . . . . . . . . . . . . . . 4.20
  32. 2.2 Energy features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.22
  33. 2.3 Spectral features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.24
  34. 2.4 Harmonic features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.25
  35. 2.5 Perceptual features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.26
  36. 2.6 Onset Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.26
  37. 2.6.1 Onset detection in frequency domain . . . . . . . . . . . . . . . . 4.27
  38. 2.6.2 Local Energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.27
  39. 2.7 A case study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.27
  40. 2.7.1 Cues from MIDI data . . . . . . . . . . . . . . . . . . . . . . . . 4.28
  41. 2.7.2 Auditory based cues . . . . . . . . . . . . . . . . . . . . . . . . . 4.28
  42. 2.7.3 Onset detection . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.30
  43. 2.7.4 Feature Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 4.35
  44. 3 Object description: MPEG-4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.37
  45. 3.1 Scope and features of the MPEG-4 standard . . . . . . . . . . . . . . . . . . 4.37
  46. 3.2 The utility of objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.38
  47. 3.3 Coded representation of media objects . . . . . . . . . . . . . . . . . . . . . 4.39
  48. 3.3.1 Composition of media objects . . . . . . . . . . . . . . . . . . . . 4.40
  49. 3.3.2 Description and synchronization of streaming data for media objects 4.44
  50. 3.4 MPEG-4 visual objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.44
  51. 3.5 MPEG-4 audio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.45
  52. 3.5.1 Natural audio . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.46
  53. 3.5.2 Synthesized audio . . . . . . . . . . . . . . . . . . . . . . . . . . 4.47
  54. 3.5.3 Sound spatialization . . . . . . . . . . . . . . . . . . . . . . . . . 4.51
  55. 3.5.4 Audio BIFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.51
  56. 4 Multimedia Content Description: Mpeg-7 . . . . . . . . . . . . . . . . . . . . . . . 4.53
  57. 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.53
  58. 4.1.1 Context of MPEG-7 . . . . . . . . . . . . . . . . . . . . . . . . . 4.53
  59. 4.1.2 MPEG-7 objectives . . . . . . . . . . . . . . . . . . . . . . . . . 4.54
  60. 4.2 MPEG-7 terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.56
  61. 4.3 Scope of the Standard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.57
  62. 4.4 MPEG-7 Applications Areas . . . . . . . . . . . . . . . . . . . . . . . . . . 4.60 4.4.4.1 Making audio-visual material as searchable as text . . . . . . . . . 4.62
  63. 4.4.2 Supporting push and pull information acquisition methods . . . . . 4.63
  64. 4.4.3 Enabling nontraditional control of information . . . . . . . . . . . 4.64
  65. 4.5 Mpeg-7 description tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.65
  66. 4.6 MPEG-7 Audio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.67
  67. 4.6.1 MPEG-7 Audio Description Framework . . . . . . . . . . . . . . 4.67
  68. 4.6.2 High-level audio description tools (Ds and DSs) . . . . . . . . . . 4.70

Related papers

AUDIO ANALYSIS Introduction to
Cleiton Santos
View PDFchevron_right
A computational model of auditory feature extraction and sound classification
Martin Coath

2005

Models of auditory processing, particularly of speech, face many difficulties. Included in these are variability among speakers, variability in speech rate, and robustness to moderate distortions such as time compression. We constructed a system based on ensembles of feature detectors derived from fragments of an onset-sensitive sound representation. This method is based on the idea of *spectro-temporal response fields' and uses convolution to measure the degree of similarity through time between the feature detectors and the stimulus. The output from the ensemble was used to derive segmentation cues and patterns of response, which were used to train an artificial neural network (ANN) classifier. This allowed us to estimate a lower bound for the mutual information between the class of the input and the class of the output. Our results suggest that there is significant information in the output of our system, and that this is robust with respect to the exact choice of feature set...

View PDFchevron_right
An Overview on Sound Features in Time and Frequency Domain
Constantin Constantinescu

International Journal of Advanced Statistics and IT&C for Economics and Life Sciences, 2023

Sound is the result of mechanical vibrations that set air molecules in motion, causing variations in air pressure that propagate as pressure waves. Represented as waveforms, these visual snapshots of sound reveal some of its characteristics. While waveform analysis offers limited insights, audio features provide a quantitative and structured way to describe sound, enabling data-driven analysis and interpretation. Different audio features capture various aspects of sound, facilitating a comprehensive understanding of the audio data. By leveraging audio features, machine learning models can be trained to recognize patterns, classify sounds, or make predictions, enabling the development of intelligent audio systems. Time-domain features, e.g., amplitude envelope, capture events from raw audio waveforms. Frequency domain features, like band energy ratio and spectral centroid, focus on frequency components, providing distinct information. In this paper, we will describe three time-domain and three frequency-domain features that we consider crucial and widely used. We will illustrate the suitability of each feature for specific tasks and draw general conclusions regarding the significance of sound features in the context of machine learning.

View PDFchevron_right
Methods of automatic audio content classification
Imane De

2007

This study presents an overview of different methods of digital signal processing and pattern recognition that are frequently applicable to automatic recognition, classification and description of audio content. Moreover, strategies for the combination of the said methods are discussed. Some of the published practical applications from different areas are cited to illustrate the use of the basic methods and the combined recognition strategies. A brief overview of human auditory perception is also given, with emphasis on the aspects that are important for audio recognition.

View PDFchevron_right
Computational Audio Analysis
Meinard Müller

Compared to traditional speech, music, or sound processing, the computational analysis of general audio data has a relatively young research history. In particular, the extraction of affective information (i. e., information that does not deal with the 'immediate' nature of the content such as the spoken words or note events) from audio signals has become an important research strand with a huge increase of interest in academia and industry. At an early stage of this novel research direction, many analysis techniques and representations were simply transferred from the speech domain to other audio domains. However, general audio signals (including their affective aspects) typically possess acoustic and structural characteristics that distinguish them from spoken language or isolated 'controlled' music or sound events. In the Dagstuhl Seminar 13451 titled " Computational Audio Analysis " we discussed the development of novel machine learning as well as signal processing techniques that are applicable for a wide range of audio signals and analysis tasks. In particular, we looked at a variety of sounds besides speech such as music recordings, animal sounds, environmental sounds, and mixtures thereof. In this report, we give an overview of the various contributions and results of the seminar. We start with an executive summary, which describes the main topics, goals, and group activities. Then, one finds a list of abstracts giving a more detailed overview of the participants' contributions as well as of the ideas and results discussed in the group meetings of our seminar. To conclude, an attempt is made to define the field as given by the views of the participants. Seminar 03.–08. November, 2013 – www.dagstuhl.de/13451 1998 ACM Subject Classification I.5.4 Applications—Waveform analysis, H.5.5 Sound and Music Computing, H.1.2 User/Machine Systems, H.5.1 Multimedia Information Systems With the rapid growth and omnipresence of digitized multimedia data, the processing, analysis, and understanding of such data by means of automated methods has become a

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved