Academia.eduAcademia.edu

Figure 27 – uploaded by Giovanni De Poli

See full PDFdownloadDownload figure
Figure 4.25: An example of an MPEG-4 scene.

Figure 4 25: An example of an MPEG-4 scene.

Related Figures (41)

Figure 4.1: Scheme for supervised classification.
Figure 4.1: Scheme for supervised classification.
Figure 4.2: Various windows on the left; on the right, three windows over the signals(n), shifted from origin by 2N, 3N e 4N samples
Figure 4.2: Various windows on the left; on the right, three windows over the signals(n), shifted from origin by 2N, 3N e 4N samples
Figure 4.3: On the top: a short excerpt from violin performance of Handel.... Other diagrams rep- resent normalized Short-Time Average Energy and Short-Time Average Magnitude , computed using rectangular windows with N=100 samples and frame rate equal to signal sampling rate (8kHz). Write two MATLAB functions to compute Short-Time Average Energy e Magnitude.
Figure 4.3: On the top: a short excerpt from violin performance of Handel.... Other diagrams rep- resent normalized Short-Time Average Energy and Short-Time Average Magnitude , computed using rectangular windows with N=100 samples and frame rate equal to signal sampling rate (8kHz). Write two MATLAB functions to compute Short-Time Average Energy e Magnitude.
Figure 4.4: On the top: a short excerpt from violin performance of Handel.... Other diagrams represent Short-Time Average Energy computed with different Hamming windows.
Figure 4.4: On the top: a short excerpt from violin performance of Handel.... Other diagrams represent Short-Time Average Energy computed with different Hamming windows.
Cepstrum method allows the separation of a signal y(n) = x(n) * h(n), (source-filter model), where the source x(n) passes through a filtered described by impulse response h(n). Signal spectrum y(n) results Y(k) = X(k) - H(k), which is the product of two spectrums (k is the discrete-frequencies index). The former is related to the source spectrum, and the latter to the filter spectrum. It’s pretty difficult to separate these two spectrums, thus what is usually done is to extract the envelope (real) of the filter, and making the phase related to the source only. Cesptrum idea is based on the properties of logarithms : log(a - b) = log(a) + log(b). Taking into account the logarithm of the absolute value of spectrum Y (k), we get:
Cepstrum method allows the separation of a signal y(n) = x(n) * h(n), (source-filter model), where the source x(n) passes through a filtered described by impulse response h(n). Signal spectrum y(n) results Y(k) = X(k) - H(k), which is the product of two spectrums (k is the discrete-frequencies index). The former is related to the source spectrum, and the latter to the filter spectrum. It’s pretty difficult to separate these two spectrums, thus what is usually done is to extract the envelope (real) of the filter, and making the phase related to the source only. Cesptrum idea is based on the properties of logarithms : log(a - b) = log(a) + log(b). Taking into account the logarithm of the absolute value of spectrum Y (k), we get:
Figure 4.6: Automatically formant estimation from cepstrally smooted log Spectra [from Schaefer Rabiner].
Figure 4.6: Automatically formant estimation from cepstrally smooted log Spectra [from Schaefer Rabiner].
Figure 4.7: (a) Trasformation from Hz to mel. (b) Mel-scale filterbank. below 1000 Hz is 200 Hz; then it will raise exponentially. Mel-cesptrum aim to estimate the spectral envelope of this filterbank output. When Y, is the logarithm of energy exiting from channel n, we can use the discrete time cosine transform DCT to obtain the mel-cepstral coefficients MFCC (mel frequency cepstral coefficients) by means of following equation:
Figure 4.7: (a) Trasformation from Hz to mel. (b) Mel-scale filterbank. below 1000 Hz is 200 Hz; then it will raise exponentially. Mel-cesptrum aim to estimate the spectral envelope of this filterbank output. When Y, is the logarithm of energy exiting from channel n, we can use the discrete time cosine transform DCT to obtain the mel-cepstral coefficients MFCC (mel frequency cepstral coefficients) by means of following equation:
Figure 4.8: Example of mel-cesptrum analysis of a clarinet tone: tone spectrum (high left); spectral envelope estimated with mel cepstrum (high right); spectral envelope estimated with LPC (low Ikeft); spectral envelope reconstructed with first 6 mel cepstral coefficients (low right).
Figure 4.8: Example of mel-cesptrum analysis of a clarinet tone: tone spectrum (high left); spectral envelope estimated with mel cepstrum (high right); spectral envelope estimated with LPC (low Ikeft); spectral envelope reconstructed with first 6 mel cepstral coefficients (low right).
Figure 4.9: Block diagram of the joint Synchrony/Mean-Rate model of Auditory Speech Processing.
Figure 4.9: Block diagram of the joint Synchrony/Mean-Rate model of Auditory Speech Processing.
Figure 4.10: Block diagram of the 40-channel critical-band linear filter bank
Figure 4.10: Block diagram of the 40-channel critical-band linear filter bank
Figure 4.11: Frequency responses of the 40-channel critical-band linear filter bank.
Figure 4.11: Frequency responses of the 40-channel critical-band linear filter bank.
Figure 4.12: Mathematical framework of the joint Synchrony/Mean-Rate model of Auditory Speech Processing.
Figure 4.12: Mathematical framework of the joint Synchrony/Mean-Rate model of Auditory Speech Processing.
Figure 4.13: Result of the application of the four modules implementing the hair-cell synapse model to a simple 1000 Hz sinusoid. Left and right plots refer to the global 60 ms stimulus and to its corresponding first 10 ms window, in different positions along the model.
Figure 4.13: Result of the application of the four modules implementing the hair-cell synapse model to a simple 1000 Hz sinusoid. Left and right plots refer to the global 60 ms stimulus and to its corresponding first 10 ms window, in different positions along the model.
Figure 4.14: Block diagram of the Generalized Synchrony Detector (GSD) module.
Figure 4.14: Block diagram of the Generalized Synchrony Detector (GSD) module.
Figure 4.15: Output of the model, as applied to a clean B Clarinet sound: (a) envelope, (b) synchrony.
Figure 4.15: Output of the model, as applied to a clean B Clarinet sound: (a) envelope, (b) synchrony.
Figure 4.16: Synchrony parameter output of the analysis of the same B Clarinet of Fig 7b, superin- posed with a gaussian random noise at a level of 5 db S/N ratio.
Figure 4.16: Synchrony parameter output of the analysis of the same B Clarinet of Fig 7b, superin- posed with a gaussian random noise at a level of 5 db S/N ratio.
Zero-crossing rate is a measure of times the signal crosses the zero axe (see Sec. 4.1.1.4). Noisy sounds tend to have high values of ZCR.
Zero-crossing rate is a measure of times the signal crosses the zero axe (see Sec. 4.1.1.4). Noisy sounds tend to have high values of ZCR.
This book is available under the CreativeCommons Attribution-NonC ommercial- ShareAlike licence, www.creativecommons.org. ©2006 by the authors where a, is the amplitude of 7-th partial; Total amplitude of the residual component, resulting by sum of absolute values of the residual within the frame:
This book is available under the CreativeCommons Attribution-NonC ommercial- ShareAlike licence, www.creativecommons.org. ©2006 by the authors where a, is the amplitude of 7-th partial; Total amplitude of the residual component, resulting by sum of absolute values of the residual within the frame:
Figure 4.18: Features and extraction processes as proposed by Cuidado Project for MPEG-7 sound descriptors. Weight of i-th harmonic with reference to the total sinusoidal component:
Figure 4.18: Features and extraction processes as proposed by Cuidado Project for MPEG-7 sound descriptors. Weight of i-th harmonic with reference to the total sinusoidal component:
This book is available under the CreativeC ommons Attribution-NonC ommercial- ShareAlike licence, www.creativecommons.org. ©2006 by the authors Spectral slope represents the amount of decreasing of the spectral amplitude. It is computed by linear regression of the spectral amplitude. Spectral skewness gives a measure of the asymmetry of a distribution around its mean value. is the spread of the spectrum around its mean value, i.e. the variance of the spectral distribution
This book is available under the CreativeC ommons Attribution-NonC ommercial- ShareAlike licence, www.creativecommons.org. ©2006 by the authors Spectral slope represents the amount of decreasing of the spectral amplitude. It is computed by linear regression of the spectral amplitude. Spectral skewness gives a measure of the asymmetry of a distribution around its mean value. is the spread of the spectrum around its mean value, i.e. the variance of the spectral distribution
Figure 4.19: Example of onset detector based on local energy: time-domain audio signal (a), 40ms windowed RMS smoothing with 75% overlap (b), peaks in slope of envelope (c).
Figure 4.19: Example of onset detector based on local energy: time-domain audio signal (a), 40ms windowed RMS smoothing with 75% overlap (b), peaks in slope of envelope (c).
Figure 4.20: fo temporal variation (with a window and one step with the values above considered) calculated by standard spectrum for the 5 first seconds of a considered signal. In this type of study it is verified the variation of the spectral shape in respect to the first maximum of the FFT’.
Figure 4.20: fo temporal variation (with a window and one step with the values above considered) calculated by standard spectrum for the 5 first seconds of a considered signal. In this type of study it is verified the variation of the spectral shape in respect to the first maximum of the FFT’.
Figure 4.21: RMS of a5 second window of signal.The blue line represents the R/S temporal variation, while that the red line represents the dynamic threshold that follows the RMS of the signal. The crosses on signal R/S represent the detected onsets. The circles represent the zones of effective onsets.
Figure 4.21: RMS of a5 second window of signal.The blue line represents the R/S temporal variation, while that the red line represents the dynamic threshold that follows the RMS of the signal. The crosses on signal R/S represent the detected onsets. The circles represent the zones of effective onsets.
Figure 4.23: Detection of onsets for a five seconds window: the last figure represents the onset detec- tion, where red lines indicate the instants for each detected onset.
Figure 4.23: Detection of onsets for a five seconds window: the last figure represents the onset detec- tion, where red lines indicate the instants for each detected onset.
Figure 4.24: Mpeg-4 high level system architecture.
Figure 4.24: Mpeg-4 high level system architecture.
Figure 4.26: Objects in a scene (a) and the corresponding BInary Format for Scene (BIFS) represen- tation (b). the nodes and graph leaves that require streaming support to retrieve media contents or ancillary data (video stream, audio stream, facial animation parameters) are logically connected to the decoding pipelines.
Figure 4.26: Objects in a scene (a) and the corresponding BInary Format for Scene (BIFS) represen- tation (b). the nodes and graph leaves that require streaming support to retrieve media contents or ancillary data (video stream, audio stream, facial animation parameters) are logically connected to the decoding pipelines.
Figure 4.27: Major components of an MPEG-4 terminal (receiver side).
Figure 4.27: Major components of an MPEG-4 terminal (receiver side).
Figure 4.28: Major components of an MPEG-4 terminal (receiver side).
Figure 4.28: Major components of an MPEG-4 terminal (receiver side).
Figure 4.29: In a traditional audio coder, the source model and perception model are defined outside of the transmission (for example, in a standards document). The codec designers do the best job they can at designing these models, but then they are fixed for all content (a). In Structured Audio, the source model is part of the content. It is transmitted in the bitstream and used to give different semantics to the signal representation for each piece of content. There can be a different source model, or multiple source models, for each different piece of content (b). Figure 4.29: In a traditional audio coder, the source model and perception model are defined outside of
Figure 4.29: In a traditional audio coder, the source model and perception model are defined outside of the transmission (for example, in a standards document). The codec designers do the best job they can at designing these models, but then they are fixed for all content (a). In Structured Audio, the source model is part of the content. It is transmitted in the bitstream and used to give different semantics to the signal representation for each piece of content. There can be a different source model, or multiple source models, for each different piece of content (b). Figure 4.29: In a traditional audio coder, the source model and perception model are defined outside of
refer to text in addition to its audio-visual information. MPEG-7 therefore has standardized different Description Tools for textual annotation and controlled vocabularies, taking into account existing standards and practices.
refer to text in addition to its audio-visual information. MPEG-7 therefore has standardized different Description Tools for textual annotation and controlled vocabularies, taking into account existing standards and practices.
Figure 4.34: Abstract representation of possible applications using MPEG-7.
Figure 4.34: Abstract representation of possible applications using MPEG-7.
Figure 4.35: Abstract representation of possible relation between Descriptors and Description Schemes.
Figure 4.35: Abstract representation of possible relation between Descriptors and Description Schemes.
Figure 4.36: Abstract representation of possible applications using MPEG-7.
Figure 4.36: Abstract representation of possible applications using MPEG-7.
Figure 4.38: AudioSpectrumEnvelope description of a pop song. The required data storage is NM values where NV is the number of spectrum bins and / is the number of time points.
Figure 4.38: AudioSpectrumEnvelope description of a pop song. The required data storage is NM values where NV is the number of spectrum bins and / is the number of time points.
Figure 4.39: A 10-basis component reconstruction showing most of the detail of the original spectro- gram including guitar, bass guitar, hi-hat and organ notes. The left vectors are an AudioSpectrumBasis Descriptor and the top vectors are the corresponding AudioSpectrumProjection Descriptor. The re- quired data storage is 10(M + N) values. high-level tools that exchange some degree of generality for descriptive richness. The five sets of audio Description Tools that roughly correspond to application areas are integrated in the standard: audio signature, musical instrument timbre, melody description, general sound recognition and indexing, and spoken content. The latter two are excellent examples of how the Audio Framework and MDS Description Tools may be integrated to support other applications.
Figure 4.39: A 10-basis component reconstruction showing most of the detail of the original spectro- gram including guitar, bass guitar, hi-hat and organ notes. The left vectors are an AudioSpectrumBasis Descriptor and the top vectors are the corresponding AudioSpectrumProjection Descriptor. The re- quired data storage is 10(M + N) values. high-level tools that exchange some degree of generality for descriptive richness. The five sets of audio Description Tools that roughly correspond to application areas are integrated in the standard: audio signature, musical instrument timbre, melody description, general sound recognition and indexing, and spoken content. The latter two are excellent examples of how the Audio Framework and MDS Description Tools may be integrated to support other applications.
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved