Academia.eduAcademia.edu

Figure 5 – uploaded by César Abad

See full PDFdownloadDownload figure
Given this, it seems to be a good idea for music analysis and synthesis to use some kind of transform that has a good time resolution at the expense of a poorer frequency resolution for high frequencies (which is somehow unnecessary in music signals, as we have seen). That is, divide the time-frequency plane in Heisemberg boxes distributed in a more advantageous way. This is where the Wavelet transform enters the scene. Figure 2.3: Short Time Fourier Transform tiling of Heisemberg boxes for a 32- point window length. Note that the more frequency resolution we need, the more time samples will be necessary to calculate the FFT, and so the less time resolution we will have.

Figure 2 Given this, it seems to be a good idea for music analysis and synthesis to use some kind of transform that has a good time resolution at the expense of a poorer frequency resolution for high frequencies (which is somehow unnecessary in music signals, as we have seen). That is, divide the time-frequency plane in Heisemberg boxes distributed in a more advantageous way. This is where the Wavelet transform enters the scene. Figure 2.3: Short Time Fourier Transform tiling of Heisemberg boxes for a 32- point window length. Note that the more frequency resolution we need, the more time samples will be necessary to calculate the FFT, and so the less time resolution we will have.

Related Figures (36)

(b) Window length: 2048 samples; hop-size: 64 samples. (a) Window length: 128 samples; hop-size: 64 samples.
(b) Window length: 2048 samples; hop-size: 64 samples. (a) Window length: 128 samples; hop-size: 64 samples.
Figure 1.2: Cross-correlation of spectral magnitudes for several hours of music radio. The straight lines indicate correlations between harmonically related frequencies (figure extracted from [1]).
Figure 1.2: Cross-correlation of spectral magnitudes for several hours of music radio. The straight lines indicate correlations between harmonically related frequencies (figure extracted from [1]).
Transform of a signal which contains all the notes of a wavetable-synthesized piano from Ap to Gg. Note that the spectral peaks (harmonics) are much more concentrated in the low frequencies, while the peaks in high frequencies are more separated. By the own nature of the musical notes, if we want to describe the location of these peaks we will need much more resolution for low frequencies than for high frequencies. Note that between 9000 Hz and 12000 Hz only 5 different peaks can occur in this piano. The same number of peaks must be resolved between 300 Hz and 400 Hz. Using an FFT with a 1024-point window at a sample frequency of 44100 Hz we have a resolution of about 43 Hz. That is, we have 2 or 3 spectral bins available around 300-400 Hz, which is not enough to resolve the 5 harmonics, while between 9000 and 12000 Hz we will have about 70 spectral bins, far more than enough to resolve the same number of harmonics. Figure 2.2: Frequency representation of a signal containing all notes from Ag to Gg in a wavetable-synthesized piano.
Transform of a signal which contains all the notes of a wavetable-synthesized piano from Ap to Gg. Note that the spectral peaks (harmonics) are much more concentrated in the low frequencies, while the peaks in high frequencies are more separated. By the own nature of the musical notes, if we want to describe the location of these peaks we will need much more resolution for low frequencies than for high frequencies. Note that between 9000 Hz and 12000 Hz only 5 different peaks can occur in this piano. The same number of peaks must be resolved between 300 Hz and 400 Hz. Using an FFT with a 1024-point window at a sample frequency of 44100 Hz we have a resolution of about 43 Hz. That is, we have 2 or 3 spectral bins available around 300-400 Hz, which is not enough to resolve the 5 harmonics, while between 9000 and 12000 Hz we will have about 70 spectral bins, far more than enough to resolve the same number of harmonics. Figure 2.2: Frequency representation of a signal containing all notes from Ag to Gg in a wavetable-synthesized piano.
Figure 2.4: Typical dyadic time frequency decomposition. A stands for approx- imation level and D for detail level. This example is a decomposition of level 5. generate an orthonormal basis, such that any finite energy signal can be decom- posed over this wavelet basis. The wavelet function ~ is associated with a high pass filter and so with the detail level of the decomposition.
Figure 2.4: Typical dyadic time frequency decomposition. A stands for approx- imation level and D for detail level. This example is a decomposition of level 5. generate an orthonormal basis, such that any finite energy signal can be decom- posed over this wavelet basis. The wavelet function ~ is associated with a high pass filter and so with the detail level of the decomposition.
look at the scalogram in search of components corresponding to the fundamen- tal frequency of the musical notes. In Fig. 3.1 the scalogram of a pure tone signal is plotted. The difference of this representation with respect to the STFT spectrogram is clear: in the vertical axis we have the fundamental frequencies of the musical notes equally spaced. That is a direct consequence of a suited multiresolution analysis. Figure 3.1: 2D time-frequency plot of CWT coefficients using a complex Morlet 1-5 wavelet. The signal being analysed is a pure tone of 440 Hz. (Figure extracted from [2]).
look at the scalogram in search of components corresponding to the fundamen- tal frequency of the musical notes. In Fig. 3.1 the scalogram of a pure tone signal is plotted. The difference of this representation with respect to the STFT spectrogram is clear: in the vertical axis we have the fundamental frequencies of the musical notes equally spaced. That is a direct consequence of a suited multiresolution analysis. Figure 3.1: 2D time-frequency plot of CWT coefficients using a complex Morlet 1-5 wavelet. The signal being analysed is a pure tone of 440 Hz. (Figure extracted from [2]).
Table 4.1: Structure of coefficients in a DWT dyadic decomposition using the wavedec MATLAB function. Let us analyze the properties of each one of the columns of a PSWS. We will use the MATLAB function wavedec. The output of this function is an “structure” of coefficients that contains one sub-vector for the approximation coefficients, and another 8 sub-vectors for the details. If we use a level 8 decomposition we have the distribution of coefficients shown in Table 4.1. Note that the Haar discrete wavelet decomposition is critically sampled: we obtain 256 wavelet coefficients from 256 temporal samples. If we place these coefficients ordered by level in one column at a time, we can build a “spectrogram-like” scalogram in which the frequency information is quite coarse (we only have 8 different levels to distinguish frequencies, each level centered in one different octave) but nevertheless can be very useful as we will see.
Table 4.1: Structure of coefficients in a DWT dyadic decomposition using the wavedec MATLAB function. Let us analyze the properties of each one of the columns of a PSWS. We will use the MATLAB function wavedec. The output of this function is an “structure” of coefficients that contains one sub-vector for the approximation coefficients, and another 8 sub-vectors for the details. If we use a level 8 decomposition we have the distribution of coefficients shown in Table 4.1. Note that the Haar discrete wavelet decomposition is critically sampled: we obtain 256 wavelet coefficients from 256 temporal samples. If we place these coefficients ordered by level in one column at a time, we can build a “spectrogram-like” scalogram in which the frequency information is quite coarse (we only have 8 different levels to distinguish frequencies, each level centered in one different octave) but nevertheless can be very useful as we will see.
Figure 4.3: A tone of 880 Hz produces a DC pattern at level 4.
Figure 4.3: A tone of 880 Hz produces a DC pattern at level 4.
Figure 4.4: We add 8 harmonics with linearly decreasing amplitudes to the 88 Hz pure tone in Figure 4.3. Note that the first, the third and the fifth harmonics are the fundamentals of the A note in octaves 6, 7 and 8 respectively, producing the corresponding characteristic pattern at levels 5, 6 and 7. Note also the reinforcement effect: the other harmonics have contributed to raise the level of the coefficients at level 4 too.
Figure 4.4: We add 8 harmonics with linearly decreasing amplitudes to the 88 Hz pure tone in Figure 4.3. Note that the first, the third and the fifth harmonics are the fundamentals of the A note in octaves 6, 7 and 8 respectively, producing the corresponding characteristic pattern at levels 5, 6 and 7. Note also the reinforcement effect: the other harmonics have contributed to raise the level of the coefficients at level 4 too.
Figure 4.5: In this case the fundamental has been removed and only the 8 har- monics are present. Note that even so, the DC pattern at level 4 indicates the presence of the fundamental of an As note, even though no tone with 880 Hz is present in the signal. When the fundamental is present, the two DC contribu- tions sum, and coefficients at the corresponding level are raised accordingly, as we see in Figure 4.4.
Figure 4.5: In this case the fundamental has been removed and only the 8 har- monics are present. Note that even so, the DC pattern at level 4 indicates the presence of the fundamental of an As note, even though no tone with 880 Hz is present in the signal. When the fundamental is present, the two DC contribu- tions sum, and coefficients at the corresponding level are raised accordingly, as we see in Figure 4.4.
Figure 4.7: PSWS of a pure tone of 1760 Hz. Note that although there are nonzero coefficients at levels 5, 6, 7 and 8, only level 5 coefficients have nonzero mean within each column. Finally, we show in Figure 4.9 the same FPSWS of Figure 4.8 but seen “from above”, as a contour field. Horizontal lines that delimit the boundaries between levels have been added. Note that each level will correspond to the fundamental frequency of the A note in one different octave.
Figure 4.7: PSWS of a pure tone of 1760 Hz. Note that although there are nonzero coefficients at levels 5, 6, 7 and 8, only level 5 coefficients have nonzero mean within each column. Finally, we show in Figure 4.9 the same FPSWS of Figure 4.8 but seen “from above”, as a contour field. Horizontal lines that delimit the boundaries between levels have been added. Note that each level will correspond to the fundamental frequency of the A note in one different octave.
Figure 4.8: Filtered PSWS (FPSWS) of a pure tone of 1760 Hz. All coefficients with zero mean have been removed by a low pass filter that operates locally at each level across the columns.
Figure 4.8: Filtered PSWS (FPSWS) of a pure tone of 1760 Hz. All coefficients with zero mean have been removed by a low pass filter that operates locally at each level across the columns.
Figure 4.9: The same FPSWS in Figure 4.8 but seen from above as a contour field. The philosophy of this representation is that, whenever a pure tone of any frequency that coincides with the fundamental frequency of the A note at any octave is present in a signal, it will appear as nonzero coefficients in the corresponding level of the FPSWS contour field representation. In any other case, the FPSWS would be ideally flat.
Figure 4.9: The same FPSWS in Figure 4.8 but seen from above as a contour field. The philosophy of this representation is that, whenever a pure tone of any frequency that coincides with the fundamental frequency of the A note at any octave is present in a signal, it will appear as nonzero coefficients in the corresponding level of the FPSWS contour field representation. In any other case, the FPSWS would be ideally flat.
Figure 5.1: Spectrogram of a sequence of 24 notes, played in order from Cs to Bg in a wavetable-synthesis piano. FFT widow size: 512 samples; hop size: 256 samples. Let us analyze a signal that is a sequence of 24 notes, played in order from C5 to Bg in a wavetable-synthesis piano. The spectrogram of this signal is shown in Figure 5.1.
Figure 5.1: Spectrogram of a sequence of 24 notes, played in order from Cs to Bg in a wavetable-synthesis piano. FFT widow size: 512 samples; hop size: 256 samples. Let us analyze a signal that is a sequence of 24 notes, played in order from C5 to Bg in a wavetable-synthesis piano. The spectrogram of this signal is shown in Figure 5.1.
Figure 5.3: 3D representation of the PSWS and the FPSWS of the signal in Figure 5.2 for note C.
Figure 5.3: 3D representation of the PSWS and the FPSWS of the signal in Figure 5.2 for note C.
Figure 5.4: 3D representation of the PSWS and the FPSWS of the signal in Figure 5.2 for note G.
Figure 5.4: 3D representation of the PSWS and the FPSWS of the signal in Figure 5.2 for note G.
Figure 5.5: HPCP representation of the same signal analyzed in Figure 5.2. We can compare this results with those obtained using the Harmonic Pitch Class Profile (HPCP) representation developed by Gémez in [32]. One impor- tant difference between the two types of representation is that HPCP do not care about the octaves of the notes: it represents the harmonic content of a signal by adding the STFT spectral peaks that contribute to each note, no matter the octave. In FPSWS representations we can estimate the octave of the note be- cause coefficients of lower levels than those that correspond to the fundamental frequency of the note detected are ideally zero, as explained in chapter 4.
Figure 5.5: HPCP representation of the same signal analyzed in Figure 5.2. We can compare this results with those obtained using the Harmonic Pitch Class Profile (HPCP) representation developed by Gémez in [32]. One impor- tant difference between the two types of representation is that HPCP do not care about the octaves of the notes: it represents the harmonic content of a signal by adding the STFT spectral peaks that contribute to each note, no matter the octave. In FPSWS representations we can estimate the octave of the note be- cause coefficients of lower levels than those that correspond to the fundamental frequency of the note detected are ideally zero, as explained in chapter 4.
Figure 5.6: Spectrogram of a sequence of the chords C - Cm7 in the 5th octave, and C - Cm7 in the 6th octave played in in a wavetable-synthesis piano. FFT widow size: 512 samples; hop size: 256 samples.
Figure 5.6: Spectrogram of a sequence of the chords C - Cm7 in the 5th octave, and C - Cm7 in the 6th octave played in in a wavetable-synthesis piano. FFT widow size: 512 samples; hop size: 256 samples.
Figure 5.7: FPSWS’s of the signal on Figure 5.6.
Figure 5.7: FPSWS’s of the signal on Figure 5.6.
Let us now analyze the same signal in Figure 5.6 in a not-so-ideal situation. We have added a charles at quarter notes from the first bar, a snare drum at half notes in the second and fourth bar, and a bass drum at half notes in the third and the fourth bar. The spectrogram of this signal is shown in Figure 5.9 Figure 5.9: Spectrogram of a sequence of the chords in Figure 5.6 plus a drum pattern in which the notes coincide exactly first with the charles (first bar), then with the charles and the snare drum (second bar), then only with the charles and the snare drum (third bar), and finally with the charles, the bass drum and and snare drum (fourth bar). FFT widow size: 512 samples; hop size: 256 samples.
Let us now analyze the same signal in Figure 5.6 in a not-so-ideal situation. We have added a charles at quarter notes from the first bar, a snare drum at half notes in the second and fourth bar, and a bass drum at half notes in the third and the fourth bar. The spectrogram of this signal is shown in Figure 5.9 Figure 5.9: Spectrogram of a sequence of the chords in Figure 5.6 plus a drum pattern in which the notes coincide exactly first with the charles (first bar), then with the charles and the snare drum (second bar), then only with the charles and the snare drum (third bar), and finally with the charles, the bass drum and and snare drum (fourth bar). FFT widow size: 512 samples; hop size: 256 samples.
Figure 5.10: FPSWS’s of the signal on Figure 5.9.
Figure 5.10: FPSWS’s of the signal on Figure 5.9.
So far, we have used only a wavetable-synthesis piano for our analyses. In the next example, an excerpt taken from a recording of the beginning of Beethoven’s “For Elisa” played in a real piano is used. The spectrogram is shown in Fig- ure 5.12. Figure 5.12: Spectrogram of the beginning of Beethoven’s “For Elisa” played in a real piano. FFT widow size: 2048 samples; hop size: 512 samples. Only low frequencies are represented in the picture.
So far, we have used only a wavetable-synthesis piano for our analyses. In the next example, an excerpt taken from a recording of the beginning of Beethoven’s “For Elisa” played in a real piano is used. The spectrogram is shown in Fig- ure 5.12. Figure 5.12: Spectrogram of the beginning of Beethoven’s “For Elisa” played in a real piano. FFT widow size: 2048 samples; hop size: 512 samples. Only low frequencies are represented in the picture.
Related topics:
Multiresolution Analysis
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved