Academia.eduAcademia.edu

Figure 9 – uploaded by Tighe Barris

See full PDFdownloadDownload figure
Figure 3.3: Customly developed user interface to manage all recording stations When a user is ready to start recording, they begin to populate the fields in green as seen in Figure 3.3. They are given an option to tag a recording session with additional information such as where the microphones are situated and what the current conditions are like.

Figure 3 3: Customly developed user interface to manage all recording stations When a user is ready to start recording, they begin to populate the fields in green as seen in Figure 3.3. They are given an option to tag a recording session with additional information such as where the microphones are situated and what the current conditions are like.

Related Figures (51)

1 Introduction
1 Introduction
Figure 2.1: Mel filter bank consisting of 10 bins spanning 0-10 KHz
Figure 2.1: Mel filter bank consisting of 10 bins spanning 0-10 KHz
Figure 2.2: Bark and Uniform banks consisting of 5 bins each spanning 0-4 KHz [8
Figure 2.2: Bark and Uniform banks consisting of 5 bins each spanning 0-4 KHz [8
Figure 2.3: Basic model for vector quantisation [4| Sonsider the case where only 2 features are extracted per 20 ms frame. For every frame, he relationship is ‘plotted’. Overtime, vectors will begin to cluster in what is known as Voronoi’ regions. Within these regions, a single point can be used to represent the cluster. [hese points are called codewords. Figure 2.3 provides a visual explanation.
Figure 2.3: Basic model for vector quantisation [4| Sonsider the case where only 2 features are extracted per 20 ms frame. For every frame, he relationship is ‘plotted’. Overtime, vectors will begin to cluster in what is known as Voronoi’ regions. Within these regions, a single point can be used to represent the cluster. [hese points are called codewords. Figure 2.3 provides a visual explanation.
Figure 2.4: Single layer feed-forward artificial neural network [13] The paper describes that an ANN is composed of several simple elements communicating in parallel within a digital network. It consists of several input nodes, hidden intercon- nected nodes and output nodes. Figure 2.4 shows an example ANN model with 4 hidden nodes. During the training phase, features are provided as inputs and the known corresponding classes as outputs. The network then begins to construct suitable weightings between all the nodes. This is iterated until it can approximate a function to map input feature vectors to output classes within a certain margin of error.
Figure 2.4: Single layer feed-forward artificial neural network [13] The paper describes that an ANN is composed of several simple elements communicating in parallel within a digital network. It consists of several input nodes, hidden intercon- nected nodes and output nodes. Figure 2.4 shows an example ANN model with 4 hidden nodes. During the training phase, features are provided as inputs and the known corresponding classes as outputs. The network then begins to construct suitable weightings between all the nodes. This is iterated until it can approximate a function to map input feature vectors to output classes within a certain margin of error.
Figure 2.5: Graph displaying DTW algorithm and computation of alignment [15] The the Euclidean distance between corresponding sample points. A length metric is then com algorithm works by aligning two signals within a matrix, where each index holds puted by calculating the shortest path from the lower left corner to the upper right corner. Several boundary conditions are developed that can reduce the computation time [6]. Two signals are more correlated if the length metric is minimised. Figure 2.5 shows an equivalent visual representation.
Figure 2.5: Graph displaying DTW algorithm and computation of alignment [15] The the Euclidean distance between corresponding sample points. A length metric is then com algorithm works by aligning two signals within a matrix, where each index holds puted by calculating the shortest path from the lower left corner to the upper right corner. Several boundary conditions are developed that can reduce the computation time [6]. Two signals are more correlated if the length metric is minimised. Figure 2.5 shows an equivalent visual representation.
Figure 2.6: Example: Histogram values for OCR classification Suppose that we use the height of the characters ‘A’ and ‘B’ as descriptive features (as per Bishop’s example). An accurate test set is compiled and the height of each character in the images measured. A histogram of potential results is plotted below:
Figure 2.6: Example: Histogram values for OCR classification Suppose that we use the height of the characters ‘A’ and ‘B’ as descriptive features (as per Bishop’s example). An accurate test set is compiled and the height of each character in the images measured. A histogram of potential results is plotted below:
Table 2.2: Confusion matrix defining test outcomes
Table 2.2: Confusion matrix defining test outcomes
Figure 3.1: Recording station: U3 CAD microphone, Raspberry Pi and WiFi dongle In order to capture ambient recordings in the least obtrusive manner, a Raspberry Pi 2 is setup along with an attached microphone. The Pi runs on-board developed software and transmits recordings to a central server. A U3 studio recording microphone manufactured by CAD is chosen. Both items provide more than enough durability and performance for the given application. Figure 3.1 shows an image of the microphone and Pi used to conduct tests. This is collectively referred to as a recording station.
Figure 3.1: Recording station: U3 CAD microphone, Raspberry Pi and WiFi dongle In order to capture ambient recordings in the least obtrusive manner, a Raspberry Pi 2 is setup along with an attached microphone. The Pi runs on-board developed software and transmits recordings to a central server. A U3 studio recording microphone manufactured by CAD is chosen. Both items provide more than enough durability and performance for the given application. Figure 3.1 shows an image of the microphone and Pi used to conduct tests. This is collectively referred to as a recording station.
Figure 3.2: Overview of software model
Figure 3.2: Overview of software model
Figure 3.4: Timeline of events describing recording process This interface is designed to give the user as much freedom when choosing how long they would like to record for (segmented by the splice length), frequency, output file format and byte format. Once all the fields are specified and the recording request activated, the server begins to synchronise all events. Figure 3.4 depicts the large majority of the communication between the client, server and recording stations to make this all happen. Two recording stations are present and each record into a single file. It is worth mentioning that at the start of recording, the server designates one of the stations as a master. The master plays a 5 KHz test tone five seconds after notification. This 5 KHz test tone is used to align all the independently captured audio files from each microphone during post analyses.
Figure 3.4: Timeline of events describing recording process This interface is designed to give the user as much freedom when choosing how long they would like to record for (segmented by the splice length), frequency, output file format and byte format. Once all the fields are specified and the recording request activated, the server begins to synchronise all events. Figure 3.4 depicts the large majority of the communication between the client, server and recording stations to make this all happen. Two recording stations are present and each record into a single file. It is worth mentioning that at the start of recording, the server designates one of the stations as a master. The master plays a 5 KHz test tone five seconds after notification. This 5 KHz test tone is used to align all the independently captured audio files from each microphone during post analyses.
Table 3.1: Colour code thresholds based on confidence for recordings containing barking
Table 3.1: Colour code thresholds based on confidence for recordings containing barking
Figure 3.5: Customly developed user interface for managing all recorded instances Figure 3.5 shows a screenshot of the logbook which, like the recording interface, is made accessible from the server over HTTP. The far left column itemises each recording session, while the far right lists meta-data regarding selected recordings. The table below the red waveform lists all the detected events and with what certainty it has been classified as a dog bark. The blue highlighting over the graphed waveform depicts an event of interest. As mentioned before, each recording instance and splice is colour coded as to the thresholds outlined in Table 3.1.
Figure 3.5: Customly developed user interface for managing all recorded instances Figure 3.5 shows a screenshot of the logbook which, like the recording interface, is made accessible from the server over HTTP. The far left column itemises each recording session, while the far right lists meta-data regarding selected recordings. The table below the red waveform lists all the detected events and with what certainty it has been classified as a dog bark. The blue highlighting over the graphed waveform depicts an event of interest. As mentioned before, each recording instance and splice is colour coded as to the thresholds outlined in Table 3.1.
Figure 3.6: Process followed to identify regions of interest within audio recordings For any given recording, which can range from 1 minute to several hours in length, areas of interest need to be identified and conditioned. For example, a recording may run from 23h00 to 06h30, in which time a single dog barks for 10 minutes at 05h00. Instead of analysing the entire 7.5 hour recording, intervals of interest are first identified and then analysed. This has the advantage of reducing the analysis duration. Figure 3.6 describes the overall process implemented to achieve this and is subsequently explained. Each recording is represented as a single row vector denoted by z. Each successive entry of Z holds a sampled value that can be used to construct an equivalent analogue signal using the Whittaker-Shannon interpolation formula. The number of samples per recording is equal to N = duration x frequency. For an unprocessed signal, the contents of # could look as follows:
Figure 3.6: Process followed to identify regions of interest within audio recordings For any given recording, which can range from 1 minute to several hours in length, areas of interest need to be identified and conditioned. For example, a recording may run from 23h00 to 06h30, in which time a single dog barks for 10 minutes at 05h00. Instead of analysing the entire 7.5 hour recording, intervals of interest are first identified and then analysed. This has the advantage of reducing the analysis duration. Figure 3.6 describes the overall process implemented to achieve this and is subsequently explained. Each recording is represented as a single row vector denoted by z. Each successive entry of Z holds a sampled value that can be used to construct an equivalent analogue signal using the Whittaker-Shannon interpolation formula. The number of samples per recording is equal to N = duration x frequency. For an unprocessed signal, the contents of # could look as follows:
Figure 3.7: Lowpass filter with 10 KHz cutoff frequency A 61 order equiripple lowpass FIR filter is used to apply the initial filtering. Its impulse response shown in Figure 3.7. The effect of the linear phase distortion below 12 KHz is not considered detrimental as it only introduces a time delay in the sampled signal. The non-linear phase distortion after 12 KHz can be considered negligible as the filter applies over 80 dB of suppression across those frequencies.
Figure 3.7: Lowpass filter with 10 KHz cutoff frequency A 61 order equiripple lowpass FIR filter is used to apply the initial filtering. Its impulse response shown in Figure 3.7. The effect of the linear phase distortion below 12 KHz is not considered detrimental as it only introduces a time delay in the sampled signal. The non-linear phase distortion after 12 KHz can be considered negligible as the filter applies over 80 dB of suppression across those frequencies.
Figure 3.8: Acoustic waveform containing 9 dog barks with 3 identified regions of interest Figure 3.8 depicts a time domain signal containing 9 barks emitted from the same dog. Each triangular rise indicates a dog bark. The signal has been treated and conditioned as per the steps outlined in Figure 3.6. The light pink areas indicate regions of interest, while the light blue area indicates the RMS threshold.
Figure 3.8: Acoustic waveform containing 9 dog barks with 3 identified regions of interest Figure 3.8 depicts a time domain signal containing 9 barks emitted from the same dog. Each triangular rise indicates a dog bark. The signal has been treated and conditioned as per the steps outlined in Figure 3.6. The light pink areas indicate regions of interest, while the light blue area indicates the RMS threshold.
Figure 3.9: Overview of MFCC extraction MFCCs, embedded within every acoustic signal, act as identifiers by numerically rep- resenting a signal’s spectral distribution. This allows for different audio signals to be compared with one another. An overview of the process followed to extract MFCCs is summarised by Figure 3.9.
Figure 3.9: Overview of MFCC extraction MFCCs, embedded within every acoustic signal, act as identifiers by numerically rep- resenting a signal’s spectral distribution. This allows for different audio signals to be compared with one another. An overview of the process followed to extract MFCCs is summarised by Figure 3.9.
Figure 3.10: Enlarged view of first identified region of interest from Figure 3.8
Figure 3.10: Enlarged view of first identified region of interest from Figure 3.8
Figure 3.11: Magnitude response of the pre-emphasis filter The identified signal, Z, is first passed through a pre-emphasis filter. This is done for two reasons. Naturally voiced sections of the frequency spectrum experience an attenuation of 20 dB/decade due to physiological reasons [18]. The filter counteracts this by introducing a zero with 20 db/decade ‘lift’. The second reason is to emphasise frequencies above 1 KHz as our auditory system is more sensitive in this region. In a study on the use of pre-emphasis filtering and different window shapes in phase-based speech recognition, they found that the use of the filter to be beneficial [19]. Figure 3.11 shows the impulse response for the filter, which has a single zero located at Z = 0.96.
Figure 3.11: Magnitude response of the pre-emphasis filter The identified signal, Z, is first passed through a pre-emphasis filter. This is done for two reasons. Naturally voiced sections of the frequency spectrum experience an attenuation of 20 dB/decade due to physiological reasons [18]. The filter counteracts this by introducing a zero with 20 db/decade ‘lift’. The second reason is to emphasise frequencies above 1 KHz as our auditory system is more sensitive in this region. In a study on the use of pre-emphasis filtering and different window shapes in phase-based speech recognition, they found that the use of the filter to be beneficial [19]. Figure 3.11 shows the impulse response for the filter, which has a single zero located at Z = 0.96.
Figure 3.12: Graphical interpretation of framing process Figure 3.12 depicts the framing process. Each row of the matrix represents a 25 ms window, with frame steps of 10 ms between rows. An estimation of the power spectral density is then computed for each row of X. First the Discrete Fourier Transom (DFT) is calculated, and then its magnitude squared and averaged as follows:
Figure 3.12: Graphical interpretation of framing process Figure 3.12 depicts the framing process. Each row of the matrix represents a 25 ms window, with frame steps of 10 ms between rows. An estimation of the power spectral density is then computed for each row of X. First the Discrete Fourier Transom (DFT) is calculated, and then its magnitude squared and averaged as follows:
Figure 3.13: First windowed frame of the signal in Figure 3.10 and its estimated PSD
Figure 3.13: First windowed frame of the signal in Figure 3.10 and its estimated PSD
The matrix containing each filter in its columns is then operated on P as follows Figure 3.14 shows a portion of the 32 computed filters. Notice that the filters are seper- ated exponentially apart with widening widths for higher frequencies. This is a result of using the mel scale. The next step is to apply the mel filter bank. A matrix, B, is constructed containing all the separate filters in each column. Preliminary, 32 filter bins are chosen and digitally computed as wide as matrix P. The bins are spaced evenly on the mel scale from 0 to 24 KHz (upper limit being the Nyquist rate) and translated back to the normal frequency scale. The following formula is used to convert between the two scales [4].
The matrix containing each filter in its columns is then operated on P as follows Figure 3.14 shows a portion of the 32 computed filters. Notice that the filters are seper- ated exponentially apart with widening widths for higher frequencies. This is a result of using the mel scale. The next step is to apply the mel filter bank. A matrix, B, is constructed containing all the separate filters in each column. Preliminary, 32 filter bins are chosen and digitally computed as wide as matrix P. The bins are spaced evenly on the mel scale from 0 to 24 KHz (upper limit being the Nyquist rate) and translated back to the normal frequency scale. The following formula is used to convert between the two scales [4].
Figure 3.15: MFCCs extracted from the signal represented in Figure 3.10 Typically only columns 2 to 14 of C are used for ASR applications [8]. This produces 13 MFCCs for every 10 ms of audio parsed. Figure 3.15 shows the MFCCs captured from the acoustic signal shown in Figure 3.10. It is interesting to observe that the most spectral fluctuations occur in the lower indexed bins typically around 2-5 KHz. Lyons suggests that a further step can be taken by calculating what is known as the A (‘delta’) and AA (‘delta delta’) coefficients [7]:
Figure 3.15: MFCCs extracted from the signal represented in Figure 3.10 Typically only columns 2 to 14 of C are used for ASR applications [8]. This produces 13 MFCCs for every 10 ms of audio parsed. Figure 3.15 shows the MFCCs captured from the acoustic signal shown in Figure 3.10. It is interesting to observe that the most spectral fluctuations occur in the lower indexed bins typically around 2-5 KHz. Lyons suggests that a further step can be taken by calculating what is known as the A (‘delta’) and AA (‘delta delta’) coefficients [7]:
Now that the MFCC features are captured, a means to compare them to a known reference needs to be developed. This section elaborates on 3 different classifiers designed to achieve this. An overview of the feature matching process is shown in Figure 3.16. Each classifier is designed to produce a confidence rating from 0 to 1 (a dog bark), where all outputs are fed into what is known as the ‘Jury’ classifier. The Jury is responsible for modeling the behavior of each classifier and combining all confidence ratings. A threshold is then applied to the overall confidence. If it is met, the acoustic signal is classified to contain dog barks. If not, the signal is classified as not containing dog barks. The sections to follow develop each of these components in detail. 3.3 Acoustic Analysis: Feature Matching
Now that the MFCC features are captured, a means to compare them to a known reference needs to be developed. This section elaborates on 3 different classifiers designed to achieve this. An overview of the feature matching process is shown in Figure 3.16. Each classifier is designed to produce a confidence rating from 0 to 1 (a dog bark), where all outputs are fed into what is known as the ‘Jury’ classifier. The Jury is responsible for modeling the behavior of each classifier and combining all confidence ratings. A threshold is then applied to the overall confidence. If it is met, the acoustic signal is classified to contain dog barks. If not, the signal is classified as not containing dog barks. The sections to follow develop each of these components in detail. 3.3 Acoustic Analysis: Feature Matching
Figure 3.17: Diagram showing difference between training and testing
Figure 3.17: Diagram showing difference between training and testing
Figure 3.18: Two-layer feed-forward ANN implemented in Matlab
Figure 3.18: Two-layer feed-forward ANN implemented in Matlab
Figure 3.19: Jury classifier Decision The exponent of a 5 is designed to spread the output confidence interval apart, while the multiplication of the two factors is to encourage an intersection between the two input weightings. Figure 3.19 is provided to depict the equivalent pictorially.
Figure 3.19: Jury classifier Decision The exponent of a 5 is designed to spread the output confidence interval apart, while the multiplication of the two factors is to encourage an intersection between the two input weightings. Figure 3.19 is provided to depict the equivalent pictorially.
Table 4.1: Number of recordings and identified instances used to conduct tests Several different measures are used to interpret the performance. The first, known as th True Positive Rate (TPR) or otherwise sensitivity is defined as: In order to conduct the trials, five different acoustic sources are prepared and itemised by Table 4.1. The table distinguishes between the number of recordings and number of instances. The later refers to the total number of regions identified for analysis as outlined in section 3.2.1. The acoustic source types are chosen to be a mixture of ambient recordings and impulse like sounds that the classifiers may struggle to recognise.
Table 4.1: Number of recordings and identified instances used to conduct tests Several different measures are used to interpret the performance. The first, known as th True Positive Rate (TPR) or otherwise sensitivity is defined as: In order to conduct the trials, five different acoustic sources are prepared and itemised by Table 4.1. The table distinguishes between the number of recordings and number of instances. The later refers to the total number of regions identified for analysis as outlined in section 3.2.1. The acoustic source types are chosen to be a mixture of ambient recordings and impulse like sounds that the classifiers may struggle to recognise.
Table 4.2: Description of conducted trials
Table 4.2: Description of conducted trials
4.2.1 Results
4.2.1 Results
Figure 4.2: Trial 2 - Result of varying the correlation threshold used in feature merging ‘igure 4.1: Trial 1 - Results obtained for different size k-mean clusters
Figure 4.2: Trial 2 - Result of varying the correlation threshold used in feature merging ‘igure 4.1: Trial 1 - Results obtained for different size k-mean clusters
Figure 4.4: Trial 4 - Results obtained using non-reduced and reduced feature vectors
Figure 4.4: Trial 4 - Results obtained using non-reduced and reduced feature vectors
Figure 4.6: Trial 6 - Results measuring outcome of using additional cepstral coefficients
Figure 4.6: Trial 6 - Results measuring outcome of using additional cepstral coefficients
‘igure 4.7: Trial 7 - Results for varying mel filter bank upper frequency limits
‘igure 4.7: Trial 7 - Results for varying mel filter bank upper frequency limits
Figure 4.8: Trial 8 - Performance of the cross-correlated based classifier
Figure 4.8: Trial 8 - Performance of the cross-correlated based classifier
Table 4.3: Fixed parameters used in Experiment 2 In this experiment, the output of each classifier is fed into the ‘Jury’ which determines the overall classification decision (section 3.3). The parameters chosen during the feature extraction phase are optimised and readjusted to match Table 4.3. These parameters are derived as a result of a discussion held in section 5.
Table 4.3: Fixed parameters used in Experiment 2 In this experiment, the output of each classifier is fed into the ‘Jury’ which determines the overall classification decision (section 3.3). The parameters chosen during the feature extraction phase are optimised and readjusted to match Table 4.3. These parameters are derived as a result of a discussion held in section 5.
Output for threshold of 0.77
Output for threshold of 0.77
Figure 4.10: Trial 9b - Classification characteristics of the combined Jury classifier Figure 4.9: Trial 9a - Classification characteristics of each classifier
Figure 4.10: Trial 9b - Classification characteristics of the combined Jury classifier Figure 4.9: Trial 9a - Classification characteristics of each classifier
Figure 4.11: Trial 10 - Classifier performance for varying SNR
Figure 4.11: Trial 10 - Classifier performance for varying SNR
Table 4.4: Breakdown of prepared validation data
Table 4.4: Breakdown of prepared validation data
Figure 4.12: Trial 11 - Jury classifier performance measured against the validation data Each recording is prepared and analysed. If the Jury classifier produces a confidence rating greater than 0.77, the inspected acoustic segment is considered to contain dog barks. Each result is manually verified and tallied.
Figure 4.12: Trial 11 - Jury classifier performance measured against the validation data Each recording is prepared and analysed. If the Jury classifier produces a confidence rating greater than 0.77, the inspected acoustic segment is considered to contain dog barks. Each result is manually verified and tallied.
Figure 8.2: Full screenshot of developed logbook interface for managing recordings
Figure 8.2: Full screenshot of developed logbook interface for managing recordings
Related topics:
Automatic Speech RecognitionVector QuantizationArtifical Neural NetworksMel-frequency Cepstral Coefficients
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved