CN108109615A - A kind of construction and application method of the Mongol acoustic model based on DNN - Google Patents

Abstract

The invention provides a method for constructing and using a DNN-based Mongolian acoustic model. The DNN deep neural network is used to replace the GMM Gaussian mixture model to estimate the posterior probability of the Mongolian acoustic state, construct a DNN‑HMM acoustic model, and use the DNN‑HMM acoustic model to recognize Mongolian speech data. The invention can effectively reduce the error rate of word recognition and the error rate of word recognition, and improve the performance of the model.

Description

A DNN-based Mongolian Acoustic Model Construction and Application Method

technical field

The invention belongs to the field of Mongolian speech recognition, and in particular relates to a method for constructing and using a DNN-based Mongolian acoustic model.

Background technique

A typical large vocabulary continuous speech recognition system (Large Vocabulary Continuous SpeechRecognition, LVCSR) is composed of feature extraction, acoustic model, language model and decoder. The acoustic model is the core component of the speech recognition system, based on the GMM model (mixed Gaussian model ) and the HMM model (Hidden Markov Model) built the GMM-HMM acoustic model was once the most widely used acoustic model in large vocabulary continuous speech recognition systems. In the GMM-HMM model, the GMM model probabilistically models the speech feature vector, and then generates the maximum probability of speech observation features through the EM algorithm (maximum expectation algorithm). When the number of mixed Gaussian distributions is large enough, the GMM model can fully fit The probability distribution of the acoustic features, the HMM model generates the temporal state of the speech from the observed state fitted by the GMM model. When the probability of the GMM model mixed Gaussian model is used to describe the distribution of speech data, the GMM model is essentially a shallow model, and the assumption of independence between the features is made when fitting the distribution of the acoustic features, so the acoustic features cannot be fully described. At the same time, the feature dimension of GMM modeling is generally tens of dimensions, which cannot fully describe the correlation between acoustic features, and the model expression ability is limited.

Contents of the invention

In the 1980s, research on building acoustic models using neural networks and HMM models began to emerge. However, due to insufficient computer computing power and insufficient training data at that time, the effect of the model was not as good as GMM-HMM. In 2010, Deng Li and Hinton's group of Microsoft Research Asia proposed a CD-DBN (Dynamic Bayesian Network)-HMM hybrid acoustic model framework for large-scale continuous speech recognition tasks, and conducted related experiments. The experimental results show that compared with the GMM-HMM acoustic model, the adoption of the CD-DBN-HMM acoustic model can increase the recognition accuracy of the speech recognition system by about 30%. The proposal of the CD-DBN-HMM hybrid acoustic model framework has completely revolutionized the original Acoustic model framework. Compared with the traditional Gaussian mixture model, the deep neural network is a deep model, which can better represent complex nonlinear functions, better capture the correlation between speech feature vectors, and easily achieve better modeling results. Based on the above achievements, the present invention proposes a method for constructing and using a Mongolian acoustic model based on a DNN model, so as to better complete the Mongolian acoustic model modeling task.

Technical scheme of the present invention is:

1. Model construction:

The DNN deep neural network is used to replace the GMM Gaussian mixture model to realize the estimation of the posterior probability of the Mongolian acoustic state. In the case of a given Mongolian acoustic feature sequence, the DNN model is first used to estimate the probability that the current feature belongs to the HMM state, and then the HMM model is used to describe the dynamic changes of the Mongolian speech signal and capture the temporal state information of the Mongolian speech information.

The training of the DNN network in the Mongolian acoustic model is divided into two stages: pre-training and tuning.

In the pre-training of the DNN network, a layer-by-layer unsupervised training algorithm is used, which belongs to the generative training algorithm. The layer-by-layer unsupervised pre-training algorithm is to train each layer of the DNN network, and only train one layer at a time. The parameters of other layers keep the original initialization parameters unchanged. During training, the input of each layer The error of output and output is reduced as much as possible to ensure that the parameters of each layer are optimal for this layer. Next, the output data of each layer trained is used as the input data of the next layer, then the input data of the next layer will have a much smaller error than the data input to the next layer through the multi-layer neural network during direct training, The layer-by-layer unsupervised pre-training algorithm can ensure that the error of input and output data between each layer is relatively small.

Better neural network initialization parameters can be obtained through the layer-by-layer unsupervised pre-training algorithm, and supervised tuning is carried out through the BP algorithm (error back-propagation algorithm) using Mongolian label data (that is, the feature state), and finally the neural network can be used for acoustics. A DNN Deep Neural Network Model for State Classification.

2. Model use:

After pre-training and tuning the DNN network, the DNN-HMM acoustic model can be used to recognize Mongolian speech data. The specific process is as follows:

Step 1: Calculate the output of the first L layer of the DNN deep neural network according to the input Mongolian acoustic feature vector.

Step 2: Use the softmax classification layer of the L layer to calculate the posterior probability of the current feature with respect to all acoustic states. That is, the probability that the current feature belongs to each Mongolian acoustic state.

Step 3: According to the Bayesian formula, divide the posterior probability of each state by its own prior probability to obtain the regularized likelihood value of each state.

Step 4: Use the Viterbi decoding algorithm to decode to obtain the optimal path.

Among them, the prior probability of the hidden state can be obtained by calculating the ratio of the total number of frames corresponding to each state to the number of all acoustic feature frames.

3. Model training process

The labeled data used in the DNN deep neural network tuning stage is obtained by forced alignment of the GMM-HMM acoustic model, and the stochastic gradient descent algorithm is used to update the model parameters. Therefore, before training the DNN-HMM, it is first necessary to train a sufficiently good The GMM-HMM Mongolian acoustic model, and then the GMM-HMM Mongolian acoustic model generates the labeled data required for DNN-HMM Mongolian acoustic model training through the Viterbi algorithm.

Since the DNN model needs Mongolian annotation data aligned with Mongolian speech frames when tuning, and the quality of the annotation data often affects the performance of the DNN model. Therefore, for practical application, we utilize the trained GMM-HMM Mongolian acoustic model to achieve forced alignment of speech features to states. Therefore, the training process of the DNN-HMM acoustic model is as follows: first train the GMM-HMM Mongolian acoustic model to obtain aligned Mongolian speech feature data; then train the deep neural network (DNN) on the basis of the aligned speech feature data. Tuning; Finally, the Hidden Markov Model (HMM) is trained according to the obtained Mongolian speech observation state.

Description of drawings

Figure 1 is a DNN-HMM Mongolian acoustic model diagram.

Figure 2 is a diagram of the DNN network pre-training process.

Fig. 3 is a graph of experimental comparison results with respect to the GMM-HMM acoustic model.

Implementation

In order to describe the technical content of the present invention more clearly, further description will be given below in conjunction with specific embodiments.

Embodiment one:

1. Model construction:

The DNN deep neural network is used to replace the GMM Gaussian mixture model to realize the estimation of the posterior probability of the Mongolian acoustic state. In the case of a given Mongolian acoustic feature sequence, the DNN model is first used to estimate the probability that the current feature belongs to the HMM state, and then the HMM model is used to describe the dynamic changes of the Mongolian speech signal and capture the temporal state information of the Mongolian speech information.

The structure of the DNN-HMM Mongolian acoustic model is shown in Figure 1. In the DNN-HMM Mongolian acoustic model, the DNN network is implemented by continuously stacking hidden layers from bottom to top. Among them, S represents the hidden state in the HMM model, A represents the state transition probability matrix, and L represents the number of layers of the DNN deep neural network (the hidden layer is L-1 layer, L0 layer is the input layer, LL layer is the output layer, The DNN network contains a total of L+1 layers), and W represents the connection matrix between layers. The DNN-HMM Mongolian acoustic model needs to train the DNN neural network before modeling the Mongolian speech recognition process. After completing the training of the DNN neural network, the modeling process of the Mongolian acoustic model is consistent with the GMM-HMM model.

The training of the DNN network in the Mongolian acoustic model is divided into two stages: pre-training and tuning.

In the pre-training of the DNN network (as shown in Figure 2), a layer-by-layer unsupervised training algorithm is used, which belongs to the generative training algorithm. The layer-by-layer unsupervised pre-training algorithm is to train each layer of the DNN network, and only train one layer at a time. The parameters of other layers keep the original initialization parameters unchanged. During training, the input of each layer The error of output and output is reduced as much as possible to ensure that the parameters of each layer are optimal for this layer. Next, the output data of each layer trained is used as the input data of the next layer, then the input data of the next layer will have a much smaller error than the data input to the next layer through the multi-layer neural network during direct training, The layer-by-layer unsupervised pre-training algorithm can ensure that the error of input and output data between each layer is relatively small. See Algorithm 1 for the pre-training algorithm.

Better neural network initialization parameters can be obtained through the layer-by-layer unsupervised pre-training algorithm, and supervised tuning is carried out through the BP algorithm (error back-propagation algorithm) using Mongolian label data (that is, the feature state), and finally the neural network can be used for acoustics. A DNN Deep Neural Network Model for State Classification. The supervised tuning algorithm is implemented using the stochastic gradient descent algorithm, see Algorithm 2 for details.

2. Model use:

Step 1: Calculate the output of the first L layer of the DNN deep neural network according to the input Mongolian acoustic feature vector. which is:

v ^α ＝f(z ^α )＝f(W ^α v ^α-1 +b ^α ),0≤α<L

(1) Among them, z ^α represents the excitation vector, z ^α =W ^α v ^α-1 +b ^α and v ^α represents the activation vector, W ^α represents the weight matrix, b ^α represents the paranoid vector, N _α represents the number of neural nodes in layer α and N _α ∈ R. V ⁰ represents the input features of the network, In the DNN-HMM acoustic model, the input feature is the acoustic feature vector. Where N ₀ =D represents the dimension of the input acoustic feature vector, Indicates the calculation process of the activation function on the excitation vector, and f( ) indicates the activation function.

Step 2: Use the softmax classification layer of the L layer to calculate the posterior probability of the current feature with respect to all acoustic states. That is, the probability that the current feature belongs to each Mongolian acoustic state, that is, the probability that the current feature belongs to each Mongolian acoustic state.

v _i ＝P _dnn (i|O)＝softmax(i) (2)

In formula (2), i∈{1,2,…,C}, where C represents the number of hidden states of the acoustic model, x _i represents the input of the i-th neural unit in the softmax layer, and v _i represents the softmax classification layer The output of the i-th neural unit, that is, the posterior probability of the input acoustic feature vector O about the i-th hidden state of the acoustic model.

Step 3: According to the Bayesian formula, divide the posterior probability of each state by its own prior probability to obtain the regularized likelihood value of each state.

Step 4: Use the Viterbi decoding algorithm to decode to obtain the optimal path.

Among them, the prior probability of the hidden state can be obtained by calculating the ratio of the total number of frames corresponding to each state to the number of all acoustic feature frames.

3. Experiment and results:

3.1 In order to verify the effectiveness of the proposed DNN-HMM Mongolian acoustic model, the following experiments are formulated:

(1) Extract the acoustic features of MFCC, and conduct experimental research on the modeling of Mongolian acoustic models of GMM-HMM and DNN-HMM. Observe the impact of different acoustic modeling units on the performance of the acoustic model, and compare the impact of different types of acoustic models on the speech recognition system.

(2) By constructing a DNN-HMM triphone Mongolian acoustic model with a deep network structure of different layers, carry out experimental research on the influence of the number of layers on the Mongolian acoustic model and the phenomenon of overfitting.

3.2 Experimental parameters:

The corpus of Mongolian speech recognition is composed of 310 Mongolian teaching voices, with a total of 2291 Mongolian vocabulary, named IMUT310 corpus. The corpus consists of three parts: audio files, pronunciation annotations, and corresponding Mongolian texts. In the experiment, the IMUT310 corpus is divided into two parts, the training set and the test set. The training set is 287 sentences, and the test set is 23 sentences. The experiment is done on the Kaldi platform. The specific experimental environment configuration of Kaldi is shown in Table 1.

Table 1 Experimental environment

During the experiment, Mongolian acoustic features were represented by MFCC acoustic features, with a total of 39 dimensional data, of which the first 13 dimensional features consisted of 12 cepstrum features and 1 energy coefficient, and the latter two 13 dimensional features were based on the previous 13 dimensional features. first-order difference and second-order difference. When extracting Mongolian MFFC features, the frame window length is 25ms, and the frame shift is 10ms. Feature extraction is performed on the training set and the test set respectively, and a total of 119,960 MFCC features are generated from all speech data, of which 112,535 features are generated from the training data, and 7,425 features are generated from the test data. During the training of the GMM-HMM acoustic model, the Mongolian speech MFCC features were experimented with 39-dimensional data. In the monophonic DNN-HMM experiment, the Mongolian MFCC phonetic features are 13-dimensional (excluding first- and second-order differential features). Mongolian MFCC features are 39-dimensional when experimenting with triphone DNN-HMM

During the training of the DNN network, the feature extraction adopts the method of context combination, that is, 5 frames are taken before and after the current frame to represent the context of the current frame. Therefore, during the experiment, the number of input nodes of the monophonic DNN network is 143 ( 13*(5+1+5)), the number of input nodes of the triphone DNN network is 429 (39*(5+1+5)). The output layer nodes of the DNN network are the number of observable Mongolian phonemes. According to the corpus labeling standard, the number of output nodes is 27; the number of hidden layer nodes of the DNN network is set to 1024, and the number of tuning training times is set to 60. The initial learning rate is set to 0.015, and the final learning rate is set to 0.002.

3.3 Experiment and results:

The four experimental units are: monophonic GMM-HMM, triphonic GMM-HMM, monophonic DNN-HMM and triphonic DNN-HMM experiments. The data of the experimental results are shown in Table 2, and the comparison results are shown in Figure 3.

Table 2 GMM-HMM and DNN-HMM Mongolian acoustic model experimental data

In accompanying drawing 3 (a), it can be found that, compared with the monophonic GMM-HMM Mongolian acoustic model, the word error rate of the monophonic DNN-HMM Mongolian acoustic model on the training set is reduced by 8.84%, and on the test set 11.14% reduction in the word recognition error rate; however, for the triphone model, the triphone DNN-HMM Mongolian acoustic model has a lower word error rate on the training set than the triphone GMM-HMM Mongolian acoustic model 1.33%, and the word recognition error rate on the test set is reduced by 7.5%. Accompanying drawing 3 (b) finds that the sentence recognition error rate of the monophone model on the training set has been reduced by 32.43%, and the sentence recognition error rate on the test set has been reduced by 17.88%; Compared with the triphone GMM-HMM Mongolian acoustic model, the sub-DNN-HMM Mongolian acoustic model has a 19.3% lower sentence recognition error rate on the training set and 13.63% lower sentence recognition error rate on the test set.

From the above analysis, it can be seen that the monophone DNN-HMM Mongolian acoustic model is significantly better than the monophone GMM-HMM Mongolian acoustic model; for the triphone model, the triphone DNN-HMM Mongolian acoustic model is better than the triphone The recognition rate of sub-GMM-HMM Mongolian acoustic model is even higher.

The DNN-HMM Mongolian acoustic model can effectively reduce the error rate of word recognition and word recognition, and improve the performance of the model.

In this specification, the invention has been described with reference to specific embodiments thereof. However, it is obvious that various modifications and changes can be made without departing from the spirit and scope of the invention. Accordingly, the specification and drawings are to be regarded as illustrative rather than restrictive.

Claims (7)

1. a method for constructing a DNN-based Mongolian acoustic model, characterized in that: The DNN deep neural network is used to replace the GMM Gaussian mixture model to realize the estimation of the posterior probability of the Mongolian acoustic state. In the case of a given Mongolian acoustic feature sequence, the DNN model is first used to estimate the probability that the current feature belongs to the HMM state, and then the HMM model is used to describe the dynamic changes of the Mongolian speech signal and capture the temporal state information of the Mongolian speech information. The training of the DNN network in the Mongolian acoustic model is divided into two stages: pre-training and tuning. In the pre-training of the DNN network, a layer-by-layer unsupervised training algorithm is used, which belongs to the generative training algorithm. The layer-by-layer unsupervised pre-training algorithm is to train each layer of the DNN network, and only train one layer at a time. The parameters of other layers keep the original initialization parameters unchanged. During training, the input of each layer The error of output and output is reduced as much as possible to ensure that the parameters of each layer are optimal for this layer. Next, the output data of each layer trained is used as the input data of the next layer, then the input data of the next layer will have a much smaller error than the data input to the next layer through the multi-layer neural network during direct training, The layer-by-layer unsupervised pre-training algorithm can ensure that the error of input and output data between each layer is relatively small. Better neural network initialization parameters can be obtained through the layer-by-layer unsupervised pre-training algorithm, and supervised tuning is carried out through the BP algorithm (error back-propagation algorithm) using Mongolian label data (that is, the feature state), and finally the neural network can be used for acoustics. A DNN Deep Neural Network Model for State Classification. 2. the construction method of a kind of DNN-based Mongolian acoustic model as claimed in claim 1, is characterized in that: described DNN network realizes by constantly stacking hidden layers from bottom to top. 3. the construction method of a kind of Mongolian acoustic model based on DNN as claimed in claim 1 or 2, is characterized in that: the pre-training of described DNN network adopts layer by layer unsupervised pre-training algorithm: Input: ∈ represents the learning rate, T represents the maximum number of iterations, and L represents the number of layers to be trained N=(n ¹ ,n ² ,…,n ^L ) indicates the number of hidden units in each hidden layer X ^j is the sequence of training data divided by mini-batch, j=(1,...,Max), where Max represents the sequence length Output: W ⁱ represents the link weight between the i-th hidden layer and the i-1-th hidden layer, i=(1,2,...,L) b ⁱ represents the bias vector of the i-th layer, i=(0,1,…,L). 4. as claimed in claim 1 or 2, a kind of construction method based on the Mongolian acoustic model of DNN is characterized in that: the optimization of described DNN network adopts stochastic gradient descent algorithm: Input: training set set, batch size batch_size Learning rate, the number of cycles epoch Output: model parameter weight. 5. A method for using a DNN-based Mongolian acoustic model, characterized in that: After pre-training and tuning the DNN network, the DNN-HMM acoustic model can be used to recognize Mongolian speech data. The specific process is as follows: Step 1: Calculate the output of the first L layer of the DNN deep neural network according to the input Mongolian acoustic feature vector. Step 2: Use the softmax classification layer of the L layer to calculate the posterior probability of the current feature with respect to all acoustic states. That is, the probability that the current feature belongs to each Mongolian acoustic state. Step 3: According to the Bayesian formula, divide the posterior probability of each state by its own prior probability to obtain the regularized likelihood value of each state. Step 4: Use the Viterbi decoding algorithm to decode to obtain the optimal path. Among them, the prior probability of the hidden state can be obtained by calculating the ratio of the total number of frames corresponding to each state to the number of all acoustic feature frames. 6. the using method of a kind of Mongolian acoustic model based on DNN as claimed in claim 5, is characterized in that: Described step one: at first according to the Mongolian acoustic feature vector of input, calculate the output of the front L layer of DNN deep neural network, namely: v ^α ＝f(z ^α )＝f(W ^α v ^α-1 +b ^α ),0≤α<L Among them, z ^α represents the excitation vector, z ^α =W ^α v ^α-1 +b ^α and v ^α represents the activation vector, W ^α represents the weight matrix, b ^α represents the paranoid vector, N _α represents the number of neural nodes in layer α and N _α ∈ R. V ⁰ represents the input features of the network, In the DNN-HMM acoustic model, the input feature is the acoustic feature vector. Where N ₀ =D represents the dimension of the input acoustic feature vector, Indicates the calculation process of the activation function on the excitation vector, and f( ) indicates the activation function. 7. the using method of a kind of Mongolian acoustic model based on DNN as claimed in claim 5, is characterized in that: The second step: using the softmax classification layer of the L layer to calculate the posterior probability of the current feature with respect to all the acoustic states, that is, the probability that the current feature belongs to each Mongolian acoustic state. v _i ＝P _dnn (i|O)＝softmax(i) Among them, i∈{1,2,…,C}, where C represents the number of hidden states of the acoustic model, x _i represents the input of the i-th neural unit in the softmax layer, v _i represents the i-th neural unit in the softmax classification layer The output of the input acoustic feature vector O is the posterior probability of the i-th hidden state of the acoustic model.