Multi-Dialect Arabic Speech Recognition

ASRU, 2017

This paper describes an Arabic Automatic Speech Recognition system developed on 15 hours of Multi-Genre Broadcast (MGB-3) data from YouTube, plus 1,200 hours of Multi-Dialect and Multi-Genre MGB-2 data recorded from the Aljazeera Arabic TV channel. In this paper, we report our investigations of a range of signal pre-processing, data augmentation , topic-specific language model adaptation, accent specific retraining , and deep learning based acoustic modeling topologies, such as feed-forward Deep Neural Networks (DNNs), Time-delay Neural Networks (TDNNs), Long Short-term Memory (LSTM) networks, Bidirectional LSTMs (BLSTMs), and a Bidirectional version of the Prioritized Grid LSTM (BPGLSTM) model. We propose a system combination for three purely sequence trained recognition systems based on lattice-free maximum mutual information, 4-gram language model re-scoring, and system combination using the minimum Bayes risk decoding criterion. The best word error rate we obtained on the MGB-3 Arabic development set using a 4-gram re-scoring strategy is 42.25% for a chain BLSTM system, compared to 65.44% baseline for a DNN system.

International Journal of Intelligent Computing and Information Sciences

End-to-end deep learning approach has greatly enhanced the performance of speech recognition systems. With deep learning techniques, the overfitting stills the main problem with a little data. Data augmentation is a suitable solution for the overfitting problem, which is adopted to improve the quantity of training data and enhance robustness of the models. In this paper, we investigate data augmentation method for enhancing Arabic automatic speech recognition (ASR) based on end-to-end deep learning. Data augmentation is applied on original corpus for increasing training data by applying noise adaptation, pitch-shifting, and speed transformation. An CNN-LSTM and attentionbased encoder-decoder method are included in building the acoustic model and decoding phase. This method is considered as state-of-art in end-to-end deep learning, and to the best of our knowledge, there is no prior research employed data augmentation for CNN-LSTM and attention-based model in Arabic ASR systems. In addition, the language model is built using RNN-LM and LSTM-LM methods. The Standard Arabic Single Speaker Corpus (SASSC) without diacritics is used as an original corpus. Experimental results show that applying data augmentation improved word error rate (WER) when compared with the same approach without data augmentation. The achieved average reduction in WER is 4.55%.

Ingénierie des systèmes d information

Speech recognition is an essential ability of human beings and is crucial for communication. Consequently, automatic speech recognition (ASR) is a major area of research that is increasingly using artificial intelligence techniques to replicate this human ability. Among these techniques, deep learning (DL) models attract much attention, in particular, convolutional neural networks (CNN) which are known due to their power to model spatial relationships. In this article, three CNN architectures that performed well in recognized competitions were implemented to compare their performance in Arabic speech recognition; these are the well-known models AlexNet, ResNet, and GoogLeNet. These models were compared based on a corpus composed of Arabic spoken digits collected from various sources, including messaging and social media applications, in addition to an online corpus. The architectures of AlexNet, ResNet, and GoogLeNet achieved respectively an accuracy of 86.19%, 83.46%, and 89.61%. The results show the superiority of GoogLeNet, and underline the potential of CNN architectures to model acoustic features of low-resource languages such as Arabic.

This chapter presents the first end-to-end recipe for an Arabic speech-to-text transcription system using the lexicon free Recurrent Neural Networks (RNNs). The developed approach does not depend on Hidden Markov Models (HMMs), Gaussian Mixture Models (GMMs), or decision trees. In addition, a character based decoder is used for searching to avoid using a word lexicon. The Connectionist Temporal Classification (CTC) objective function is used to maximize the output character sequences given the acoustic features as an input. The recipe was evaluated using 1200 hours corpus of Aljazeera multi-Genre broadcast programs. On the development set, we report Word Error Rate (WER) of 12.03% for non-overlapped speech.

2009 IEEE Workshop on Automatic Speech Recognition & Understanding, 2009

Abstract Short vowels in Arabic are normally omitted in written text which leads to ambiguity in the pronunciation. This is even more pronounced for dialectal Arabic where a single word can be pronounced quite differently based on the speaker's nationality, level of education, social class and religion. In this paper we focus on pronunciation modeling for Iraqi-Arabic speech. We introduce multiple pronunciations into the Iraqi speech recognition lexicon, and compare the performance, when weights computed via forced alignment are assigned to ...

This paper presents the systems submitted by the safina team to the Arabic Dialect Identification (ADI) shared task at the VarDial Evaluation Campaign 2018. The ADI shared task included five Arabic dialects: Modern Standard Arabic (MSA), Egyptian, Gulf, Levantine, and North-African. The proposed approach is to use character-level convolution neural network in addition to dialect embedding vectors, a low dimensional representation extracted from linguistic features, to distinguish the 5 dialects. We submitted three models with the same architecture except for the first layer. The first system uses one-hot character representation as input to the convolution layer. The second system uses an embedding layer before the convolution layer. The third system uses a recurrent layer before the convolution layer. The best results were obtained using the third model achieving 57.6% F1-score, ranked the second among six teams. 1

IEEE access, 2024

Speech recognition is a captivating process that revolutionizes human-computer interactions, allowing us to interact and control machines through spoken commands. The foundation of speech recognition lies in understanding a given language's linguistic and textual characteristics. Although automatic speech recognition (ASR) systems flawlessly convert speech into text for various international languages, their implementation for Arabic remains inadequate. In this research, we diligently explore the current state of Arabic ASR systems and unveil the challenges encountered during their development. We categorize these challenges into two groups: those specific to the Arabic language and those that are more general. Additionally, we propose strategies to overcome these obstacles and emphasize the need for ASR architectures tailored to the Arabic language's unique grammatical and phonetic structure. In addition, we provide a comprehensive and explicit description of various feature extraction methods, language models, and acoustic models utilized in the Arabic ASR system.

2022

Recent advancements in self-supervised speech-representation learning for automatic speech recognition (ASR) approaches have significantly improved the results on many benchmarks with low-cost data labeling. In this paper, we train two self-supervised frameworks for ASR, namely wav2vec, and data2vec, in which we conduct multiple experiments and analyze their results. Furthermore, we introduce Aswat dataset, which covers multiple genres and features speakers with vocal variety. Aswat contains 732 hours of clean Arabic speech that can be used in the pretraining task for learning latent speech representations, which results in achieving a lower word error rate (WER) in Arabic ASR. We report the baseline results and achieve state-ofthe-art WERs of 11.7% and 10.3% on Common Voice (CV) and the second round of Multi-Genre Broadcast (MGB-2) respectively, as a result of including our dataset Aswat.

2011

Availability of databases is a necessity in the speech processing field. The publically available databases in Arabic language are few. In this paper we describe a rich database for Arabic language. The database is rich in many dimensions: in text, environments, microphone type, number of recording sessions, recording system, the transmission channel, the country of origin, and the mother language. This richness makes the database an important resource for research in Arabic Language processing and very useful in many speech processing tasks, such as speaker recognition, speech recognition, and accent identification. The speakers were speaking in Modern Standard Arabic (MSA).

2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2011

We describe the Arabic broadcast transcription system elded by IBM in the GALE Phase 4 machine translation evaluation. Key advances over our Phase 3.5 system include improvements to contextdependent modeling in vowelized Arabic acoustic models; the use of neural-network features provided by the International Computer Science Institute; Model M language models; a neural network language model that uses syntactic and morphological features; and improvements to our system combination strategy. These advances were instrumental in achieving a word error rate of 8.9% on the Phase 4 evaluation set, and an absolute improvement of 1.6% word error rate over our 2008 system on the unsequestered Phase 3.5 evaluation data.