Influence of NLP Models on Arabic Linguistic Applications

Journal of King Saud University - Computer and Information Sciences

At present, language technologies are instrumental to millions of people, who use them every day with little if any awareness of their existence and role. Popular machine translation systems or web search engines rely more and more on levels of linguistic information automatically overlaid by batch processing tools using language technologies. This development has not only had a huge impact on our daily life, but has also deeply affected the way we think about language as an object of scientific inquiry. The present special issue of JKSU is intended to explore the contribution of computational language models to a better understanding of linguistic, psycholinguistic, sociolinguistic, and literary issues of the Arabic language and culture. The wide array of contributions offered here, ranging from text diacriticisation to psycho-computational modelling of Arabic lexical organisation, bears witness to the maturity of the field and highlight a few general lessons we can learn from current research on Arabic Natural Language Processing. Computational models of language are, primarily, models of language usage. They focus on those aspects of language performance that are involved in, but are not limited to, language acquisition, lexical access, speech and optical character recognition, text translation, text reading, text understanding, knowledge and ontology extraction, sentiment analysis. What all these tasks have in common is their use of language as a means of conveying information to respond to specific communicative needs and goals. Dealing with language performance ultimately requires bringing language variety and subjectivity to inter-subjective invariance, to a shared representation of its content and structure. From this perspective, emotional undertones, variations in style, speed, pitch, handwriting, topic or dialect are superadded signal complications, which are nonetheless inseparable from language performance. On reflection, in real language-based communication, noise is not simply overlaid on the message, but is actually PART of the message. Upon hearing the voice of a speaker or reading a text, one can get a lot of information about the speaker/writer's own sex, age, bodily features, cultural level, social status, personal attitude, political bias and even ethnicity. Arabic language processing happens to throw all these performance-related aspects in sharp relief. A first, significant level of variation can already be found in spelling, where underspecified (i.e. non-diacriticized) written words are amenable to a large vari

ACM Transactions on Asian Language Information Processing, 2009

The Arabic language presents researchers and developers of natural language processing (NLP) applications for Arabic text and speech with serious challenges. The purpose of this article is to describe some of these challenges and to present some solutions that would guide current and future practitioners in the field of Arabic natural language processing (ANLP). We begin with general features of the Arabic language in Sections 1, 2, and 3 and then we move to more specific properties of the language in the rest of the article. In Section 1 of this article we highlight the significance of the Arabic language today and describe its general properties. Section 2 presents the feature of Arabic Diglossia showing how the sociolinguistic aspects of the Arabic language differ from other languages. The stability of Arabic Diglossia and its implications for ANLP applications are discussed and ways to deal with this problematic property are proposed. Section 3 deals with the properties of the Arabic script and the explosion of ambiguity that results from the absence of short vowel representations and overt case markers in contemporary Arabic texts. We present in Section 4 specific features of the Arabic language such as the nonconcatenative property of Arabic morphology, Arabic as an agglutinative language, Arabic as a pro-drop language, and the challenge these properties pose to ANLP. We also present solutions that have already been adopted by some pioneering researchers in the field. In Section 5 we point out to the lack of formal and explicit grammars of Modern Standard Arabic which impedes the progress of more advanced ANLP systems. In Section 6 we draw our conclusion.

Communications of the ACM, 2021

THE TERM NATURAL language refers to any system of symbolic communication (spoken, signed, or written) that has evolved naturally in humans without intentional human planning and design. This distinguishes natural languages such as Arabic and Japanese from artificially constructed languages such as Esperanto or Python. Natural language processing (NLP), also called computational linguistics or human Arabic and Its Challenges Arabic today poses a number of modeling challenges for NLP: morphological richness, orthographic ambiguity,

arXiv (Cornell University), 2022

Large Language Models (LLMs) have ushered in a major paradigm shift in Natural Language Processing (NLP), where large pre-trained Language models (LMs) have become a fundamental component of most NLP tasks. These models are intelligent enough to find relevant and meaningful representations of a language without any supervision. They are used to finetune typical NLP tasks with substantially higher precision than conventional shallow learning techniques. However, training these models requires a massively large corpus that adequately represents a language. Due to the availability of enormous corpora, English LLMs typically perform better than their counterparts. This effort focuses on the design and development of a large Arabic corpus. The corpus comprises over 500 GB of Arabic cleaned text, intended to improve cross-domain knowledge and downstream generalization capability of LLMs. The corpus was employed in the training of a large Arabic LLM. In order to assess the efficacy of the LLM, a variety of typical NLP tasks were fine-tuned. The fine-tuned tasks exhibited a significant boost in accuracy ranging between 4.5 and 8.5%, when compared to those downstreamed from multilingual BERT (mBERT). To the best of our knowledge, this is currently the largest clean and diverse Arabic corpus ever assembled.

2009

This special issue is dedicated to the reporting of the recent Arabic natural language processing advances. A special issue of a journal allows, of course, only a partial representation of the current development in a field. However, we believe that the articles in this special issue are representative of at least some Arabic natural language processing fields. The following is a brief summary of each of the main articles in this issue.

International Journal of Machine Learning and Computing, 2019

This paper defines a Standard Arabic Profiling (SAP) toolset that helps researchers for textual analysis and comparing between different Arabic corpora. Since tools for Arabic language are needed, we present the SAP toolset to simplify the textual analysis process. The approach consists of three profilers: The Part of Speech (POS) profiler that gives statistical analysis for a given document, vocabulary profiler which provides user with an indication out the vocabulary used in a document with reference to Open Source Arabic Corpus (OSAC) of two news agencies (CNN and BBC). The process is accomplished by computing similarity between documents and corpus using Log likelihood measure. Lastly the newly added profiler is the Readability profiler which is used to 1) assess the readability level for a document according to Flesch Reading Ease Readability Formula, and 2) measure the simplicity and ambiguity levels of the document. We described the current part-of-speech for this toolset and how we can extend its functionality to embrace vocabulary and readability profiling. Index Terms-Arabic natural language processing, part-of-speech tagging (POST), text analysis, software. I. INTRODUCTION Research in natural language processing (NLP) has witnessed a rapid progress since 1950. Research in the area of NLP focuses on processing the written text, therefore, it addresses practical applications for the written text. This includes, but is not limited to, opinion mining, information extractions and text summarization. This growth is due to the huge web contents being published. It is important to notice that the new trend in NLP is to apply compositional rather than lexical semantics, leading to the so-called next-generation next narrative-based NLP technology [1]. The Arabic language is considered as one of the most widely used languages in the world. In fact, it is the native for about 330 million in the world. However, the current work on Arabic natural language processing is still limited due to several challenges. The main reasons for these challenges are: the rich morphology of Arabic language, its high degree of ambiguity, and Arabic dialects [2]. As a result; several Arabic tools have been implemented. Some of them are implemented for basic tasks of Arabic NLP: such as "MADAMIRA" [2], segmenters such as "FARASA" [3], libraries such as "AraNLP" [4] and "coreNLP", or as an intermediary step for other NLP steps such as "ADAM" [5]

2013 1st International Conference on Communications, Signal Processing, and their Applications (ICCSPA), 2013

This paper introduces the UNL framework as a collaborative framework that encourages and promotes the participation of linguists and non-linguists in the development of an integral natural language processing workbench. The UNL workbench includes a multitude of user-friendly back-end and front-end applications that facilitate the process of learning the UNL basics, participating in the development of resources within the UNL framework, as well as applications that perform several NLP tasks such as machine translation and editing. This workbench claims the ability to analyze automatically natural languages into their abstract semantic meanings, with the aim of finding the common denominator between all languages.

It is ironical to note that worldwide the Internet content in the Arabic language is mere 1%, whereas 5% of the world population speaks Arabic. This speaks of the disproportionate presence of on-line content of Arabic language as compared to other languages which may be due to many reasons including a lack of experts in the field of the Arabic language. This research study will investigate the impact of such Machine Translation (MT) software and TM tools that are widely used by the Arab community for their academic and business purposes. The study aims at finding whether it is possible to bring a paradigm shift from Arabic Localization to Arabic Globalization; hence, facilitating the usage of NLP techniques in the human interface with the computer. For this study; a few machine translation software (e.g. SYSTRAN, IBM Watson) shall be studied for their content and applications, to determine their usage without human intervention and retaining the meaning of the original text.

Arabic natural language processing (ANLP) has gained increasing interest over the last decade. However, the development of ANLP tools depends on the availability of large corpora. It turns out unfortunately that the scientific community has a deficit in large and varied Arabic corpora, especially ones that are freely accessible. With the Internet continuing its exponential growth, Arabic Internet content has also been following the trend, yielding large amounts of textual data available through different Arabic websites. This paper describes the TALAA corpus, a voluminous general Arabic corpus, built from daily Arabic newspaper websites. The corpus is a collection of more than 14 million words with 15,891,729 tokens contained in 57,827 different articles. A part of the TALAA corpus has been tagged to construct an annotated Arabic corpus of about 7000 tokens, the POS-tagger used containing a set of 58 detailed tags. The annotated corpus was manually checked by two human experts. The methodology used to construct TALAA is presented and various metrics are applied to it, showing the usefulness of the corpus. The corpus can be made available to the scientific community upon authorisation.

2017

This paper presents a language identification system designed to detect the language of each word, in its context, in a multilingual documents as generated in social media by bilingual/multilingual communities, in our case speakers of Algerian Arabic. We frame the task as a sequence tagging problem and use supervised machine learning with standard methods like HMM and Ngram classification tagging. We also experiment with a lexicon-based method. Combining all the methods in a fall-back mechanism and introducing some linguistic rules, to deal with unseen tokens and ambiguous words, gives an overall accuracy of 93.14%. Finally, we introduced rules for language identification from sequences of recognised words.