Corpus Design For classical Arabic, we have compiled and included the Quranic corpus for its richness of Arabic words and to experiment with the current usage of these words in the modern standard Arabic. Finally, we have used two Arabic dictionaries: a modern one (Al-Mojam Alwaseet) and an old dictionary (Lisan Al- Arab). The size of the collected data is close to 41 MB of text data assembled in 61 thousand files. The corpus has a total of 7.5 million words of which 707,483 words were distinct across all genres (cf. Table1). Starting with a raw corpus collected from different sources and genres, we attempted to build a clean structured corpus. A set of tools were developed to convert the HTML files into UTF-8 plain text files. Other tools were needed to parse the text and split it into sentences. In addition, we used a stemmer to extract the words’ roots (Figure 1). An ongoing work is planned to annotate and POS tag the text corpus, and to convert it into an XML structure for various kind of studying and use on Arabic language (Figure 2). where r is the rank of a word, f is the frequency of occurrence of the word, and c is a constant that depends on the analyzed text. Table 2 and Figure 3 show that our corpus is_ highly balanced. Table 2: Zipf’s results for UJAC, R*= Linear Regression The Corpus Interface and Tools Figure 4: The UJAC System Figure 4, depicts the corpus GUI and the set of tools which we have conducted to experiment with the corpus. The following subsections describe the tools briefly as we have a limited space. Figure 5. Word Form Tool (word = “uall””) Figure 6: Distribution of words based on length Figure 7: Before & after a word 6. Searching for words/roots within the corpus.
Connect with 287M+ leading minds in your field
Discover breakthrough research and expand your academic network
