Cross Language Information Retrieval

Natural Language Processing is a programmed approach to analyze text that is based on both a set of theories and a set of technologies. This forum aims to bring together researchers who have designed and build software that will analyze, understand, and generate languages that humans use naturally to address computers.

On-line keyword searching from documents in Chinese tends to use inverted indexing as the main technique, which has its difficulties. Suffix Array is widely used for processing text in Western languages. However, it fails to get widely used in Chinese processing because of the speciality of Chinese. Suffix Array is a powerful tool. However it costs too much space. That is the major bottleneck of suffix Array. A data
structure called Keyword-driven Suffix Array is proposed in this paper for on-line keyword searching from documents in Chinese, based on observation of on-line search pattern and traits of Chinese. Space efficiency is improved a lot using this data structure. When the document database is large enough, space efficiency is improved by about 5/6 using this data structure without sacrificing its time efficiency.

Linguistics and the science of Anthropology have much in common. In fact, to a large
extent the two fields overlap. Field workers utilize research models of the ethnographic type
as well as approaches that are experimental, methods that are qualitative as well as
quantitative, for example. The study of language contact and bilingualism, topic of this paper,
presents a good opportunity for drawing on contributions from the two overlapping fields.
The focus of the following review of current research will be mainly from the cognitive
science point of view, divided into four areas of recent work: (1) bilingual development, first
and second language learning and language loss, (2) creolization and convergence, (3)
codeswitching and borrowing, and (4) problems related to the distinction between language
and dialect.

Natural Language Processing is a programmed approach to analyze text that is based on both a set of theories and a set of technologies. This forum aims to bring together researchers who have designed and build software that will analyze, understand, and generate languages that humans use naturally to address computers.

This paper reports on the first Oromo-English CLIR system that is based on dictionary-based query translation techniques. The basic objective of the study is to design and develop an Oromo-English CLIR system with a view to enable Afaan Oromo speakers to access and retrieve the vast online information sources that are available in English by using their own (native) language queries. We describe the major approaches and procedures that have been used in designing and developing Oromo-English CLIR system together with the information retrieval evaluation experiments recently conducted at CLEF 2006 ad hoc track. The purpose of the current initial evaluation experiments was to assess the over all performance of the Oromo-English CLIR system by using different fields of Afaan Oromo topics. Thus we submitted three official runs (experiments) that differed in terms of utilized fields in the topic set, i.e. title run (OMT), title and description run (OMTD), and title, description and narration run (OMTDN) to CLEF 2006. Since appropriate online language processing resources and information retrieval tools are not available for Afaan Oromo, only limited linguistic resources such as Oromo-English dictionary and Afaan Oromo stemmer that had been designed and developed at our research center were used in conducting the experiments. Yet we found the performances of our evaluation experiments at CLEF 2006 very encouraging which is a good news for development and application of CLIR systems in other indigenous African languages.

TWLT is an acronym of Twente Workshop(s) on Language Technology. These workshops on natural language theory and technology are organised by the Parlevink Project, a language theory and technology project of the . For each workshop proceedings are published containing the papers that were presented. TWLT 14, has been organised together with the German Research Center for Artificial Intelligence, DFKI Saarbrücken, Germany. The idea for this workshop grew out of a longstanding cooperation between the University of Twente, TNO-TPD in Delft and DFKI. This co-operation manifested itself for the first time in the Twenty-One project, which inspired a whole series of other projects, such as Pop-Eye and Olive, but which also led to a close contact and exchange with independently established projects such as Mulinex and MIETTA for which DFKI was responsible. All of these projects had in common that they were funded by the Telematics Application Programme of the European Commission, all, except for Twenty-One, by the Language Engineering Sector.

Sentence alignment consists in estimating which sentence or sentences in the source language correspond with which sentence or sentences in a target language. We present in this paper a new approach to aligning sentences from a parallel corpus based on a cross-language information retrieval system. This approach consists in building a database of sentences of the target text and considering each sentence of the source text as a "query" to that database. The cross-language information retrieval system is a weighted Boolean search engine based on a deep linguistic analysis of the query and the documents to be indexed. This system is composed of a multilingual linguistic analyzer, a statistical analyzer, a reformulator, a comparator and a search engine. The multilingual linguistic analyzer includes a morphological analyzer, a part-of-speech tagger and a syntactic analyzer. The linguistic analyzer processes both documents to be indexed and queries to produce a set of normalize...

Natural Language Processing is a programmed approach to analyze text that is based on both a set of theories and a set of technologies. This forum aims to bring together researchers who have designed and build software that will analyze, understand, and generate languages that humans use naturally to address computers.

Integrating Different Strategies for Cross-Language Information Retrieval in the MIETTA Project Paul Buitelaar, Klaus Netter, Feiyu Xu DFKI Language Technology Lab Stuhlsatzenhausweg 3, 66123 Saarbrücken, Germany {paulb, netter, feiyu}@ dfki. de ABSTRACT In this paper ...

Today we are living in modern Internet era. We can get all our information from the internet anytime and from anywhere using a desktop PC or a smart phone. However, the underlying technology for relevant information retrieval from the internet is not so trivial, as internet is a huge repository of all different kinds of information. Moreover, data collection in the intern Retrieving the relevant information from the internet in the Big Data era is same as finding a needle in the haystack. This paper explores information retrieval models and experiments Semantic Indexing (LSI) first and then with the more efficient topic modeling algorithm of Latent Dirichlet Allocation (LDA). Comparisons between the two models are described clearly and concisely in their ef topic modeling. Various applications of topic modeling are also reviewed in this paper from the literature.

Cross-language information retrieval (CLIR) systems cater for the requirements of users who need to access a pool of information published in a language that they do not speak. A CLIR system uses an information retrieval (IR) architecture with the addition of a machine translation component.
The purpose of this paper was to evaluate the performance of two statistical machine translation (SMT) systems within a CLIR pipeline: KantanMT, a cloud-based machine translation (MT) platform, and Moses, an open-source software. In order to train the MT systems we used the 1,073,225-sentence-pair parallel bilingual (Greek-English) EMEA corpus (Tiedemann 2009) and the 62,452-sentence-pair parallel bilingual (Greek-English) QTLP corpus. Both corpora were from the medical domain.
For the IR part of the experiment, the OHSUMED medical test collection was used. OHSUMED (Hersh et al 1994) contains 233,445 abstracts from MEDLINE, and 63 English queries along with their correct answers. Prof. Theodore Kalamboukis from the Athens University of Economics and Business (AUEB) very generously provided the Greek version of the queries (Kotsonis et al 2008). The Greek queries were then translated back into English using the two MT systems. Finally, the machine translated queries were used for the retrieval of relevant documents from the OHSUMED database using Apache Solr.
Three experiments were conducted: One using the queries translated by KantanMT and Moses to retrieve relevant documents from the OHSUMED collection, one where we calculated the BLEU score for each MT system using the independent 2,469-sentence ECDC corpus and one using the original human-produced queries to retrieve relevant documents. The top 10 retrieved relevant documents from each set of 63 queries (KantanMT, Moses, human-produced) were compared to the gold standard provided in the OHSUMED collection.
In the first experiment, KantanMT’s precision (0.12) was found to be slightly better than the precision of Moses (0.10) and the same applied to F-measure, where KantanMT achieved a score of 0.07 and Moses a score of 0.05. However, both systems produced the same recall score (0.06). In the second experiment, Moses was found to yield a higher BLEU score (17.72) than KantanMT (11.74), which seems to confirm the theory that there is no correlation between translation quality and IR performance. In the third experiment, we concluded that the original English queries produced around double the F-measure (0.13) compared to the queries from KantanMT (0.07) and Moses (0.05).
Finally, we conclude that there is a lot of room for more research on the use of full SMT systems in CLIR applications, especially involving the Greek language.

We have aligned Japanese and English news articles and sentences to make a large parallel corpus. We first used a method based on cross-language information retrieval (CLIR) to align the Japanese and English articles and then used a method based on dynamic programming (DP) matching to align the Japanese and English sentences in these articles. However, the results included many incorrect alignments. To remove these, we propose two measures (scores) that evaluate the validity of alignments. The measure for article alignment uses similarities in sentences aligned by DP matching and that for sentence alignment uses similarities in articles aligned by CLIR. They enhance each other to improve the accuracy of alignment. Using these measures, we have successfully constructed a largescale article and sentence alignment corpus available to the public.

Resumo -Este artigo procura identificar dificuldades encontradas por estudantes da área médica na busca de informação técnica de medicina. O objetivo principal é mostrar a importância da recuperação de informação na área médica por meio de máquinas de busca e do Sistema Morphosaurus. Aplicou-se um questionário contendo perguntas pertinentes à área médica a alunos do curso de medicina, visando mensurar o seu conhecimento em informática, idioma inglês e portais para busca de informações. A partir dos resultados, discutem-se formas pelas quais as máquinas de busca podem prestar apoio direcionado a profissionais da área de saúde.

This paper reviews literature on dictionary-based cross-language information retrieval (CLIR) and presents CLIR research done at the University of Tampere (UTA). The main problems associated with dictionary-based CLIR, as well as appropriate methods to deal with the problems are discussed. We will present the structured query model by Pirkola and report findings for four different language pairs concerning the effectiveness of query structuring. The architecture of our automatic query translation and construction system is presented. This paper contributes a literature review on dictionary-based CLIR and summarizes our own research at the University of Tampere (UTA). We will discuss the methods, linguistic and technical problems, and empirical findings in dictionary-based CLIR. At UTA, our CLIR research is based on Pirkola's method (Pirkola 1998) which consists of dictionary translation and the structured query model Järvelin 1998, Pirkola 1998). The structuring of queries refers to the grouping of search keys, and the use of proper query operators. We summarize empirical findings for four different language pairs, namely Finnish to English, English to Finnish, Swedish to English, and German to English query translation are reported. We will discuss briefly other CLIR related research done at UTA, which involves language typology research for CLIR , and studies of statistical methods for keyword assessment (Pirkola and Järvelin 2001a).

In this paper, we describe a web-based multilingual tool for Arabic information retrieval based on ontology in the legal domain. We illustrate the manual construction of the ontology and the way it is edited using Protégé2000. Using Arabic (UN) documents we identify the legal terms and the semantic relations between them before mapping them onto their position in the ontology. The process of search for Arabic documents uses a query expansion. In addition to finding related documents in Arabic, the retrieval process is further enriched by enabling the user to translate the query into another language, The query expansion is achieved using a semantic word thesaurus namely, Wordnet, which is available on the Web. with a view to retrieve documents in these languages too. A set of query words is used to enable the machine translation of the query from Arabic into English and from Arabic into French.

Data available on the web is growing at an exponential rate, creating Knowledge or extracting information is of paramount importance. Information Retrieval (IR) plays a crucial role in Knowledge management as it helps us to find the relevant information from the existing data. This paper compares the performance of keyword-based retrieval and other architectural styles in information retrieval system with ontology-based retrieval on documents in regional language

Despite all existing methods to identify a person, such as fingerprint, iris, and facial recognition, the personal name is still one of the most common ways to identify an individual. In this paper, we propose two novel algorithms: The first one uses sound techniques to create a multi-dimensional vector representation of names to compute a degree of similarity. This algorithm compares names that are written in different languages (cross-language) and transliterated names in English. The second algorithm builds on the first one and measures the similarities between full names, taking into account the full name structure. We evaluate the algorithms for transliterated names and across languages using Arabic and English as an example. Significant results are achieved by both algorithms compared with other existing algorithms.

We employ Automorphology, an MDL-based algorithm that determines the suffixes present in a language-sample with no prior knowledge of the language in question, and describe our experiments on the usefulness of this approach for Information Retrieval, employing this stemmer in a SMARTbased IR engine.

In this paper, we present our Hindi to English and Marathi to English CLIR systems developed as part of our participation in the CLEF 2007 Ad-Hoc Bilingual task. We take a query translation based approach using bi-lingual dictionaries. Query words not found in the dictionary are transliterated using a simple rule based transliteration approach. The resultant transliteration is then compared with the unique words of the corpus to return the 'k' words most similar to the transliterated word. The resulting multiple translation/transliteration choices for each query word are disambiguated using an iterative page-rank style algorithm which, based on term-term co-occurrence statistics, produces the final translated query. Using the above approach, for Hindi, we achieve a Mean Average Precision (MAP) of 0.2366 using title and a MAP of 0.2952 using title and description. For Marathi, we achieve a MAP of 0.2163 using title.

It is well known that the use of a good Machine Transliteration system improves the retrieval performance of Cross-Language Information Retrieval (CLIR) systems when the query and document languages have different orthography and phonetic alphabets. However, the effectiveness of a Machine Transliteration system in CLIR is limited by its ability to produce relevant transliterations, i.e. those transliterations which are actually present in the relevant documents. In this work, we propose a new approach to the problem of finding transliterations for out-of-vocabulary query terms. Instead of "generating" the transliterations using a Machine Transliteration system, we "mine" them, using a transliteration similarity model, from the top CLIR results for the query. We treat the query and each of the top results as "comparable" documents and search for transliterations in these comparable document pairs. We demonstrate the effectiveness of our approach using queries in two languages from two different linguistic families to retrieve English documents from two standard CLEF collections. We also compare our results with those of a state-of-the-art Machine Transliteration system.

Information Retrieval, the results is not encouraging. Proper names are problematic for cross language information retrieval (CLIR), detecting and extracting proper noun in Arabic language is a primary key for improving the effectiveness of the system. The value of information in the text usually is determined by proper nouns of people, places, and organizations, to collect this information it should be detected first. The proper nouns in Arabic language do not start with capital letter as in many other languages such as English language so special treatment is required to find them in a text. Little research has been conducted in this area; most efforts have been based on a number of heuristic rules used to find proper nouns in the text. In this research we use a new technique to retrieve proper nouns from the Arabic text by using set of keywords and particular rules to represent the words that might form a proper noun and the relationships between them.

This paper describes the participation of MIRACLE in NTCIR 2005 CLIR task. Although our group has a strong background and long expertise in Computational Linguistics and Information Retrieval applied to European languages and using Latin and Cyrillic alphabets, this was our first attempt on East Asian languages. Our main goal was to study the particularities and distinctive characteristics of Japanese,

The main objective of our project is to extract clinical information from thoracic radiology reports in Portuguese using Machine Translation (MT) and cross language information retrieval techniques. To accomplish this task we need to evaluate the involved machine translation system. Since human MT evaluation is costly and time consuming we opted to use automated methods. We propose an evaluation methodology using NIST/BLEU and METEOR algorithms and a controlled medical vocabulary, the Unified Medical Language System (UMLS). A set of documents are generated and they are either machine translated or used as evaluation references. This methodology is used to evaluate the performance of our specialized Portuguese-English translation dictionary. A significant improvement on evaluation scores after the dictionary incorporation into a commercial MT system is demonstrated. The use of UMLS and automated MT evaluation techniques can help the development of applications on the medical domain. ...

Цель работы: изучение особенностей научно-технического стиля в аспекте машинного перевода. Задачи работы: описать в общих чертах стилистику научно-технического текста; описать принципы работы, основные типы систем машинного перевода; изучить особенности языковых средств научного стиля; выяснить какие из языковых средств остаются в тексте после его перевода; сравнить текст, переведенный машиной и профессиональным переводчиком. Объектом исследования являются преобразования, возникающие при машинном переводе текста научно-технической литературы. Предметом исследования являются особенности структурно-семантических трансформаций при машинном переводе определенной научной работы с английского языка на русский. В ходе исследования были использованы следующие методы: аналитический; сравнительный; изучение монографических публикаций и статей; метод сплошной выборки; методы лингвистического анализа; метод обобщения. Теоретическая значимость работы заключается в попытке собрать и систематизировать информацию о малоизученном аспекте взаимодействия двух областей знаний: об особенностях стиля научно-технической литературы; об особенностях систем машинного перевода. Практическая ценность работы определяется: описанием особенностей научного стиля в виде диаграмм и схем, что, по нашему мнению, является наиболее доходчивым и методически оправданным. комплексом собранных сведений относительно систем машинного перевода, которые также представлены в виде диаграмм и схем; приведенным стилистическим анализом отрывков из классического труда Д. Кнута «Искусство программирования»; сопоставлением машинного и «человеческого» перевода приведенного отрывка. В качестве теоретической основы исследования были использованы как классические работы А. И. Гальперина И. В. Арнольд, посвященные стилистике английского языка, Я. И. Рецкера, А. Д. Швейцера, посвященные общей теории перевода, так и современные труды Ю. Н. Марчука, П. Н. Хроменкова, Б. Н. Рахимбердиева, посвященные машинному переводу. Особенно хотелось бы отметить работу С. Рассела, в которой дается объяснение принципов работы многих приложений искусственного интеллекта, в том числе машинного перевода.

In this article we present an advanced version of Dual-PECCS, a cognitively-inspired knowledge representation and reasoning system aimed at extending the capabilities of artificial systems in conceptual categorization tasks. It combines different sorts of common-sense cat-egorization (prototypical and exemplars-based categorization) with standard monotonic cate-gorization procedures. These different types of inferential procedures are reconciled according to the tenets coming from the dual process theory of reasoning. On the other hand, from a representational perspective, the system relies on the hypothesis of conceptual structures represented as heterogeneous proxytypes. Dual-PECCS has been experimentally assessed in a task of conceptual categorization where a target concept illustrated by a simple common-sense linguistic description had to be identified by resorting to a mix of categorization strategies, and its output has been compared to human responses. The obtained results suggest that our approach can be beneficial to improve the representational and reasoning conceptual capabilities of standard cognitive artificial systems, and –in addition– that it may be plausibly applied to different general computational models of cognition. The current version of the system , in fact, extends our previous work, in that Dual-PECCS is now integrated and tested into two cognitive architectures, ACT-R and CLARION, implementing different assumptions on the underlying invariant structures governing human cognition. Such integration allowed us to extend our previous evaluation.

Searching is inherently a user-centered process; people pose the questions for which machines seek answers, and ultimately people judge the degree to which retrieved documents meet their needs. Rapid development of interactive systems that use queries expressed in one language to search documents written in another poses five key challenges: (1) interaction design, (2) query formulation, (3) cross-language search, (4) construction of translated summaries, and (5) machine translation. This article describes the design of MIRACLE, an easily extensible system based on English queries that has previously been used to search French, German, and Spanish documents, and explains how the capabilities of MIRACLE were rapidly extended to accommodate Cebuano and Hindi. Evaluation results for the cross-language search component are presented for both languages, along with results from a brief full-system interactive experiment with Hindi. The article concludes with some observations on directions for further research on interactive crosslanguage information retrieval.

This paper introduces a general framework for the use of translation probabilities in cross-language information retrieval based on the notion that information retrieval fundamentally requires matching what the searcher means with what the author of a document meant. That perspective yields a computational formulation that provides a natural way of combining what have been known as query and document translation. Two well-recognized techniques are shown to be a special case of this model under restrictive assumptions. Cross-language search results are reported that are statistically indistinguishable from strong monolingual baselines for both French and Chinese documents.

The problem of finding documents written in a language that the searcher cannot read is perhaps the most challenging application of cross-language information retrieval technology. In interactive applications, that task involves at least two steps: (1) the machine locates promising documents in a collection that is larger than the searcher could scan, and (2) the searcher recognizes documents relevant to their intended use from among those nominated by the machine. This article presents the results of experiments designed to explore three techniques for supporting interactive relevance assessment: (1) full machine translation, (2) rapid term-by-term translation, and (3) focused phrase translation. Machine translation was found to better support this task than term-by-term translation, and focused phrase translation further improved recall without an adverse effect on precision. The article concludes with an assessment of the strengths and weaknesses of the evaluation framework used in this study and some remarks on implications of these results for future evaluation campaigns.

In our participation to the 2010 LogCLEF track we focused on the analysis of the European Library (TEL) logs and in particular we experimented with the identification of the natural language used in the queries. Language identification is in fact a key task within Cross Language Information Retrieval systems and the challenge is particularly difficult in the case of search queries where the contextual information available is scarce; function words (grammar particles highly connotative of a specific language like prepositions, pronouns, conjunctions, etc) are usually missing and the relevant presence of Named Entities can be misleading for the correct identification of the language used in the query. In order to face this challenge with acceptable performances the techniques applied should be different form the ones adopted for language guessing with more extensive and syntactically richer text fragments, like metadata or textual documents. In particular we experimented combining together different strategies: corpus based, character model based and a priori hypothesis. Since no official evaluation of the task is available we manually evaluated a sample of 100 queries and the results obtained are quite promising.

European languages. Monolingual Chinese retrieval experiments, by contrast often find that character bigrams perform as well as (and sometimes better than) automatically segmented words. During the Mandarin-English Information (MEI) project at the Johns Hopkins Summer 2000 Workshop, Pirkola's structured queries were compared with an alternative technique known as balanced translation. The results suggested that balanced translation coupled with post-translation character bigram resegmentation might outperform Pirkola's word-based technique. The NTCIR-2 English/Chinese Information Retrieval (ECIR) evaluation provided the opportunity to replicate this experiment on a far larger collection. The results showed that on the ECIR collection, (1) the best word-based and overlapping character bigram-based techniques we tried performed comparably; (2) with word-based retrieval, Pirkola's structured queries outperformed balanced translation; (3) with overlapping character bigrambased retrieval, a technique analogous to Pirkola's method performed comparably with balanced translation; and (4) using all known translation alternatives significantly outperformed using the three most frequent translation alternatives.

Starting in 1997, the National Institute of Standards and Technology conducted 3 years of evaluation of cross-language information retrieval systems in the Text REtrieval Conference (TREC). Twentytwo participating systems used topics (test questions) in one language to retrieve documents written in English, French, German, and Italian. A large-scale multilingual test collection has been built and a new technique for building such a collection in a distributed manner was devised.

This paper reports the work of Middlesex University for the CLEF bilingual task. We have carried out experiments using Portuguese queries to retrieve documents in English. The approach used was Latent Semantic Indexing, which is an automatic method not requiring dictionaries or thesauri. We describe the methods used along with an analysis of the results obtained.

The paper presents a study of a challeng-ing task in machine translation and cross-language information retrieval – transla-tion of toponyms. Due to their linguistic and extra-linguistic nature, toponyms de-serve a special treatment. The overall translation process includes ...

Dies ist eine Internet-Sonderausgabe des Aufsatzes „Language-specific encoding in multilingual corpora: Requirements and solutions“ von Jost Gippert (1999). Sie sollte nicht zitiert werden. Zitate sind der Originalausgabe in Multilinguale Corpora: Codierung, Strukturierung, Analyse. 11. Jahrestagung der Gesellschaft für Linguistische Datenverarbeitung (ed. J. Gippert / P. Olivier), Praha 1999, 371-384 zu entnehmen.

We present a report on our participation in the Indonesian-English ad hoc bilingual task of the 2006 Cross-Language Evaluation Forum (CLEF). This year we compare the use of several language resources to translate Indonesian queries into English. We used several readable machine dictionaries to perform the translation. We also used two machine translation techniques to translate the Indonesian queries. In addition to translating an Indonesian query set into English, we also translated English documents into Indonesian using the machine readable dictionaries and a commercial machine translation tool. The results show performing the task by translating the queries is better than translating the documents. Combining several dictionaries produced better result than only using one dictionary. However, the query expansion that we applied to the translated queries using the dictionaries reduced the retrieval effectiveness of the queries.

This article presents an ongoing project that aims to design and develop a robust and agile web-based application capable of semi-automatically compiling monolingual and multilingual comparable corpora, which we named iCompileCorpora. The dimensions that comprise iCompileCorpora can be represented in a layered model comprising a manual, a semi-automatic and a Cross-Language Information Retrieval (CLIR) layer. This design option will not only permit to increase the flexibility of the compilation process, but also to hierarchically extend the manual layer features to the semi-automatic web-based layer and then to the semi-automatic CLIR layer. The manual layer presents the option of compiling monolingual or multilingual corpora. It will allow the manual upload of documents from a local or remote directory onto the platform. The second layer will permit the exploitation of either monolingual or multilingual corpora mined from the Internet. As nowadays there is an increasing demand for systems that can somehow cross the language boundaries by retrieving information of various languages with just one query, the third layer aims to answer this demand by taking advantage of CLIR techniques to find relevant information written in a language different from the one semi-automatically retrieved by the methodology used in the previous layer.

The goal of Cross Language Information Retrieval (CLIR) is to provide users with access to information that is in a different language from their queries. It has the ability to issue a query in one language and retrieve documents in another. This is achieved by designing a system where a query in one language can be compared with documents in another. Afaan Oromo is one of the major languages that are widely spoken and used in Ethiopia. Despite the fact that Afaan Oromo has a large number of speakers, little effort has been put in conducting researches which aim at making English documents available to Afaan Oromo speakers. This study is, therefore, an attempt to develop Afaan Oromo-English CLIR system which enables Afaan Oromo native speakers to access and retrieve the vast online information sources that are available in English by writing queries using their own (native) language. Evaluation of the system is conducted by both monolingual and bilingual retrievals. The performance of the system was measured by recall and precision.

In this our rst joint participation as the CoLesIR group, our team has participated in the Portuguese monolingual ad-hoc task and in all robust ad-hoc tasks |all monolingual tasks, the English-to-German bilingual task, and the multilingual task. We have developed an n-gram model inspired by the previous work of the Johns Hopkins University Applied Physics Lab. Our approach makes generalized use of freely available resources |such as the Europarl parallel corpus, the GIZA++ wordalignment toolkit, and the Terrier retrieval platform|, and employs a new n-gram direct translation technique. This new technique takes as input previously existing aligned word lists and obtains as output aligned n-gram lists. It can also handle word translation probabilities, as in the case of statistical word alignments. This new n-gram-based approach shares the main advantages of the original proposal. This solution avoids the need for word normalization during indexing or translation, and it can also deal...

Large-scale parallel corpus has become a reliable resource to cross the language barriers between the user and the web. These parallel texts provide the primary training material for statistical translation models and testing machine translation systems. Arabic-English parallel texts are not available in sufficient quantities and manual construction is time consuming. Therefore, this paper presents a technique that aims to construct an Arabic-English corpus automatically through web mining. The proposed technique is straightforward, automated, and portable to any pair of languages.

The paper reports on experiments carried out in transitive translation, a branch of cross-language information retrieval (CLIR). By transitive translation we mean translation of search queries into the language of the document collection through an intermediate (or pivot) language. In our experiments, queries constructed from CLEF 2000 and 2001 Swedish, Finnish and German topics were translated into English through Finnish and Swedish by an automated translation process using morphological analyzers, stopword lists, electronic dictionaries, n-gramming of untranslatable words, and structured and unstructured queries. The results of the transitive runs were compared to the results of the bilingual runs, i.e. runs translating the same queries directly into English. The transitive runs using structured target queries performed well. The differences ranged from-6.6 to +2.9 % units (or-25.5 to + 7.8 %) between the approaches. Thus transitive translation challenges direct translation and considerably simplifies global CLIR efforts.

For TREC 10 we participated in the Named Page Finding Task and the Cross-Lingual Task. In the web track, we explored the use of linear combinations of term collections based on document structure. Our goal was to examine the effects of different term collection statistics based on document structure in respect to known item retrieval. We parsed documents into structural components and built specific term indexes based on that document structure. Each of those indices have their own collection statistics for term weighting based on the type of language used for that structure in the collection. For producing a single ranked list, we examined a weighted linear combination approach to merging results. Our approach to known item retrieval was equal or above the median 58% of the time and 71% above the mean score of submitted runs. In the Arabic track we participated in Arabic Cross-language Information Retrieval (CLIR) and in Arabic monolingual information retrieval. For the monolingual retrieval, we examined the use of two stemming algorithms. The first is a deeper approach, and the second is a pattern-based approach. For the Arabic CLIR, we explored the retrieval effectiveness by using a machine translation (MT) system and translation probabilities obtained from parallel documents collection provided by the United Nations (UN).

Information Retrieval, the results is not encouraging. Proper names are problematic for cross language information retrieval (CLIR), detecting and extracting proper noun in Arabic language is a primary key for improving the effectiveness of the system. The value of information in the text usually is determined by proper nouns of people, places, and organizations, to collect this information it should be detected first. The proper nouns in Arabic language do not start with capital letter as in many other languages such as English language so special treatment is required to find them in a text. Little research has been conducted in this area; most efforts have been based on a number of heuristic rules used to find proper nouns in the text. In this research we use a new technique to retrieve proper nouns from the Arabic text by using set of keywords and particular rules to represent the words that might form a proper noun and the relationships between them.

While a lot of research has focused on the effectiveness of system functionality, few studies have examined information needs and social aspects related to cross-language information retrieval. This chapter aims to speculate the human and social aspects of cross-language information retrieval. It explores CLIR users' unique social and cultural contexts, their psychological and cognitive structures, and their distinctive relevance judgment. It examines in depth the barriers embedded in cultural, linguistic, and cognitive dimensions, which might hinder further advancement in cross-language information retrieval.

In this paper, we present some approaches to improve translation accuracy in web-based translation extraction. In previous work, the term extraction techniques that researchers used are proposed under large static corpus. We proposed some approaches that can improve the translation accuracy in web-based translation extraction which relies on small dynamic small corpus. We also analyzed the difference in using local text corpus and web corpus as disambiguation source.

Named entity (NE) translation plays an important role in many applications, such as information retrieval and machine translation. In this paper, we focus on translating NEs from Korean to Chinese in order to improve Korean-Chinese cross-language information retrieval (KCIR). The ideographic nature of Chinese makes NE translation difficult because one syllable may map to several Chinese characters. We propose a hybrid NE translation system. First, we integrate two online databases to extend the coverage of our bilingual dictionaries. We use Wikipedia as a translation tool based on the inter-language links between the Korean edition and the Chinese or English editions. We also use Naver.com's people search engine to find a query name's Chinese or English translation. The second component of our system is able to learn Korean-Chinese (K-C), Korean-English (K-E), and English-Chinese (E-C) translation patterns from the web. These patterns can be used to extract K-C, K-E and E-C pairs from Google snippets. We found KCIR performance using this hybrid configuration over five times better than that a dictionary-based configuration using only Naver people search. Mean average precision was as high as 0.3385 and recall reached 0.7578. Our method can handle Chinese, Japanese, Korean, and non-CJK NE translation and improve performance of KCIR substantially.