Scalable Multilingual Information Access

Lecture Notes in Computer Science, 2004

We investigates the effectiveness of language-dependent approaches to document retrieval, such as stemming and decompounding, and constrast them with language-independent approaches, such as character n-gramming. In order to reap the benefits of more than one type of approach, we also consider the effectiveness of the combination of both types of approaches. We focus on document retrieval in nine European languages: Dutch, English, Finnish, French, German, Italian, Russian, Spanish, and Swedish. We look at four different cross-lingual information retrieval tasks: monolingual, bilingual, multilingual, and domain-specific retrieval. The experimental evidence is obtained using the 2003 test suite of the cross-language evaluation forum (CLEF).

1997

The EU project Twenty-One will support cross language queries in a multilingual document base. A prototype version of the Twenty-One system has been subjected to the Cross Language track tests in order to set baseline performances. The runs were based on query translation using dictionaries and corpus based disambiguation methods.

Computer Speech & Language, 2016

The field of Cross-Language Information Retrieval relates techniques close to both the Machine Translation and Information Retrieval fields, although in a context involving characteristics of its own. The present study looks to widen our knowledge about the effectiveness and applicability to that field of nonclassical translation mechanisms that work at character n-gram level. For the purpose of this study, an n-gram based system of this type has been developed. This system requires only a bilingual machine-readable dictionary of n-grams, automatically generated from parallel corpora, which serves to translate queries previously n-grammed in the source language. n-Gramming is then used as an approximate string matching technique to perform monolingual text retrieval on the set of n-grammed documents in the target language. The tests for this work have been performed on CLEF collections for seven European languages, taking English as the target language. The performance attained, close to the upper baseline, confirms the validity of character n-gram based approaches for Cross Language Information Retrieval tasks, both for indexing-retrieval and translation purposes, these not being tied to a given implementation. NOTICE: this is the authors version of a work that was accepted for publication in Computer Speech and Language. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in

Information processing & …, 2000

In this paper, we present the system MULINEX, a fully implemented system which supports cross-lingual search of the WWW. Users can formulate, expand and disambiguate queries, filter the search results and read the retrieved documents by using only their native language. This multilingual functionality is achieved by the use of dictionary-based query translation, multilingual document categorisation and automatic translation of summaries and documents. The system supports French, German and English and has been installed and tested in the online services of two European internet content and service provider companies. This paper focuses on the techniques and algorithms used in the MULINEX system, explaining how each component works and how it contributes to the overall functionality of the integrated system. The primary system functionalities are outlined from the user perspective, followed by a description of the document database used in the system. The technologies and linguistic resources used in the various system components are then described in detail.

2004

We introduce an interlingua-based approach to cross-language information retrieval, in which queries, as well as documents, are mapped onto a language-independent concept layer on which retrieval operations are performed. This approach is contrasted with one which directly translates non-English queries (German and Portuguese, in our experiments) to English ones which, subsequently, are processed on English documents. We report on the empirical evaluation of both approaches on a large medical document collection (the OHSUMED corpus).

Te exclusion of monolingual natives from cyberspace is a global socioeconomic and cultural problem. Eforts at addressing this problem have been socioeconomic, culminating in training, empowerment, and digital access with the indelible hurt of language inequities. Tis paper is aimed at the cyber-inclusion of monolingual natives. Since cyber participation is basically through human interaction with cyber-applications in a human language, encapsulating these applications for interaction in any human language will help evade the hurt of language inequities. Information retrieval system (IRS) remains a fundamental cyber-application. Consequently, adopting the design science research methodology, we introduced a lingual agnostic IRS architecture designed on the principle of transparency on user language detection, information translations, and caching. Te detailed design of the architecture was done using the unifed modeling language. Te designed IRS architecture has been implemented using the agile and component-based software engineering approaches. Te resultant lingual agnostic IRS (LAIRS) was evaluated using heuristics and system evaluation methods for parity of language of interaction against the default language and was excellently stable across queries and languages, guaranteeing 86% parity with the default language in the use of other languages for information access and retrieval. Furthermore, it has been shown that LAIRS is the most appropriate IRS to address the problem of language barriers to cyber-inclusion compared with existing IRSs.

ProLISSA conference, 2002

As Internet resources become accessible to more and more countries, there is a need to develop efficient methods for information retrieval across languages. In the present paper, we focus on query expansion techniques to improve the effectiveness of an information retrieval. A combination to a dictionary-based translation and statistical-based disambiguation is indispensable to overcome translation's ambiguity. We propose a model using multiple sources for query reformulation and expansion to select expansion terms and retrieve information needed by a user. Relevance feedback, thesaurus-based expansion, as well as a new feedback strategy, based on the extraction of domain keywords to expand user's query, are introduced and evaluated. We tested the effectiveness of the proposed combined method, by an application to a French-English Information Retrieval. Experiments using CLEF data collection proved a great effectiveness of the proposed combined query expansion techniques.

Journal of the American Society for Information Science and Technology, 2007

IR systems' ability to retrieve highly relevant documents has become more and more important in the age of extremely large collections, such as the WWW. Our aim was to find out how corpus-based CLIR manages in retrieving highly relevant documents. We created a Finnish-Swedish comparable corpus and used it as a source of knowledge for query translation. Finnish test queries were translated into Swedish and run against a Swedish test collection. Graded relevance assessments were used in evaluating the results and three relevance criterion levels -liberal, regular, and stringent -were applied. The runs were also evaluated with generalized recall and precision, which weight the retrieved documents according to their relevance level. The performance of our Comparable Corpus Translation system (Cocot) was compared to that of a dictionary-based query translation program; the two translation methods were also combined. The results indicate that corpus-based CLIR performs particularly well with highly relevant documents. In average precision, Cocot even matched the monolingual baseline on the highest rele-vance level. The performance of the different query translation methods was further analyzed by finding out reasons for poor rankings of highly relevant documents.

The number of web users accessing information’s over the net is growing hastily with each passing day. A massive quantity of information in extraordinary language is to be had on internet which can be accessed by way of all and sundry. Information retrieval(IR) is the technology which offers with locating beneficial statistics from a large series data, specifically unstructured, based and semi-based information. Records retrieval can be labeled into exclusive lessons which include monolingual information retrieval, move language information retrieval (CLIR) and multilingual information retrieval (MLIR). The sector has turn out to be global village naw and the range of data and language boundaries are the principal issues for verbal exchange and cultural trade internationally. To remedy such issues and to take away those barriers, move language information retrieval (CLIR) device is in strong demand now days. CLIR is the IR device in which the question or documents may seem in one-of-a-kind languages. on this paper we are able to offer an overview of the brand new application areas of CLIR. We are able to also overview the strategies which are used in the manner of CLIR research for question and record translation. Furthermore, we will additionally try and identify some of demanding situations and troubles in CLIR systems.

2021

Two key assumptions shape the usual view of ranked retrieval: (1) that the searcher can choose words for their query that might appear in the documents that they wish to see, and (2) that ranking retrieved documents will suffice because the searcher will be able to recognize those which they wished to find. When the documents to be searched are in a language not known by the searcher, neither assumption is true. In such cases, Cross-Language Information Retrieval (CLIR) is needed. This chapter reviews the state of the art for cross-language information retrieval and outlines some open research questions.