CONCEPT-BASED INDEXING IN TEXT INFORMATION RETRIEVAL

Information Processing & Management, 2005

Traditional index weighting approaches for information retrieval from texts depend on the term frequency based analysis of the text contents. A shortcoming of these indexing schemes, which consider only the occurrences of the terms in a document, is that they have some limitations in extracting semantically exact indexes that represent the semantic content of a document. To address this issue, we developed a new indexing formalism that considers not only the terms in a document, but also the concepts. In this approach, concept clusters are defined and a concept vector space model is proposed to represent the semantic importance degrees of lexical items and concepts within a document. Through an experiment on the TREC collection of Wall Street Journal documents, we show that the proposed method outperforms an indexing method based on term frequency (TF), especially in regard to the few highest-ranked documents. Moreover, the index term dimension was 80% lower for the proposed method than for the TF-based method, which is expected to significantly reduce the document search time in a real environment.

2005

It is well known that ontologies will become a key piece, as they allow making the semantics of Semantic Web content [16] explicit. In spite of the big advantages that the Semantic Web promises, there are still several problems to solve. Those concerning ontologies include their availability, development and evolution. In the area of information retrieval, the dimension of document vectors plays an important role. Firstly, with higher index dimensions the indexing structures suffer from the "curse of dimensionality" and their efficiency rapidly decreases. Secondly, we may not use exact words when looking for a document, thus we miss some relevant documents. LSI is a numerical method, which discovers latent semantics in documents by creating concepts from existing terms. In this paper we present a basic method of mapping LSI concepts on given ontology (WordNet), used both for retrieval recall improvement and dimension reduction. We offer experimental results for this method on a subset of

International Journal of Computer Applications, 2012

Information retrieval (IR) is the area of study concerned with searching documents or information within documents. The user describes information needs with a query which consists of a number of words. Finding weight of a query term is useful to determine the importance of a query. Calculating term importance is fundamental aspect of most information retrieval approaches and it is traditionally determined through Term Frequency-Inverse Document Frequency (IDF). This paper proposes a new term weighting technique called concept-based term weighting (CBW) to give a weight for each query term to determine its significance by using WordNet Ontology.

2013

This paper presents a functional approach towards the problem domain of Information Retrieval System built upon a narration based search text. The presented system retrieves documents from the background collection by extracting the domain specific significant keywords and other relevant phrases from a given narrative search text. The narrative search text can be a description or scenario which poses a great difficulty in the problem domain to retrieve the relevant document sets with efficiency and accuracy from the background data repository. We have adopted a different approach where the significant keywords are extracted from the narration text to form a search query and alternative sets of queries are also formulated by expanding the search query from a Conceptual Semantic Network built for the purpose. Inclusion of the Conceptual Semantic Network and WordNet synonym sets for the search query expansion plays an important role in the retrieval mechanism. Experiments were carried ...

2002

In the Web search, since users' queries usually consist of only a few words, it is hard to identify their information needs. To solve this problem, many approaches have been tried to expand initial queries and to reweight the terms in the expanded queries using users' relevance judgments. Although relevance feedback is most effective when relevance information about retrieved documents is provided by users, it is not a fully automatic method. Another solution is to use correlated terms for query expansion. The main problem with this approach is how to construct the term-term correlations that can be used effectively to improve retrieval performance. In this study, we try to construct query concepts that denote users' information needs from a document space, rather than to reformulate initial queries using the term correlations and/or users' relevance feedback. To make query concepts, we extract features from each document, and cluster the features into primitive concepts that are used to form query concepts. Experiments are performed on a TREC collection.

ACM Transactions on Information Systems, 2011

Information retrieval systems traditionally rely on textual keywords to index and retrieve documents. Keyword-based retrieval may return inaccurate and incomplete results when different keywords are used to describe the same concept in the documents and in the queries. Furthermore, the relationship between these related keywords may be semantic rather than syntactic, and capturing it thus requires access to comprehensive human world knowledge. Concept-based retrieval methods have attempted to tackle these difficulties by using manually built thesauri, by relying on term cooccurrence data, or by extracting latent word relationships and concepts from a corpus. In this article we introduce a new concept-based retrieval approach based on Explicit Semantic Analysis (ESA), a recently proposed method that augments keyword-based text representation with concept-based features, automatically extracted from massive human knowledge repositories such as Wikipedia. Our approach generates new tex...

Proceedings of the 16th International Conference on Applied Computing 2019, 2019

An Information Retrieval System (IRS) offers a number of tools and techniques, which enable to locate and visualize the relevant information needed. This information, is expressed by the user in the form of a query natural language. However, the representation of documents and the query in a traditional IRS lead to a lexical-centered relevance estimation which is, in fact, less efficient than a semantic-focused estimation. As a consequence, the documents that are actually relevant are not being recovered if they do not share words with the query, while the documents non relevant, which are words in common with the query, are recovered even though at times they do not have the meaning intended. This paper tackles this problem while suggesting a solution in the level of indexation of an IRS allowing it to improve its performance. To be more precise, we suggest a new approach of semantic indexation allowing to lead to the exact meaning of each term in a document or query undergoing a contextual analysis at the sentence level. In fact, if the system is able to comprehend the need of the user, then consequently it is perfectly capable to respond to it. Add to that, we suggest a simple method allowing to apply any model of IR on our new index table without changing its original bases making it faster. In order to validate this proposed approach, this new created system is evaluated base on numerous collections naming "TIME", "BBC", "The Guardian" and "BigThink". The results based on the experiments indicate the efficacy of our hypothesis compared to traditional IR approaches.

IBIMA 2005, 2005

Indexing a document is the method for describing its content for sake of easier subsequent retrieval in a document storage. This paper describes the implementation of the automatic indexing of various term weighting schemes in an IR (Information Retrieval) system using CISI documents collection which constitutes of abstracts for information retrieval papers and NPL collection which constitutes of abstracts for electronic engineering documents. The system starts with a simple form of text representation in which extracts keywords that represent documents as vectors of weights that represent the importance of keywords in documents of the documents collection and then evaluates, compares the retrieval effectiveness of various search models based on automatic text-word indexing and presents experimental results conduct to study the improvements made on the effectiveness of the text retrieval by successively applying these approaches.

2009

Term-weighting techniques are responsible for giving a weight for each query term to determine its significance. This term significance is based on the generality or specificity of that term. Traditional weighting models, such as the Inverse Document Frequency (IDF), measure the generality/specificity of terms through document collection statistics. This paper presents a technique that employs the WordNet ontology to determine the significance of query terms without depending on document collection statistics. The experiments are carried out on the WT2g document collection under the Terrier Information Retrieval Platform.

Proc. of the …, 2005

Automatic indexing is one of the important technologies used for Textual Data Analysis applications. Standard document indexing techniques usually identify the most relevant keywords in the documents. This paper presents an alternative approach that aims at performing document indexing by associating concepts with the document to index instead of extracting keywords out of it. The concepts are extracted out of the EDR Electronic Dictionary that provides a concept hierarchy based on hyponym/hypernym relations. An experimental evaluation based on a probabilistic model was performed on a sample of the INSPEC bibliographic database and we present the promising results that were obtained during the evaluation experiments.