Information Retrieval on Text using Concept Similarity

Traditional information retrieval systems rely on keywords to index documents and queries. In such systems, documents are retrieved based on the number of shared keywords with the query. This lexical-focused retrieval leads to inaccurate and incomplete results when different keywords are used to describe the documents and queries. Semantic-focused retrieval approaches attempt to overcome this problem by relying on concepts rather than on keywords to indexing and retrieval. The goal is to retrieve documents that are semantically relevant to a given user query. This paper addresses this issue by proposing a solution at the indexing level. More precisely, we propose a novel approach for semantic indexing based on concepts identified from a linguistic resource. In particular, our approach relies on the joint use of WordNet and WordNetDomains lexical databases for concept identification. Furthermore, we propose a semantic-based concept weighting scheme that relies on a novel definition of concept centrality. The resulting system is evaluated on the TIME test collection. Experimental results show the effectiveness of our proposition over traditional IR approaches.

2005

Semantic Similarity relates to computing the similarity between concepts which are not lexicographically similar. We investigate approaches to computing semantic similarity by mapping terms (concepts) to an ontology and by examining their relationships in that ontology. Some of the most popular semantic similarity methods are implemented and evaluated using WordNet as the underlying reference ontology. Building upon the idea of semantic similarity, a novel information retrieval method is also proposed. This method is capable of detecting similarities between documents containing semantically similar but not necessarily lexicographically similar terms. The proposed method has been evaluated in retrieval of images and documents on the Web. The experimental results demonstrated very promising performance improvements over state-of-the-art information retrieval methods.

ACM Transactions on Information Systems, 2011

Information retrieval systems traditionally rely on textual keywords to index and retrieve documents. Keyword-based retrieval may return inaccurate and incomplete results when different keywords are used to describe the same concept in the documents and in the queries. Furthermore, the relationship between these related keywords may be semantic rather than syntactic, and capturing it thus requires access to comprehensive human world knowledge. Concept-based retrieval methods have attempted to tackle these difficulties by using manually built thesauri, by relying on term cooccurrence data, or by extracting latent word relationships and concepts from a corpus. In this article we introduce a new concept-based retrieval approach based on Explicit Semantic Analysis (ESA), a recently proposed method that augments keyword-based text representation with concept-based features, automatically extracted from massive human knowledge repositories such as Wikipedia. Our approach generates new tex...

— Traditional internet search engines are keyword-based. Such a mechanism is effective once the user is aware of precisely the right words within the websites they are trying to find. In many web applications semantic similarity between two terms can be computed. For computing semantic similarity initially there are two approaches namely knowledge based and corpus based approaches. Existing system is able to find similarity between words but, this approach can compute similarity between multi-word expressions. A semantic network is an efficient and effective approach for measuring semantic similarity between two terms. A new clustering approach introduced to improve the accuracy of computing similarity of two terms. This approach is more effective because, it computes semantic similarity between multi-word expressions with ambiguity.

2005

It is well known that ontologies will become a key piece, as they allow making the semantics of Semantic Web content [16] explicit. In spite of the big advantages that the Semantic Web promises, there are still several problems to solve. Those concerning ontologies include their availability, development and evolution. In the area of information retrieval, the dimension of document vectors plays an important role. Firstly, with higher index dimensions the indexing structures suffer from the "curse of dimensionality" and their efficiency rapidly decreases. Secondly, we may not use exact words when looking for a document, thus we miss some relevant documents. LSI is a numerical method, which discovers latent semantics in documents by creating concepts from existing terms. In this paper we present a basic method of mapping LSI concepts on given ontology (WordNet), used both for retrieval recall improvement and dimension reduction. We offer experimental results for this method on a subset of

The diverse and huge volume of information available over networks makes it difficult for the user to find their accurate information need. Previous information retrieval system mostly uses keywords to retrieve and index the documents, which may return inaccurate result when different keywords are employed to express the same concept in the document and queries presented by the user. However, to overcome the problem of retrieving accurate information on the Web intelligently, current information retrieval approaches need to be improved. Concept-based information retrieval are playing major role in retrieving meaningful information intelligently. In this paper, we propose a novel concept-based approach using Wikipedia-based Explicit Semantic Analysis and Ontology. We propose approach that first specifies the given context of query words semantically by using Ontology, which contains Wikipedia-based concept to query words disambiguation, and then extracts corresponding concepts for each word based on the given context. As a result the retrieved documents will be more relevant to the user query and Concept-based information retrieval systems performance will improve.

Proceedings of the 10th International Conference on Web Information Systems and Technologies, 2014

Recently, web users feel difficult to find the desired information on the internet despite a lot of useful information since it takes more time and effort to find it. In order to solve this problem, the query expansion is considered as a new alternative. It is the process of reformulating a query to improve retrieval performance in information retrieval operations. Although there are a few techniques of query expansion, synonym identification is one of them. Therefore, this paper proposes the method to measure the semantic similarity between two words by using the keyword-based web documents. The formal concept analysis and our proposed expansion algorithm are used to estimate the similarity between two words. To evaluate the performance of our method, we conducted two experiments. As the results, the average of similarity between synonym pairs is much higher than random pairs. Also, our method shows the remarkable performance in comparison with other method. Therefore, the suggested method in this paper has the contribution to find the synonym among a lot of candidate words.

International Journal of Emerging Trends in Engineering Research, 2020

A predicate-based document query language is proposed to allow users to define the search criteria precisely and reliably, and their knowledge of the documents to be retrieved. A guided search tool is built as an intelligent user interface oriented to the natural language in order to help users formulate queries. Supported by a generator of intelligent questions, an inference engine, a query base. A problem is faced when using the modern IR systems. It's represented in the vocabulary problem. This problem is represented in the inconsistencies between the terms which are used to describe the terms and the documents that are used by the investigator for describing their need for knowledge. The researcher has an automated thesaurus. This device has been designed using the Vector Space Model (VSM). The researcher used the similarity calculation of Cosine in this method. He used the selected 242 abstract Arabic documents in this article. All these abstracts include the information and computer science process. This paper aimed at building and designing automated Arabic thesauri through the use of the term similarity which could be employed in any particular domain or field for improving the process of expansion and obtaining greater number of relevant documents for the user query. In terms of recall and precision rates, it was found that the similar thesaurus is more capable than the conventional information retrieval system to enhance the recall process and precision.

2004

Noun phrases in queries are identified and classified into four types: proper names, dictionary phrases, simple phrases and complex phrases. A document has a phrase if all content words in the phrase are within a window of a certain size. The window sizes for different types of phrases are different and are determined using a decision tree. Phrases are more important than individual terms. Consequently, documents in response to a query are ranked with matching phrases given a higher priority. We utilize WordNet to disambiguate word senses of query terms. Whenever the sense of a query term is determined, its synonyms, hyponyms, words from its definition and its compound words are considered for possible additions to the query. Experimental results show that our approach yields between 23% and 31% improvements over the best-known results on the TREC 9, 10 and 12 collections for short (title only) queries, without using Web data.

Lecture Notes in Computer Science, 2009

Semantic relation is an important concept of information science. Now a days it is widely used in semantic web. This paper aims to present a measure to automatically determine semantic relation between words using web as knowledge source. It explores whether two words are related or not even if they are dissimilar in meaning. The proposed formula is a function of frequency of occurrences of the words in each document in the corpus. This relationship measure will be useful to extract semantic information from the web. Experimental evaluation on ten manually selected word pairs using the WebKb data as information source demonstrates the effectiveness of the proposed semantic relation.