IRJET- Identification of Text Similarity Based On Context

2021

1Student, Dept. of IT Engineering, Pillai College of Engineering, New Panvel, Maharashtra, India 2Student, Dept. of IT Engineering, Pillai College of Engineering, New Panvel, Maharashtra, India 3Student, Dept. of IT Engineering, Pillai College of Engineering, New Panvel, Maharashtra, India ---------------------------------------------------------------------------***---------------------------------------------------------------------------AbstractText similarity computing plays an important role in natural language processing. The similarity calculation of short text is influenced by the small feature of text words and the accuracy is low. so it is a common improvement method to calculate the similarity of short texts with word semantic similarity. The word similarity calculation method combines two word semantic similarity by some strategies. Instead of doing a word for word comparison, we also need to pay attention to context in order to capture more of the semantics. Calculating...

ACM Transactions on Knowledge Discovery From Data, 2008

We present a method for measuring the semantic similarity of texts using a corpus-based measure of semantic word similarity and a normalized and modified version of the Longest Common Subsequence (LCS) string matching algorithm. Existing methods for computing text similarity have focused mainly on either large documents or individual words. We focus on computing the similarity between two sentences or two short paragraphs. The proposed method can be exploited in a variety of applications involving textual knowledge representation and knowledge discovery. Evaluation results on two different data sets show that our method outperforms several competing methods.

International Journal of Innovations in Engineering and Technology, 2017

Meaningful sentences are the combination of meaning words, if a system wants to process natural language it should have essential knowledge regarding words and their meanings. The assessment of semantic similarity between the words of a short text is one of the challenging task knowledge based tasks and the tasks in NLP like text summarization, information retrieval, search, categorization of text and machine learning etc. which uses the sentence similarity measures for assessing the similarity between the short-text or sentences. In this paper the survey of two techniques is done which are helpful in generating the extractive text summaries WordNet and Vector space analysis. In vector space model words can be represented as numeric vectors based on different semantic similarity measures, the similarity between the word numeric vectors can be calculated with the semantic measures called WordNet. The information regardin the word and teir meaning in earlier days was provided with the help of traditional dictionaries, but these dictionaries were only helpful for human readers not for machine, WordNet provide combination of traditional lexicographic information and modern information. It is a online lexical database designed for use under the program control, it uses the measure like for calculating the semantic similarity between the concepts. Nouns, verbs, adjectives, adverbs are organized into set of synonyms and semantic relationship between the synonym sets called as Synset.

International Journal of Advanced Computer Science and Applications, 2016

Text similarity plays an important role in natural language processing tasks such as answering questions and summarizing text. At present, state-of-the-art text similarity algorithms rely on inefficient word pairings and/or knowledge derived from large corpora such as Wikipedia. This article evaluates previous word similarity measures on benchmark datasets and then uses a hybrid word similarity in a novel text similarity measure (TSM). The proposed TSM is based on information content and WordNet semantic relations. TSM includes exact word match, the length of both sentences in a pair, and the maximum similarity between one word and the compared text. Compared with other well-known measures, results of TSM are surpassing or comparable with the best algorithms in the literature.

IEEE Transactions on …, 2006

Sentence similarity measures play an increasingly important role in text-related research and applications in areas such as text mining, Web page retrieval, and dialogue systems. Existing methods for computing sentence similarity have been adopted from approaches used for long text documents. These methods process sentences in a very high-dimensional space and are consequently inefficient, require human input, and are not adaptable to some application domains. This paper focuses directly on computing the similarity between very short texts of sentence length. It presents an algorithm that takes account of semantic information and word order information implied in the sentences. The semantic similarity of two sentences is calculated using information from a structured lexical database and from corpus statistics. The use of a lexical database enables our method to model human common sense knowledge and the incorporation of corpus statistics allows our method to be adaptable to different domains. The proposed method can be used in a variety of applications that involve text knowledge representation and discovery. Experiments on two sets of selected sentence pairs demonstrate that the proposed method provides a similarity measure that shows a significant correlation to human intuition.