Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Work
Data
Results
Error Analysis
Conclusion and Future Work
References

Semantic Clustering using Bag-of-Bag-of-Features

Profile image of Pascale SebillotPascale Sebillot
https://doi.org/10.24348/CORIA.2012.229
visibility

…

description

16 pages

link

1 file

Abstract

Le calcul de distances entre représentations textuelles est au coeur de nombreuses applications du Traitement Automatique des Langues. Les approches standard initiallement développées pour la recherche d'information sont alors le plus souvent utilisées. Dans la plupart des cas, il est donc adopté une description sac-de-mots (ou sac-d'attributs) avec des pondérations de type TF-IDF ou des variantes, une représentation vectorielle et des fonctions classiques de similarité comme le cosinus. Dans ce papier, nous nous intéressons à l'une de ces tâches, à savoir le clustering sémantique d'entités extraites d'un corpus. Nous défendons l'idée que pour ce type de tâches, il est possible d'utiliser des représentations et des mesures de similarités plus adaptées que celles usuellement employées. Plus précisément, nous explorons l'utilisation de représentations alternatives des entités appelées sacs-de-vecteurs ou sacs-de-sacs-de-mots. Dans ce modèle, chaque entité est définie non pas par un unique vecteur, mais par un ensemble de vecteurs, chacun de ces vecteurs étant construit à partir d'une occurrence de l'entité. Pour utiliser cette représentation, nous utilisons et définissons des extensions des mesures classiques du modèle vectoriel (cosinus, Jaccard, produit scalaire...). Ces différents constituants sont testés sur notre tâche de clustering, et nous montrons que cette représentation en sac-de-vecteurs améliore significativement les résultats par rapport à une approche standard en sac- de-mots. 1 ABSTRACT. Computing distances between textual representation is at the heart of many Natural Language Processing tasks. The standard approaches initially developed for Information Retrieval are then used; most often they rely on a bag-of-words (or bag-of-feature) description with a TF-IDF (or variants) weighting, a vectorial representation and classical similarity functions like cosine. In this paper, we are interested in such a task, namely the semantic clustering of entities extracted from a text. We argue that for this kind of tasks, more suited representations 1. Ces travaux ont été (en partie) réalisés dans le cadre du programme QUAERO, financé par OSEO, agence française pour l'innovation. CORIA 2012, pp. 229-244, Bordeaux, 21-23 mars 2012 2. This work was achieved as part of the Quaero Programme, funded by OSEO, French State agency for innovation.

Related papers

Semantically enhanced document clustering
Ivan Stankov

2013

View PDFchevron_right
Performance of ontology-based semantic similarities in clustering
Montserrat Batet

Artificial Intelligence and Soft Computing, 2010

Traditionally, clustering was applied on numerical and categorical information. However, textual information is acquiring an increasing importance with the appearance of methods for textual data mining. This paper proposes the use of classical clustering algorithms with a mixed function that combines numerical, categorical and semantic features. The content of the semantic features is extracted from textual data. As the semantic features must be compared using a semantic similarity function and several measures have been developed, this paper analyses and compares the behavior of some of them using WordNet as background ontology. The different partitions obtained are compared to human classifications in order to see which one approximates better the human reasoning. Moreover, the interpretability of clusters obtained is discussed. The results show that those similarity measures that provide better results when compared using a standard benchmark also provide better and more interpretable partitions.

View PDFchevron_right
Rich Document Representation for Document Clustering
Farhad Oroumchian

In traditional document clustering models, a document is considered as a bag of words. In this paper we present a new method for generating feature vectors, using the sentence fragments that are called logical terms and statements, in PLIR system. PLIR is a Knowledge-Based Information system based on the theory of the Plausible Reasoning. We have conducted a number of experiments using OHSUMED document collection and the clustering methods K-Means with seven different similarity measures between documents. The Experiments seem to indicate that the use of richer features such as logical terms or statements for clustering tends to perform better than the simp le bag of words approaches within our domain of experiments that is second phase of a twostage retrieval system.

View PDFchevron_right
Semantic clustering for a functional text classification task
Thomas Lippincott

2009

Abstract. We describe a semantic clustering method designed to address shortcomings in the common bag-of-words document representation for functional semantic classification tasks. The method uses WordNet-based distance metrics to construct a similarity matrix, and expectation maximization to find and represent clusters of semanticallyrelated terms. Using these clusters as features for machine learning helps maintain performance across distinct, domain-specific vocabularies while reducing the size of the document representation.

View PDFchevron_right
Towards Semantically Sensitive Text Clustering: A Feature Space Modeling Technology Based on Dimension Extension
Publishing India Group

Journal of Applied Information Science, 2020

Content bunching is a large use of knowledge mining. It's concerned about gathering related content archives together. Proper now paper, a number of models are worked to bunch capstone venture archives using three grouping systems: okay-implies, ok-implies rapid, and k-medoids. Our dataset is acquired from the library of the University Pc and Information Sciences, King Saud tuition, Riyadh. Three closeness measure are tried: Cosine likeness, Jacquard similitude, and Correlation Coefficient. The nature of the got models is assessed and checked out. The results display that the great execution is comprehensive utilizing k-implies and okay-medoids joined with cosine similitude. We watch style in the nature of bunching based on the assessment measure utilized. Additionally, as the estimation of okay builds, the character of the next crew improves. At long last, we find the classifications of commencement ventures provided in the information technological know-how division for female understudies.

View PDFchevron_right
Semantic smoothing for text clustering
George Tsatsaronis

In this paper we present a new semantic smoothing vector space kernel (S-VSM) for text documents clustering. In the suggested approach semantic relatedness between words is used to smooth the similarity and the representation of text documents. The basic hypothesis examined is that considering semantic relatedness between two text documents may improve the performance of the text document clustering task. For our experimental evaluation we analyze the performance of several semantic relatedness measures when embedded in the proposed (S-VSM) and present results with respect to different experimental conditions, such as: (i) the datasets used, (ii) the underlying knowledge sources of the utilized measures, and (iii) the clustering algorithms employed. To the best of our knowledge, the current study is the first to systematically compare, analyze and evaluate the impact of semantic smoothing in text clustering based on 'wisdom of linguists', e.g., WordNets, 'wisdom of crowds', e.g., Wikipedia, and 'wisdom of corpora', e.g., large text corpora represented with the traditional Bag of Words (BoW) model. Three semantic relatedness measures for text are considered; two knowledge-based (Omiotis [1] that uses WordNet, and WLM [2] that uses Wikipedia), and one corpus-based (PMI [3] trained on a semantically tagged SemCor version). For the comparison of different experimental conditions we use the BCubed F-Measure evaluation metric which satisfies all formal constraints of good quality cluster. The experimental results show that the clustering performance based on the S-VSM is better compared to the traditional VSM model and compares favorably against the standard GVSM kernel which uses word co-occurrences to compute the latent similarities between document terms. 1 Though there are slight conceptual differences between the terms 'semantic similarity' and 'semantic relatedness', for the purposes of this work this differentiation is not important. Therefore, the two terms might be used interchangeably for the remaining of the paper.

View PDFchevron_right
Semantic Based Vector Space Model to Improve the Clustering Accuracy in Knowledge Repositories
Journal of Computer Science IJCSIS

When a huge amount of the document is stored in the database it is difficult to find any specific authors document. So the clustering process is one of the methods to identify the document from the huge database. To search the document from any huge data is a tough work, and to solve out this problem, we are using the clustering process. This clustering process is providing a good output over the specific users query. The user can only retrieve the document with the topic name or with the specific text; they can't search any document with the name of particular author .so the user may face the trouble in searching the related author document. This is a very annoying situation when searching any particular author document in the database without any labeled category. We have proposed a novel technique called NLP to overcome this issue and it provides an option to search any document with the name of particular author. For this approach, we are considering over the clustering technique based on the semantic vector space model. In pre-processing step, the tokenization process is taken a place, where POS tagging is classifying the vowel and prevent the use of tagging like Noun, pronoun, Verb, Adjective etc. The process of stemming is preventing the repetition or matching word in database. A technique is being proposed for performing the following operations: Replacement of the word with their real meaning, tagging of the parsed document, applying semantic grammar, and calculation of semantic weights for the document. The similarity measure is being obtained between the documents and then the documents are clustered using Semantic clustering algorithm. Several techniques have also been evolved in this work like NLP; Semantic based clustering and user query retrieval. The method adopted in this work is evaluated on different data sets with standard performance measures and the effectiveness of these methods in developing a meaningful cluster has been proved.

View PDFchevron_right
Semantic Clustering for Complex Data Items
Nico Cinefra

From a general point of view, semantic clustering is a software technique (a class of algorithms and methods) that aims to group source artifacts (e.g., sentences, user profiles, items) based on their semantic value. This concerns with partitioning points of a data set into distinct groups (clusters) such that two points from one cluster are semantically similar to each other whereas two points from distinct clusters are not. In order to do this, a similarity measure is required to be passed as input to the clustering function. In this report I am trying to indagate about key features of complex data semantic classification models and their main operational aspects. The goal of these techniques, different from the classical data mining approaches, is to discover a path to a more wide and general knowledge taking advantage of the meanings related to the components and associating them with each semantic unit. Then, in the last section, I will investigate about possible applications of the techniques described above, without avoiding personal considerations.

View PDFchevron_right
Matching Similarity for Keyword-Based Clustering
Mohammad Rezaei

Lecture Notes in Computer Science, 2014

View PDFchevron_right
Using semantics in document representation: A lexical chain approach
Dinakar Jayarajan

2009

Automatic classification and clustering are two of the most common operations performed on text documents. Numerous algorithms have been proposed for this and invariably, all of these algorithms use some variation of the vector space model to represent the documents. Traditionally, the Bag of Words (BoW) representation is used to model the documents in a vector space. The BoW scheme is a simple and popular scheme, but it suffers from numerous drawbacks. The chief among them is that the feature vectors ...

View PDFchevron_right

References (26)

  1. Bibliographie
  2. Bagga A., Baldwin B., « Entity-based cross-document coreferencing using the Vector Space Model », Proceedings of the 36th Annual Meeting of the Association for Computational Lin- guistics and 17th International Conference on Computational Linguistics -Volume 1, ACL '98, Association for Computational Linguistics, Stroudsburg, PA, USA, p. 79-85, 1998.
  3. Bunke H., « Recent developments in graph matching », Proceedings of International Confer- ence on Pattern Matching, p. 2117-2124, 2000.
  4. Chieu H. L., Ng H. T., « Named entity recognition: a maximum entropy approach using global information », Proceedings of the 19th international conference on Computational linguis- tics -Volume 1, COLING '02, Association for Computational Linguistics, Stroudsburg, PA, USA, p. 1-7, 2002.
  5. Collins M., Singer Y., « Unsupervised models for named entity classification », Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, 1999.
  6. Ekbal A., Sourjikova E., Frank A., Ponzetto S. P., « Assessing the challenge of fine-grained named entity recognition and classification », Proceedings of the 2010 Named Entities Workshop, NEWS '10, Association for Computational Linguistics, Stroudsburg, PA, USA, p. 93-101, 2010.
  7. Fleischman M., Hovy E., « Fine grained classification of named entities », Proceedings of the 19th International Conference on Computational Linguistics, p. 1-7, 2002.
  8. Gosselin P., Cord M., Philipp-Foliguet S., « Kernels on Bags of Fuzzy Regions for Fast Object retrieval », image processing, 2007. ICIP 2007. IEEE International Conference on, vol. 1, p. 177-180, 16 2007-oct. 19, 2007.
  9. Hubert L., Arabie P., « Comparing partitions », Journal of Classiffication, 1985.
  10. Isozaki H., Kazawa H., « Efficient support vector classifiers for named entity recognition », Proceedings of the 19th international conference on Computational linguistics -Volume 1, COLING '02, Association for Computational Linguistics, Stroudsburg, PA, USA, p. 1-7, 2002.
  11. Kazama J., Torisawa K., « Exploiting Wikipedia as External Knowledge for Named Entity Recognition », Proceedings of the 2007 Joint Conference on Empirical Methods in Natu- ral Language Processing and Computational Natural Language Learning, Association for Computational Linguistics, Prague, p. 698-707, June, 2007.
  12. Kondor R., Jebara T., « A kernel between sets of vectors », Proceedings of the International Conference on Machine Learning (ICML), Washington, États-Unis, 2003.
  13. Kozareva Z., « Bootstrapping named entity recognition with automatically generated gazetteer lists », Proceedings of the Eleventh Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, Trento, Italy, p. 15-21, April, 2006.
  14. Liao W., Veeramachaneni S., « A Simple Semi-supervised Algorithm For Named Entity Recog- nition », Proceedings of the NAACL HLT Workshop on Semi-supervised Learning for Nat- ural Language Processing, Association for Computational Linguistics, Boulder, Colorad, p. 58-65, 2009.
  15. Manning C. D., Schütze H., Foundations of Statistical Natural Language Processing, MIT Press. Cambridge, may, 1999.
  16. Manning C., Raghavan P., Schütze H., Introduction to information retrieval, Cambridge Uni- versity Press, 2008.
  17. McCallum A., Li W., « Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons », Proceedings of the seventh confer- ence on Natural language learning at HLT-NAACL 2003 -Volume 4, CONLL '03, Associ- ation for Computational Linguistics, Stroudsburg, PA, USA, p. 188-191, 2003.
  18. Nadeau D., Satoshi S., « A survey of named entity recognition and classification », Lingvisticae Investigationes, vol. 30, p. 3-26, 2007.
  19. Rand W. M., « Objective Criteria for the Evaluation of Clustering Methods », , vol. 66, n°336, p. pp. 846-850, 1971.
  20. Sang T. K., Erik F., De Meulder F., « Introduction to the CoNLL-2003 shared task: language- independent named entity recognition », Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003 -Volume 4, CONLL '03, Association for Compu- tational Linguistics, Stroudsburg, PA, USA, p. 142-147, 2003.
  21. Schmid H., « Probabilistic part-of-speech tagging using decision trees », international Confer- ence on New Methods in Language Processing, p. 44-49, 1995.
  22. Sobhana N., Pabitra M. G. S., « Conditional Random Field Based Named Entity Recognition in Geological Text », International Journal of Computer Applications, 2010.
  23. Takeuchi K., Collier N., « Use of support vector machines in extended named entity recog- nition », Proceedings of the 6th conference on Natural language learning -Volume 20, COLING-02, Association for Computational Linguistics, Stroudsburg, PA, USA, p. 1-7, 2002.
  24. van Dongen S., Graph Clustering by Flow Simulation, PhD thesis, University of Utrecht, 2000.
  25. Vinh N., Epps J., Bailey J., « Information Theoretic Measures for Clusterings Comparison », Journal of Machine Learning Research, 2010.
  26. Zhou G., Su J., « Named entity recognition using an HMM-based chunk tagger », Proceed- ings of the 40th Annual Meeting on Association for Computational Linguistics, ACL '02, Association for Computational Linguistics, Stroudsburg, PA, USA, p. 473-480, 2002.

Related papers

Proper Noun Semantic Clustering using Bag-of-Vectors
Pascale Sebillot

In this paper, we propose a model for semantic cluster-ing of entities extracted from a text, and we apply it to a Proper Noun classification task. This model is based on a new method to compute the similarity between the entities. Indeed, the classical way of calculating similarity is to build a feature vector or Bag-of-Features for each entity and then use classical similarity functions like Cosine. In practice, the features are contextual, such as words around the different occurrences of each entity. Here, we propose to use an alternative representation for en-tities, called Bag-of-Vectors, or Bag-of-Bags-of-Features. In this new model, each entity is not defined as a unique vec-tor but as a set of vectors, in which each vector is built based on the contextual features of one occurrence of the entity. In order to use Bag-of-Vectors for clustering, we introduce new versions of classical similarity functions such as Cosine and Scalar Products. Experimentally, we show that the Bag-...

View PDFchevron_right
Exploiting Document Level Semantics in Document Clustering
Muhammad Rafi

International Journal of Advanced Computer Science and Applications, 2016

Document clustering is an unsupervised machine learning method that separates a large subject heterogeneous collection (Corpus) into smaller, more manageable, subject homogeneous collections (clusters). Traditional method of document clustering works around extracting textual features like: terms, sequences, and phrases from documents. These features are independent of each other and do not cater meaning behind these word in the clustering process. In order to perform semantic viable clustering, we believe that the problem of document clustering has two main components: (1) to represent the document in such a form that it inherently captures semantics of the text. This may also help to reduce dimensionality of the document and (2) to define a similarity measure based on the lexical, syntactic and semantic features such that it assigns higher numerical values to document pairs which have higher syntactic and semantic relationship. In this paper, we propose a representation of document by extracting three different types of features from a given document. These are lexical α, syntactic β and semantic γ features. A meta-descriptor for each document is proposed using these three features: first lexical, then syntactic and in the last semantic. A document to document similarity matrix is produced where each entry of this matrix contains a three value vector for each lexical α, syntactic β and semantic γ. The main contributions from this research are (i) A document level descriptor using three different features for text like: lexical, syntactic and semantics. (ii) we propose a similarity function using these three, and (iii) we define a new candidate clustering algorithm using three component of similarity measure to guide the clustering process in a direction that produce more semantic rich clusters. We performed an extensive series of experiments on standard text mining data sets with external clustering evaluations like: F-Measure and Purity, and have obtained encouraging results.

View PDFchevron_right
Statistical semantics for enhancing document clustering
Ahmed Farahat

Knowledge and Information Systems, 2011

Document clustering algorithms usually use vector space model (VSM) as their underlying model for document representation. VSM assumes that terms are independent and accordingly ignores any semantic relations between them. This results in mapping documents to a space where the proximity between document vectors does not reflect their true semantic similarity. This paper proposes new models for document representation that capture semantic similarity between documents based on measures of correlations between their terms. The paper uses the proposed models to enhance the effectiveness of different algorithms for document clustering. The proposed representation models define a corpus-specific semantic similarity by estimating measures of term-term correlations from the documents to be clustered. The corpus of documents accordingly defines a context in which semantic similarity is calculated. Experiments have been conducted on thirteen benchmark data sets to empirically evaluate the effectiveness of the proposed models and compare them to VSM and other well-known models for capturing semantic similarity.

View PDFchevron_right
Document Clustering based on Semantic Notions
Muhammad Rafi

2017

ing for each document. There are several possible extensions to this work:  The proposed document clustering approach has many practical applications. One direction is to apply this technique on some specific application area along with application specific optimizations to see the outcome. For example: web search results can be clustered using this approach. The snippets for each cluster are generated to see the quality of these snippets.  In the proposed approach each term, whether it is from lexical chain or from topic maps, has an equal effect on similarity calculation for a pair of documents. One possible direction is to introduce discriminative feature weighting for the features in this approach. Discriminative feature weighting has encouraging results for both text clustering and classification tasks.

View PDFchevron_right
Concept-based vector space model for improving text clustering
ali shokouhi
View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved