Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Background and Related Work
Task Analysis
Use Cases
Discussion
References
All Topics
Computer Science
Natural Language Processing

Interactive Analysis of Word Vector Embeddings

Profile image of Michael GleicherMichael Gleicher

2018, Computer Graphics Forum

https://doi.org/10.1111/CGF.13417
visibility

…

description

13 pages

link

1 file

Abstract

Word vector embeddings are an emerging tool for natural language processing. They have proven beneficial for a wide variety of language processing tasks. Their utility stems from the ability to encode word relationships within the vector space. Applications range from components in natural language processing systems to tools for linguistic analysis in the study of language and literature. In many of these applications, interpreting embeddings and understanding the encoded grammatical and semantic relations between words is useful, but challenging. Visualization can aid in such interpretation of embeddings. In this paper, we examine the role for visualization in working with word vector embeddings. We provide a literature survey to catalogue the range of tasks where the embeddings are employed across a broad range of applications. Based on this survey, we identify key tasks and their characteristics. Then, we present visual interactive designs that address many of these tasks. The designs integrate into an exploration and analysis environment for embeddings. Finally, we provide example use cases for them and discuss domain user feedback.

Related papers

MSc Thesis: Word Vector-Space Embeddings of Natural Language Data over Time
Chiraag Lala

Words are often mapped to vectors in a vector-space (Euclidean-space). Such map- pings, also called embeddings, are used in many Natural Language Processing (NLP) tasks. These word embeddings are, generally, intended to reflect the usage, semantic similarities and relatedness of the words they represent. Simply put, word embeddings reflect the meaning of the words relative to other words. However, word meanings are known to change over time (semantic change). Current publicly available word vector-space embeddings are ‘static’ in nature with no temporal component. Creat- ing ‘dynamic’ word embeddings by adding temporal information opens the possibility of capturing the phenomenon of semantic change. These embeddings (with temporal component) can be used to produce visual animation of semantic change and change in word relations over time. It also has the potential to improve performance of various NLP tasks, particularly those involving time like the task of Diachronic Text Evalua- tion. This project achieves the following: (1) Create word embeddings with time component (dynamic embeddings) that captures the meaning/usage/similarities of words across various times ranging between the years 1800 and 2008. (2) Develop a tool/software that animates changes in word relations using the dynamic embeddings. (3) Evaluate the dynamic embeddings created using word similarity measures and Diachronic Text Evaluation task.

View PDFchevron_right
Visualizing Natural Language Resources
Kristina Kocijan

2015

As we move through the era of Big Data, data visualization is increasingly taking a leading role in the data presentation. Because of the disparity in the amount of data and time we have to process it, it has become extremely important to find the right way, i.e. the right picture that will convey a story our data is holding. Although not falling within Big Data type of data, a dictionary of nouns with a description of case paradigms still represents a large amount of data that needs to be understood. In this paper, the distribution of Croatian nouns and paradigms used for all singular cases existing in the NooJ linguistic environment, as well as the relations among the case endings and existing paradigms will be visually presented. Tableau software is used for the first task and Cytoscape for the second. The structure of presented data should help both those learning the language and those learning about the language.

View PDFchevron_right
Visualising word similarity clusters
Kris Heylen
View PDFchevron_right
How to Use Word Embeddings for Natural Language Processing
Barbara McGillivray

This how-to guide introduces computational methods for the analysis of word meaning, covering the main concepts of distributional semantics and word embedding models. At their core, distributional approaches to lexical semantics (i.e., the study of word meanings) are based on the idea that by looking at the context of use of a word (in terms of other words it occurs with in a text, for instance) we can infer important features of its meaning. This simple idea has turned out to be extremely powerful because it can be directly implemented computationally from text data. Word embeddings have been used extensively in social science research and they have the potential to highlight expected and less expected similarities between words and how these change across factors such as time. This guide will summarise the strengths of word embeddings for applied text-based research, but also their limitations and the features that needs to be considered when using or training them in your own research. By the end of this guide, readers should be able to . . . • Define the basic principles behind distributional methods in the analysis of word meaning. • Articulate the main differences between different computational techniques to represent word meaning from text • Evaluate the limitations of word embeddings methods in different research settings.

View PDFchevron_right
NLP Scholar: An Interactive Visual Explorer for Natural Language Processing Literature
Saif M Mohammad

arXiv (Cornell University), 2020

As part of the NLP Scholar project, we created a single unified dataset of NLP papers and their meta-information (including citation numbers), by extracting and aligning information from the ACL Anthology and Google Scholar. In this paper, we describe several interconnected interactive visualizations (dashboards) that present various aspects of the data. Clicking on an item within a visualization or entering query terms in the search boxes filters the data in all visualizations in the dashboard. This allows users to search for papers in the area of their interest, published within specific time periods, published by specified authors, etc. The interactive visualizations presented here, and the associated dataset of papers mapped to citations, have additional uses as well including understanding how the field is growing (both overall and across sub-areas), as well as quantifying the impact of different types of papers on subsequent publications.

View PDFchevron_right
The Visualization of Change in Word Meaning over Time using Temporal Word Embeddings
Chiraag Lala

We describe a visualization tool that can be used to view the change in meaning of words over time. The tool makes use of existing (static) word embedding datasets together with a timestamped n-gram cor- pus to create temporal word embeddings.

View PDFchevron_right
Visual execution and data visualisation in natural language processing
Peter Rodgers

1997

We describe GGI, a visual system that allows the user to execute an automatically generated data flow graph containing code modules that perform natural language processing tasks. These code modules operate on text documents. GGI has a suite of text visualisation tools that allows the user useful views of the annotation data that is produced by the modules in the executable graph. GGI forms part of the GATE natural language engineering system.

View PDFchevron_right
Looking at word meaning. An interactive visualization of Semantic Vector Spaces for Dutch synsets
Heylen Kris

Proceedings of the EACL 2012 …, 2012

In statistical NLP, Semantic Vector Spaces (SVS) are the standard technique for the automatic modeling of lexical semantics. However, it is largely unclear how these black-box techniques exactly capture word meaning. To explore the way an SVS structures the individual occurrences of words, we use a non-parametric MDS solution of a token-by-token similarity matrix. The MDS solution is visualized in an interactive plot with the Google Chart Tools. As a case study, we look at the occurrences of 476 Dutch nouns grouped in 214 synsets.

View PDFchevron_right
ETNLP: A Toolkit for Extraction, Evaluation and Visualization of Pre-trained Word Embeddings
Thanh Vu

ArXiv, 2019

In this paper, we introduce a comprehensive toolkit, ETNLP, which can evaluate, extract, and visualize multiple sets of pre-trained word embeddings. First, for evaluation, ETNLP analyses the quality of pre-trained embeddings based on an input word analogy list. Second, for extraction ETNLP provides a subset of the embeddings to be used in the downstream NLP tasks. Finally, ETNLP has a visualization module which is for exploring the embedded words interactively. We demonstrate the effectiveness of ETNLP on our pre-trained word embeddings in Vietnamese. Specifically, we create a large Vietnamese word analogy list to evaluate the embeddings. We then utilize the pre-trained embeddings for the name entity recognition (NER) task in Vietnamese and achieve the new state-of-the-art results on a benchmark dataset for the NER task. A video demonstration of ETNLP is available at this https URL. The source code and data are available at https: //github.com/vietnlp/etnlp.

View PDFchevron_right
Interpreting Word Embeddings with Eigenvector Analysis
Jamin Shin

2018

Dense word vectors have proven their values in many downstream NLP tasks over the past few years. However, the dimensions of such embeddings are not easily interpretable. Out of the d-dimensions in a word vector, we would not be able to understand what high or low values mean. Previous approaches addressing this issue have mainly focused on either training sparse/non-negative constrained word embeddings, or post-processing standard pre-trained word embeddings. On the other hand, we analyze conventional word embeddings trained with Singular Value Decomposition, and reveal similar interpretability. We use a novel eigenvector analysis method inspired from Random Matrix Theory and show that semantically coherent groups not only form in the row space, but also the column space. This allows us to view individual word vector dimensions as human-interpretable semantic features.

View PDFchevron_right

References (62)

  1. challenging implementation aspect was to ensure interac- tive response times for the lens ( §4.3). To retrieve and summarize https://graphics.cs.wisc.edu/Vis/EmbVis 1860 1870 1880 1890 1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 References [ABHR * ] ALPER B., BACH B., HENRY RICHE N., ISENBERG T., FEKETE J.-D.: Weighted graph comparison techniques for brain con- nectivity analysis. 8
  2. ALEXANDER E., GLEICHER M.: Task-driven comparison of topic models. IEEE Transactions on Visualization and Computer Graph- ics 22, 1 (January 2016), 320-329. 2, 6
  3. AKV * 14] ALEXANDER E., KOHLMANN J., VALENZA R., WITMORE M., GLEICHER M.: Serendip: Topic model-driven visual exploration of text corpora. In IEEE Conference on Visual Analytics Science and Technology (2014), IEEE, pp. 173-182. 2
  4. ALL * 16] ARORA S., LI Y., LIANG Y., MA T., RISTESKI A.: A latent variable model approach to pmi-based word embeddings. Transactions of the Association for Computational Linguistics 4 (2016), 385-399. 2
  5. ARORA S., LIANG Y., MA T.: A simple but tough-to-beat baseline for sentence embeddings. In Proc. of ICLR (2017). 2
  6. BCZ * 16] BOLUKBASI T., CHANG K.-W., ZOU J. Y., SALIGRAMA V., KALAI A. T.: Man is to computer programmer as woman is to home- maker? debiasing word embeddings. In Neural Information Processing (2016), pp. 4349-4357. 3, 4
  7. BARONI M., DINU G., KRUSZEWSKI G.: Don't count, pre- dict! a systematic comparison of context-counting vs. context-predicting semantic vectors. In Proc. of ACL (2014), pp. 238-247. 3, 4 [BGH * 17] BOWMAN S. R., GOLDBERG Y., HILL F., LAZARIDOU A., LEVY O., REICHART R., SØGAARD A. (Eds.):. Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP, RepE- val@EMNLP 2017, Copenhagen, Denmark, September 8, 2017 (2017), Association for Computational Linguistics. 1
  8. BERGER M., MCDONOUGH K., SEVERSKY L. M.: cite2vec: Citation-driven document exploration via word embeddings. IEEE Transactions on Visualization and Computer Graphics 23, 1 (Jan 2017), 691-700. 3
  9. BERTINI E., RIGAMONTI M., LALANNE D.: Extended excentric labeling. In EuroVIS'09. 3, 8
  10. BSP * 93] BIER E. A., STONE M. C., PIER K., BUXTON W., DEROSE T. D.: Toolglass and magic lenses: The see-through interface. In Proc. of the 20th Annual Conference on Computer Graphics and Interac- tive Techniques (New York, NY, USA, 1993), SIGGRAPH '93, ACM, pp. 73-80. 8
  11. COLLINS C., CARPENDALE S., PENN G.: Docuburst: Visual- izing document content using language structure. Computer Graphics Forum 28, 3 (2009), 1039-1046. 3
  12. CHOO J., LEE C., REDDY C. K., PARK H.: Utopian: User- driven topic modeling based on interactive nonnegative matrix factoriza- tion. IEEE Transactions on Visualization and Computer Graphics 19, 12 (Dec 2013), 1992-2001. 2
  13. COLLINS C., VIEGAS F. B., WATTENBERG M.: Parallel tag clouds to explore and analyze faceted text corpora. In Visual Analytics Science and Technology, 2009. VAST 2009. IEEE Symposium on (2009), IEEE, pp. 91-98. 6
  14. FAST E., CHEN B., BERNSTEIN M. S.: Empath: Understand- ing topic signals in large-scale text. In Proc. of CHI (2016), pp. 4647- 4657. 1, 2
  15. FDJ * 14] FARUQUI M., DODGE J., JAUHAR S. K., DYER C., HOVY E., SMITH N. A.: Retrofitting word vectors to semantic lexicons. arXiv preprint arXiv:1411.4166 (2014). 3, 4
  16. FEKETE J.-D., PLAISANT C.: Excentric labeling: Dynamic neighborhood labeling for data visualization. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New York, NY, USA, 1999), CHI '99, ACM, pp. 512-519. 3, 8
  17. FULDA N., RICKS D., MURDOCH B., WINGATE D.: What can you do with a rock? affordance extraction via word embeddings. arXiv preprint arXiv:1703.03429 (2017). 3
  18. GLADKOVA A., DROZD A.: Intrinsic evaluations of word em- beddings: What can we do better? In Proc. of the 1st Workshop on Eval- uating Vector-Space Representations for NLP (2016), pp. 36-42. 5
  19. GLEICHER M.: Explainers: Expert explorations with crafted pro- jections. IEEE Transactions on Visualization and Computer Graphics 19, 12 (Dec 2013), 2042-2051. 2, 8
  20. GLEICHER M.: A framework for considering comprehensibility in modeling. Big data 4, 2 (2016), 75-88. 2
  21. HELLRICH J., HAHN U.: Bad company-neighborhoods in neural embedding spaces considered harmful. In COLING (2016), pp. 2785- 2796. 10
  22. HELLRICH J., HAHN U.: Don't get fooled by word embeddings: better watch their neighborhood. In Conference of the Alliance of Digital Humanities Organizations (Abstracts) (2017). 10
  23. HJH * 16] HEIMERL F., JOHN M., HAN Q., KOCH S., ERTL T.: Docu- compass: Effective exploration of document landscapes. In IEEE Con- ference on Visual Analytics Science and Technology (Oct 2016), pp. 11- 20. 3, 8
  24. HEIMERL F., KOCH S., BOSCH H., ERTL T.: Visual classi- fier training for text document retrieval. IEEE Transactions on Visual- ization and Computer Graphics 18, 12 (Dec 2012), 2839-2848. 2
  25. HAMILTON W. L., LESKOVEC J., JURAFSKY D.: Diachronic word embeddings reveal statistical laws of semantic change. In Proc. of ACL (2016), pp. 1489-1501. 1, 2, 3, 4, 9
  26. JATOWT A., DUH K.: A framework for analyzing semantic change of words across time. In Proc. of JCDL (2014), pp. 229-238. 8
  27. JEFFERS R. J., LEHISTE I.: Principles and Methods for Histori- cal Linguistics, 1 ed. The MIT Press, 1979. 9, 10
  28. KIM H., CHOO J., PARK H., ENDERT A.: Interaxis: Steering scatterplot axes via observation-level interaction. IEEE Transactions on Visualization and Computer Graphics 22, 1 (Jan 2016), 131-140. 2, 8
  29. KESSLER J. S.: Scattertext: a browser-based tool for visualizing how corpora differ. CoRR abs/1703.00565 (2017). 3
  30. KUSNER M., SUN Y., KOLKIN N., WEINBERGER K.: From word embeddings to document distances. In International Conference on Machine Learning (2015), pp. 957-966. 1
  31. LBT * 17] LIU S., BREMER P. T., THIAGARAJAN J. J., SRIKUMAR V., WANG B., LIVNAT Y., PASCUCCI V.: Visual exploration of semantic relationships in neural word embeddings. IEEE Transactions on Visual- ization and Computer Graphics PP, 99 (2017), 1-1. 1, 2, 5
  32. LEVY O., GOLDBERG Y.: Linguistic regularities in sparse and explicit word representations. In Proc. of CoNLL (2014), pp. 171-180. 1, 3 [LHK * 16] LEVY O., HILL F., KORHONEN A., CHO K., REICHART R., GOLDBERG Y., BORDES A. (Eds.):. Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, RepEval@ACL 2016, Berlin, Germany, August 2016 (2016), Association for Computational Linguistics. 1
  33. LJW * 15] LIU Q., JIANG H., WEI S., LING Z.-H., HU Y.: Learning semantic word embeddings based on ordinal knowledge constraints. In Association for Computational Linguistics (2015), pp. 1501-1511. 3, 4 [LSC * 17] LIU M., SHI J., CAO K., ZHU J., LIU S.: Analyzing the training processes of deep generative models. IEEE Transactions on Visualization and Computer Graphics (2017). 2
  34. LSL * 17] LIU M., SHI J., LI Z., LI C., ZHU J., LIU S.: Towards better analysis of deep convolutional neural networks. IEEE transactions on visualization and computer graphics 23, 1 (2017), 91-100. 2
  35. MIKOLOV T., CHEN K., CORRADO G., DEAN J.: Effi- cient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013). 1, 2, 4, 5
  36. MAATEN L. V. D., HINTON G.: Visualizing data using t-sne. Journal of Machine Learning Research 9, Nov (2008), 2579-2605. 2
  37. M/"UHLBACHER T., LINHARDT L., MÖLLER T., PIRINGER H.: Treepod: Sensitivity-aware selection of pareto-optimal decision trees. IEEE Transactions on Visualization and Computer Graphics PP, 99 (2017), 1-1. 2
  38. MSC * 13] MIKOLOV T., SUTSKEVER I., CHEN K., CORRADO G., DEAN J.: Distributed representations of words and phrases and their compositionality. In Proc. of NIPS (2013), pp. 3111-3119. 2, 9
  39. MUNZNER T.: Visualization analysis and design. CRC press, 2014. 3, 4
  40. NAYAK N., ANGELI G., MANNING C. D.: Evaluating word embeddings using a representative suite of practical tasks. In Proc. of the ACL Workshop on Evaluating Vector-Space Representations for NLP (2016), pp. 31-35.
  41. PKL * 17] PARK D., KIM S., LEE J., CHOO J., DIAKOPOULOS N., ELMQVIST N.: Conceptvector: Text visual analytics via interactive lexi- con building using word embedding. IEEE Transactions on Visualization and Computer Graphics PP, 99 (2017), 1-1. 3
  42. PENNINGTON J., SOCHER R., MANNING C. D.: Glove: Global vectors for word representation. In Proc. of EMNLP (2014), pp. 1532-1543. 1, 2, 3, 4
  43. RONG X., ADAR E.: Visual tools for debugging neural language models. In Proc. of ICML Workshop on Visualization for Deep Learning (2016). 2
  44. ROSSIELLO G., BASILE P., SEMERARO G.: Centroid-based text summarization through compositionality of word embeddings. In Proc. of the MultiLing 2017 Workshop on Summarization and Summary Evaluation Across Source Types and Genres (04 2017), pp. 12-21. 3, 4
  45. RUBENSTEIN H., GOODENOUGH J. B.: Contextual correlates of synonymy. Communications of the ACM 8, 10 (Oct. 1965), 627-633.
  46. SBM * 13] SOCHER R., BAUER J., MANNING C. D., ET AL.: Pars- ing with compositional vector grammars. In Proc. of the 51st Annual Meeting of the Association for Computational Linguistics (2013), vol. 1, pp. 455-465. 1
  47. STASKO J., GÖRG C., LIU Z.: Jigsaw: supporting investigative analysis through interactive visualization. Information visualisation 7, 2 (2008), 118-132. 2
  48. STROBELT H., GEHRMANN S., PFISTER H., RUSH A. M.: Lstmvis: A tool for visual analysis of hidden state dynamics in recur- rent neural networks. IEEE Transactions on Visualization and Computer Graphics (2017). 2
  49. SHNEIDERMAN B.: The eyes have it: A task by data type taxon- omy for information visualizations. In Proc. of the IEEE Symposium on Visual Languages (1996), IEEE, pp. 336-343. 5
  50. SCHNABEL T., LABUTOV I., MIMNO D., JOACHIMS T.: Evaluation methods for unsupervised word embeddings. In Proc. of EMNLP (2015), pp. 298-307. 3
  51. SEDLMAIR M., MEYER M., MUNZNER T.: Design study methodology: Reflections from the trenches and the stacks. IEEE trans- actions on visualization and computer graphics 18, 12 (2012), 2431- 2440. 3
  52. SUZUKI J., NAGATA M.: Right-truncatable neural word embed- dings. In HLT-NAACL (2016), pp. 1145-1151. 3
  53. SCHULZ H.-J., NOCKE T., HEITZLER M., SCHUMANN H.: A design space of visualization tasks. IEEE Transactions on Visualiza- tion and Computer Graphics 19, 12 (Dec. 2013), 2366-2375. 4, 5 [STN * 16] SMILKOV D., THORAT N., NICHOLSON C., REIF E., VIÉGAS F. B., WATTENBERG M.: Embedding projector: Interac- tive visualization and interpretation of embeddings. arXiv preprint arXiv:1611.05469 (2016). 2
  54. TALBOT J., LEE B., KAPOOR A., TAN D. S.: Ensemblema- trix: Interactive visualization to support machine learning with multiple classifiers. 2
  55. VAN DEN ELZEN S., VAN WIJK J. J.: Baobabview: Interac- tive construction and analysis of decision trees. In IEEE Conference on Visual Analytics Science and Technology (Oct 2011), pp. 151-160. 2
  56. VAN HAM F., WATTENBERG M., VIEGAS F. B.: Mapping text with phrase nets. IEEE Transactions on Visualization and Computer Graphics 15, 6 (Nov 2009), 1169-1176. 3
  57. WLL * 16] WANG X., LIU S., LIU J., CHEN J., ZHU J., GUO B.: Top- icpanorama: A full picture of relevant topics. IEEE Transactions on Vi- sualization and Computer Graphics 22, 12 (Dec 2016), 2508-2521. 2 [WSW * 17] WONGSUPHASAWAT K., SMILKOV D., WEXLER J., WIL- SON J., MANÃL' D., FRITZ D., KRISHNAN D., VIÃL'GAS F. B., WAT- TENBERG M.: Visualizing dataflow graphs of deep learning models in tensorflow. IEEE Transactions on Visualization and Computer Graphics PP, 99 (2017), 1-1. 2
  58. WATTENBERG M., VIÃL'GAS F. B.: The word tree, an interac- tive visual concordance. IEEE Transactions on Visualization and Com- puter Graphics 14, 6 (Nov 2008), 1221-1228. 3, 7
  59. YIN W., SCHÜTZE H.: Learning word meta-embeddings. In Proc. of ACL (2016), pp. 1351-1360. 3
  60. YWL * 16] YIN R., WANG Q., LI P., LI R., WANG B.: Multi- granularity chinese word embedding. In EMNLP (2016), pp. 981-986. 3, 4
  61. YU L.-C., WANG J., LAI K. R., ZHANG X.: Refining word embeddings for sentiment analysis. In EMNLP (2017), pp. 545-550. 1
  62. ZOU W. Y., SOCHER R., CER D., MANNING C. D.: Bilin- gual word embeddings for phrase-based machine translation. In EMNLP (2013), pp. 1393-1398. 2

Related papers

Interactive Visualizations of Word Embeddings for K-12 Students
Neel Pawar

Proceedings of the AAAI Conference on Artificial Intelligence

Word embeddings, which represent words as dense feature vectors, are widely used in natural language processing. In their seminal paper on word2vec, Mikolov and colleagues showed that a feature space created by training a word prediction network on a large text corpus will encode semantic information that supports analogy by vector arithmetic, e.g., "king" minus "man" plus "woman" equals "queen". To help novices appreciate this idea, people have sought effective graphical representations of word embeddings. We describe a new interactive tool for visually exploring word embeddings. Our tool allows users to define semantic dimensions by specifying opposed word pairs, e.g., gender is defined by pairs such as boy/girl and father/mother, and age by pairs such as father/son and mother/daughter. Words are plotted as points in a zoomable and rotatable 3D space, where the third ”residual” dimension encodes distance from the hyperplane defined by all th...

View PDFchevron_right
Vec2graph: a Python library for visualizing word embeddings as graphs
Andrey Kutuzov, Надежда Катричева

Analysis of Images, Social Networks and Texts (AIST), 2019

Visualization as a means of easy conveyance of ideas plays a key role in communicating linguistic theory through its applications. User-friendly NLP visualization tools allow researchers to get important insights for building, challenging, proving or rejecting their hypotheses. At the same time, visualizations provide general public with some understanding of what computational linguists investigate. In this paper, we present vec2graph: a ready-to-use Python 3 library visualizing vector representations (for example, word embeddings) as dynamic and interactive graphs. It is aimed at users with beginners' knowledge of software development, and can be used to easily produce visualizations suitable for the Web. We describe key ideas behind vec2graph, its hyperparameters, and its integration into existing word embedding frameworks.

View PDFchevron_right
Visualising WordNet Embeddings: some preliminary results
Csaba Veres

2019

AutoExtend is a method for learning unambiguous vector embeddings for word senses. We visualise these word embeddings with t-SNE, which further compresses the vectors to the x,y plane. We show that the t-SNE co-ordinates can be used to reveal interesting semantic relations between word senses, and propose a new method that uses the simple x,y coordinates to compute semantic similarity. This can be used to propose new links and alterations to existing ones in WordNet. We plan to add this approach to the existing toolbox of methods in an attempt to understand learned semantic relations in word embeddings.

View PDFchevron_right
Embedding Projector: Interactive Visualization and Interpretation of Embeddings
Nikhil Thorat

2016

Embeddings are ubiquitous in machine learning, appearing in recommender systems, NLP, and many other applications. Researchers and developers often need to explore the properties of a specific embedding, and one way to analyze embeddings is to visualize them. We present the Embedding Projector, a tool for interactive visualization and interpretation of embeddings.

View PDFchevron_right
Semantic Visualization for Short Texts with Word Embeddings
Hady W. Lauw

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, 2017

Semantic visualization integrates topic modeling and visualization, such that every document is associated with a topic distribution as well as visualization coordinates on a low-dimensional Euclidean space. We address the problem of semantic visualization for short texts. Such documents are increasingly common, including tweets, search snippets, news headlines, or status updates. Due to their short lengths, it is difficult to model semantics as the word co-occurrences in such a corpus are very sparse. Our approach is to incorporate auxiliary information, such as word embeddings from a larger corpus, to supplement the lack of co-occurrences. This requires the development of a novel semantic visualization model that seamlessly integrates visualization coordinates, topic distributions, and word vectors. We propose a model called GaussianSV, which outperforms pipelined baselines that derive topic models and visualization coordinates as disjoint steps, as well as semantic visualization baselines that do not consider word embeddings.

View PDFchevron_right

Related topics

  • Computer Science
  • Natural Language Processing
  • Visualization

    • Cited by

    VNLP: Visible natural language processing
    Tom Cheesman

    Information Visualization, 2021

    In general, Natural Language Processing (NLP) algorithms exhibit black-box behavior. Users input text and output are provided with no explanation of how the results are obtained. In order to increase understanding and trust, users value transparent processing which may explain derived results and enable understanding of the underlying routines. Many approaches take an opaque approach by default when designing NLP tools and do not incorporate a means to steer and manipulate the intermediate NLP steps. We present an interactive, customizable, visual framework that enables users to observe and participate in the NLP pipeline processes, explicitly manipulate the parameters of each step, and explore the result visually based on user preferences. The visible NLP (VNLP) pipeline design is then applied to a text similarity application to demonstrate the utility and advantages of a visible and transparent NLP pipeline in supporting users to understand and justify both the process and results...

    View PDFchevron_right
    A User‐based Visual Analytics Workflow for Exploratory Model Analysis
    Michael Gleicher

    Computer Graphics Forum, 2019

    Figure : The proposed EMA visual analytics workflow for discovery and generation of machine learning models. In step 1, the system uses interactive visualizations (such as histograms or graphs) to provide an initial data overview. The system then generates a number of possible modeling problems based on analyzing the data set (step 2) from which the user analyzes and selects one to try (step 3). Next, (step 4) an automated ML system trains and generates candidate models based on the data set and given problem. In step 5, the system shows comparisons of the generated prediction models through interactive visualizations of their predictions on a holdout set. Lastly, in step 6, users can select a number of preferable models, which are then exported by the system during step 7 for predictions on unseen test data. At any time, users can return to step 3 and try different modeling problems on the same dataset.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved