Academia.eduAcademia.edu

Outline

Title
Abstract
Data
Datasets
Method
Summary
References
All Topics
Computer Science

Deep Learning for Semantic Relations

Profile image of Stan SzpakowiczStan Szpakowicz

2020

Abstract

The second edition of "Semantic Relations Between Nominals" (by Vivi Nastase, Stan Szpakowicz, Preslav Nakov and Diarmuid \'O S\'eaghdha) will be published by Morgan & Claypool. A new Chapter 5 of the book discusses relation classification/extraction in the deep-learning paradigm which arose after the first edition appeared. This is a preview of Chapter 5, made public by the kind permission of Morgan & Claypool.

Related papers

Neural relation extraction: a review
Furkan Özbay

Turkish J. Electr. Eng. Comput. Sci., 2021

Neural relation extraction discovers semantic relations between entities from unstructured text using deep learning methods. In this study, we make a clear categorization of the existing relation extraction methods in terms of data expressiveness and data supervision, and present a comprehensive and comparative review. We describe the evaluation methodologies and the datasets used for model assessment. We explicitly state the common challenges in relation extraction task and point out the potential of the pretrained models to solve them. Accordingly, we investigate additional research directions and improvement ideas in this field.

View PDFchevron_right
Deep Neural Networks for Relation Extraction
Tapas Nayak

ArXiv, 2021

Relation extraction from text is an important task for automatic knowledge base population. In this thesis, we first propose a syntax-focused multi-factor attention network model for finding the relation between two entities. Next, we propose two joint entity and relation extraction frameworks based on encoder-decoder architecture. Finally, we propose a hierarchical entity graph convolutional network for relation extraction across documents.

View PDFchevron_right
ILK: Machine learning of semantic relations with shallow features and almost no data
Roser Morante

2007

This paper summarizes our approach to the Semeval 2007 shared task on "Classification of Semantic Relations between Nominals". Our overall strategy is to develop machine-learning classifiers making use of a few easily computable and effective features, selected independently for each classifier in wrapper experiments. We train two types of classifiers for each of the seven relations: with and without any WordNet information.

View PDFchevron_right
Deep Neural Architectures for End-to-End Relation Extraction
Tung Khanh Tran

2020

View PDFchevron_right
MIT at SemEval-2017 Task 10: Relation Extraction with Convolutional Neural Networks
Peter Szolovits

Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)

Over 50 million scholarly articles have been published: they constitute a unique repository of knowledge. In particular, one may infer from them relations between scientific concepts, such as synonyms and hyponyms. Artificial neural networks have been recently explored for relation extraction. In this work, we continue this line of work and present a system based on a convolutional neural network to extract relations. Our model ranked first in the SemEval-2017 task 10 (ScienceIE) for relation extraction in scientific articles (subtask C).

View PDFchevron_right
Comparative Analysis of Contextual Relation Extraction based on Deep Learning Models
Priyadharshini Ravichandran

An International Journal of Engineering Science, 2023

Contextual Relation Extraction (CRE) is mainly used for constructing a knowledge graph with a help of ontology. It performs various tasks such as semantic search, query answering, and textual entailment. Relation extraction identifies the entities from raw texts and the relations among them. An efficient and accurate CRE system is essential for creating domain knowledge in the biomedical industry. Existing Machine Learning and Natural Language Processing (NLP) techniques are not suitable to predict complex relations from sentences that consist of more than two relations and unspecified entities efficiently. In this work, deep learning techniques have been used to identify the appropriate semantic relation based on the context from multiple sentences. Even though various machine learning models have been used for relation extraction, they provide better results only for binary relations, i.e., relations occurred exactly between the two entities in a sentence. Machine learning models are not suited for complex sentences that consist of the words that have various meanings. To address these issues, hybrid deep learning models have been used to extract the relations from complex sentence effectively. This paper explores the analysis of various deep learning models that are used for relation extraction.

View PDFchevron_right
MIT-MEDG at SemEval-2018 Task 7: Semantic Relation Classification via Convolution Neural Network
Elena Sergeeva

Proceedings of The 12th International Workshop on Semantic Evaluation, 2018

SemEval 2018 Task 7 tasked participants to build a system to classify two entities within a sentence into one of the 6 possible relation types. We tested 3 classes of models: Linear classifiers, Long Short-Term Memory (LSTM) models, and Convolutional Neural Network (CNN) models. Ultimately, the CNN model class proved most performant, so we specialized to this model for our final submissions. We improved performance beyond a vanilla CNN by including a variant of negative sampling, using custom word embeddings learned over a corpus of ACL articles, training over corpora of both tasks 1.1 and 1.2, using reversed feature, using part of context words beyond the entity pairs and using ensemble methods to improve our final predictions. We also tested attention based pooling, upsampling, and data augmentation, but none improved performance. Our model achieved rank 6 out of 28 (macro-averaged F1-score: 72.7) in subtask 1.1, and rank 4 out of 20 (macro F1: 80.6) in subtask 1.2.

View PDFchevron_right
ECNU: Effective semantic relations classification without complicated features or multiple external corpora
Jian Su

2010

This paper describes our approach to the automatic identification of semantic relations between nominals in English sentences. The basic idea of our strategy is to develop machine-learning classifiers which: (1) make use of class-independent features and classifier; (2) make use of a simple and effective feature set without high computational cost; (3) make no use of external annotated or unannotated corpus at all. At SemEval 2010 Task 8 our system achieved an F-measure of 75.43% and a accuracy of 70.22%.

View PDFchevron_right
Improving Neural Relation Extraction with Implicit Mutual Relations
ullya rahmawati

2020 IEEE 36th International Conference on Data Engineering (ICDE)

Relation extraction (RE) aims at extracting the relation between two entities from the text corpora. It is a crucial task for Knowledge Graph (KG) construction. Most existing methods predict the relation between an entity pair by learning the relation from the training sentences, which contain the targeted entity pair. In contrast to existing distant supervision approaches that suffer from insufficient training corpora to extract relations, our proposal of mining implicit mutual relation from the massive unlabeled corpora transfers the semantic information of entity pairs into the RE model, which is more expressive and semantically plausible. After constructing an entity proximity graph based on the implicit mutual relations, we preserve the semantic relations of entity pairs via embedding each vertex of the graph into a low-dimensional space. As a result, we can easily and flexibly integrate the implicit mutual relations and other entity information, such as entity types, into the existing RE methods. Our experimental results on a New York Times and another Google Distant Supervision datasets suggest that our proposed neural RE framework provides a promising improvement for the RE task, and significantly outperforms the state-of-the-art methods. Moreover, the component for mining implicit mutual relations is so flexible that can help to improve the performance of both CNN-based and RNN-based RE models significant.

View PDFchevron_right
Semantic Relation Extraction: A Review of Approaches, Datasets, and Evaluation Methods
Hamid Gharagozlou

ACM Transactions on Asian and Low-Resource Language Information Processing

A large volume of unstructured data, especially text data, is generated and exchanged daily. Consequently, the importance of extracting patterns and discovering knowledge from textual data is significantly increasing. As the task of automatically recognizing the relations between two or more entities, semantic relation extraction has a prominent role in the exploitation of raw text. This paper surveys different approaches and types of relation extraction in English and the most prominent proposed methods in Persian. We also introduce, analyze, and compare the most important datasets available for relation extraction in Persian and English. Furthermore, traditional and emerging evaluation metrics for supervised, semi-supervised, and unsupervised methods are described, along with pointers to commonly used performance evaluation datasets. Finally, we briefly describe challenges in extracting relationships in Persian and English and dataset creation challenges.

View PDFchevron_right

References (157)

  1. Kosuke Akimoto, Takuya Hiraoka, Kunihiko Sadamasa, and Mathias Niepert. Cross- Sentence N-ary Relation Extraction using Lower-Arity Universal Schemas. In Proc. 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6225-6231, Hong Kong, China, 2019. Association for Computational Linguistics. URL www.aclweb.org/anthology/D19-1645. (Cited on pp. 26 and 50.)
  2. Fan Bai and Alan Ritter. Structured Minimally Supervised Learning for Neural Relation Extraction. In Proc. 2019 Conference of the North American Chapter of the Associa- tion for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3057-3069, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. URL www.aclweb.org/anthology/N19-1310. (Cited on p. 27.)
  3. Iz Beltagy, Kyle Lo, and Waleed Ammar. Combining Distant and Direct Supervision for Neural Relation Extraction. In Proc. 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1858-1867, 2019. (Cited on p. 31.)
  4. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A Neural Probabilistic Language Model. J. Machine Learning Research, 3:1137-1155, 2003. (Cited on pp. 7 and 8.)
  5. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent Dirichlet Allocation. Journal of Machine Learning Research, 3:993-1022, 2003. ISSN 1532-4435. URL dl.acm.org/citation.cfm?id=944919.944937. (Cited on p. 7.)
  6. Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. Learning Struc- tured Embeddings of Knowledge Bases. In Proc. Twenty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2011, pages 301-306, 2011. (Cited on p. 14.)
  7. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Ok- sana Yakhnenko. Translating Embeddings for Modeling Multi-relational Data. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Wein- berger, editors, Advances in Neural Information Processing Systems 26, pages 2787-2795. Curran Associates, Inc., 2013. URL papers.nips.cc/paper/ 5071-translating-embeddings-for-modeling-multi-relational-data.pdf. (Cited on pp. 35, 36, 48, and 49.)
  8. Duy-Cat Can, Hoang-Quynh Le, Quang-Thuy Ha, and Nigel Collier. A Richer-but- Smarter Shortest Dependency Path with Attentive Augmentation for Relation Ex- traction. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2902-2912, Minneapolis, Minnesota, 2019. Association for Com- putational Linguistics. URL www.aclweb.org/anthology/N19-1298. (Cited on pp. 21 and 40.)
  9. Kai-Wei Chang, Wen-tau Yih, Bishan Yang, and Christopher Meek. Typed Tensor Decomposition of Knowledge Bases for Relation Extraction. In Proc. 2014 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP) , pages 1568- 1579. Association for Computational Linguistics, 2014. URL aclweb.org/anthology/ D14-1165. (Cited on p. 12.)
  10. Jiyu Chen, Karin Verspoor, and Zenan Zhai. A Bag-of-concepts Model Improves Re- lation Extraction in a Narrow Knowledge Domain with Limited Data. In Proc. 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Student Research Workshop, pages 43-52, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-3007. URL www.aclweb.org/anthology/N19-3007. (Cited on p. 50.)
  11. Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In Proc. 2014 Con- ference on Empirical Methods in Natural Language Processing, EMNLP' 14, pages 1724-1734, Doha, Qatar, 2014. URL {www.aclweb.org/anthology/D14-1179}. (Cited on p. 18.)
  12. Fenia Christopoulou, Makoto Miwa, and Sophia Ananiadou. A Walk-based Model on Entity Graphs for Relation Extraction. In Proc. 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 81- 88, Melbourne, Australia, 2018. Association for Computational Linguistics. URL www.aclweb.org/anthology/P18-2014. (Cited on p. 44.)
  13. Ronan Collobert and Jason Weston. A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning. In Proc. 25th Inter- national Conference on Machine Learning, ICML '08, pages 160-167, New York, NY, USA, 2008. ACM. ISBN 978-1-60558-205-4. URL doi.acm.org/10.1145/1390156. 1390177. (Cited on p. 8.)
  14. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116, 2019. (Cited on p. 9.)
  15. Nilesh Dalvi, Pedro Domingos, Mausam, Sumit Sanghai, and Deepak Verma. Adversarial Classification. In Proc. Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '04, pages 99-108, New York, NY, USA, 2004. ACM. ISBN 1-58113-888-1. URL doi.acm.org/10.1145/1014052.1014066. (Cited on p. 29.)
  16. Rajarshi Das, Arvind Neelakantan, David Belanger, and Andrew McCallum. Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks. arXiv preprint arXiv:1607.01426, 2016. (Cited on p. 37.)
  17. Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolu- tional 2D Knowledge Graph Embeddings. In Proc. Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 1811-1818, 2018. URL www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/ view/17366. (Cited on pp. 36, 37, and 38.)
  18. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre- training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. (Cited on pp. 4, 9, and 23.)
  19. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre- training of Deep Bidirectional Transformers for Language Understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, 2019. URL www.aclweb.org/anthology/N19-1423. (Cited on p. 9.)
  20. Cícero Nogueira dos Santos, Bing Xiang, and Bowen Zhou. Classifying Relations by Ranking with Convolutional Neural Networks. In Proc. 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 626-634, 2015. (Cited on pp. 40 and 43.)
  21. Jinhua Du, Jingguang Han, Andy Way, and Dadong Wan. Multi-Level Structured Self- Attentions for Distantly Supervised Relation Extraction. In Proc. 2018 Conference on Empirical Methods in Natural Language Processing, pages 2216-2225, Brussels, Belgium, 2018. Association for Computational Linguistics. URL www.aclweb.org/ anthology/D18-1245. (Cited on p. 32.)
  22. Kawin Ethayarajh, David Duvenaud, and Graeme Hirst. Towards Understanding Linear Word Analogies. In Proc. 57th Annual Meeting of the Association for Computational Linguistics, pages 3253-3262, Florence, Italy, 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1315. (Cited on pp. 8 and 21.)
  23. Miao Fan, Deli Zhao, Qiang Zhou, Zhiyuan Liu, Thomas Fang Zheng, and Edward Y. Chang. Distant Supervision for Relation Extraction with Matrix Completion. In Proc. 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 839-849, Baltimore, Maryland, 2014. Association for Compu- tational Linguistics. doi: 10.3115/v1/P14-1079. URL www.aclweb.org/anthology/ P14-1079. (Cited on p. 28.)
  24. Miao Fan, Qiang Zhou, Thomas Fang Zheng, and Ralph Grishman. Distributed repre- sentation learning for knowledge graphs with entity descriptions. Pattern Recognition Letters, 93(2017):31-37, 2016. (Cited on p. 49.)
  25. Jun Feng, Minlie Huang, Li Zhao, Yang Yang, and Xiaoyan Zhu. Reinforcement learning for relation classification from noisy data. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018. (Cited on p. 33.)
  26. Gregory Finley, Stephanie Farmer, and Serguei Pakhomov. What Analogies Reveal about Word Vectors and their Compositionality. In Proc. 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), pages 1-11. Association for Com- putational Linguistics, 2017. URL www.aclweb.org/anthology/S17-1001. (Cited on p. 21.)
  27. Ruiji Fu, Jiang Guo, Bing Qin, Wanxiang Che, Haifeng Wang, and Ting Liu. Learning Semantic Hierarchies via Word Embeddings. In Proc. 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1199-1209, Baltimore, Maryland, 2014. Association for Computational Linguistics. URL www. aclweb.org/anthology/P14-1113. (Cited on p. 42.)
  28. George W. Furnas, Scott Deerwester, Susan T. Dumais, Thomas K. Landauer, Richard A. Harshman, Lynn A. Streeter, and Karen E. Lochbaum. Information re- trieval using a singular value decomposition model of latent semantic structure. In Proc. 11th Annual International ACM SIGIR Conference on Research and Develop- ment in Information Retrieval, pages 465-480, 1988. (Cited on p. 6.)
  29. Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. FewRel 2.0: Towards More Challenging Few-Shot Relation Classification. In Proc. Conference on Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing (EMNLP-IJCNLP 2019), pages 6250-6255, 2019. URL www.aclweb.org/anthology/D19-1649. (Cited on p. 27.)
  30. Alberto García-Durán and Mathias Niepert. Learning Graph Representations with Em- bedding Propagation. In Proc. 31st International Conference on Neural Information Processing Systems, pages 5125-5136, 2017. (Cited on p. 40.)
  31. Matt Gardner and Tom Mitchell. Efficient and Expressive Knowledge Base Completion Using Subgraph Feature Extraction. In Proc. 2015 Conference on Empirical Meth- ods in Natural Language Processing, pages 1488-1498. Association for Computational Linguistics, 2015. URL aclweb.org/anthology/D15-1173. (Cited on pp. 26 and 37.)
  32. Matt Gardner, Partha Pratim Talukdar, Bryan Kisiel, and Tom Mitchell. Improving Learning and Inference in a Large Knowledge-Base using Latent Syntactic Cues. In Proc. 2013 Conference on Empirical Methods in Natural Language Processing, pages 833-838. Association for Computational Linguistics, 2013. URL www.aclweb.org/ anthology/D13-1080. (Cited on pp. 45 and 46.)
  33. Matt Gardner, Partha Talukdar, Jayant Krishnamurthy, and Tom Mitchell. Incorporat- ing Vector Space Similarity in Random Walk Inference over Knowledge Bases. In Proc. 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 397-406, Doha, Qatar, 2014. Association for Computational Linguistics. URL www.aclweb.org/anthology/D14-1044. (Cited on pp. 13 and 46.)
  34. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sher- jil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Ad- vances in neural information processing systems, pages 2672-2680, 2014. (Cited on pp. 29 and 30.)
  35. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. www.deeplearningbook.org. (Cited on p. 3.)
  36. Matthew R. Gormley, Mo Yu, and Mark Dredze. Improved Relation Extraction with Feature-Rich Compositional Embedding Models. In Proc. 2015 Conference on Em- pirical Methods in Natural Language Processing, pages 1774-1784, Lisbon, Portugal, 2015. Association for Computational Linguistics. URL www.aclweb.org/anthology/ D15-1205. (Cited on p. 17.)
  37. Aditya Grover and Jure Leskovec. node2vec: Scalable Feature Learning for Networks. In Proc. 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 855-864, 2016. (Cited on p. 15.)
  38. Michael Gutmann and Aapo Hyvarinen. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. The Journal of Ma- chine Learning Research, 13(1):307-361, 2012. (Cited on p. 35.)
  39. Kelvin Guu, John Miller, and Percy Liang. Traversing Knowledge Graphs in Vec- tor Space. In Proc. 2015 Conference on Empirical Methods in Natural Language Processing, pages 318-327. Association for Computational Linguistics, 2015. doi: 10.18653/v1/D15-1038. URL aclweb.org/anthology/D15-1038. (Cited on p. 37.)
  40. Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), pages 4803-4809, 2018. URL www.aclweb.org/ anthology/D18-1514. (Cited on p. 27.)
  41. Shizhu He, Kang Liu, Guoliang Ji, and Jun Zhao. Learning to Represent Knowledge Graphs with Gaussian Embedding. In Proc. 24th ACM International on Conference on Information and Knowledge Management, CIKM '15, pages 623-632, New York, NY, USA, 2015. ACM. ISBN 978-1-4503-3794-6. URL doi.acm.org/10.1145/2806416. 2806502. (Cited on p. 35.)
  42. Johannes M. Heuckmann, Michael Hölzel, Martin L. Sos, Stefanie Heynck, Hyatt Balke- Want, Mirjam Koker, Martin Peifer, Jonathan Weiss, Christine M. Lovly, Christian Grütter, Daniel Rauh, William Pao, and Roman K. Thomas. ALK Mutations Con- ferring Differential Resistance to Structurally Diverse ALK Inhibitors. Clinical Can- cer Research, 17(23):7394-7401, 2011. URL www.ncbi.nlm.nih.gov/pmc/articles/ PMC3382103/. (Cited on p. 49.)
  43. Sepp Hochreiter and Jürgen Schmidhuber. Long Short-Term Memory. Neural Compu- tation, 9(8):1735-1780, 1997. URL doi.org/10.1162/neco.1997.9.8.1735. (Cited on p. 18.)
  44. Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S. Weld. Knowledge-based Weak Supervision for Information Extraction of Overlapping Rela- tions. In Proc. 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies -Volume 1, HLT '11, pages 541-550, Stroudsburg, PA, USA, 2011. Association for Computational Linguistics. ISBN 978-1-932432-87-9. URL dl.acm.org/citation.cfm?id=2002472.2002541. (Cited on pp. 27 and 28.)
  45. Ignacio Iacobacci, Mohammad Taher Pilehvar, and Roberto Navigli. SensEmbed: Learning Sense Embeddings for Word and Relational Similarity. In Proc. 53rd An- nual Meeting of the Association for Computational Linguistics and the 7th Interna- tional Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 95-105, Beijing, China, 2015. Association for Computational Linguistics. URL www.aclweb.org/anthology/P15-1010. (Cited on p. 9.)
  46. Sharmistha Jat, Siddhesh Khandelwal, and Partha P. Talukdar. Improving Distantly Supervised Relation Extraction using Word and Entity Based Attention. CoRR, abs/1804.06987, 2018. URL arxiv.org/abs/1804.06987. (Cited on p. 26.)
  47. Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowledge graph embed- ding via dynamic mapping matrix. In Proc. 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 687-696, 2015. (Cited on p. 35.)
  48. Guoliang Ji, Kang Liu, Shizhu He, and Jun Zhao. Knowledge graph completion with adaptive sparse transfer matrix. In Proc. Thirtieth AAAI Conference on Artificial Intelligence, 2016. (Cited on p. 35.)
  49. Guoliang Ji, Kang Liu, Shizhu He, and Jun Zhao. Distant Supervision for Relation Extraction with Sentence-Level Attention and Entity Descriptions. In Proc. Thirty- First AAAI Conference on Artificial Intelligence (AAAI-17), pages 3060-3066, 2017. (Cited on p. 32.)
  50. Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and Philip S. Yu. A Survey on Knowledge Graphs: Representation, Acquisition and Applications, 2020. (Cited on pp. 12, 34, and 36.)
  51. Robin Jia, Cliff Wong, and Hoifung Poon. Document-Level N-ary Relation Extrac- tion with Multiscale Representation Learning. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume 1 (Long and Short Papers), pages 3693-3704, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. URL www.aclweb.org/anthology/N19-1370. (Cited on p. 50.)
  52. Xiaotian Jiang, Quan Wang, and Bin Wang. Adaptive Convolution for Multi-Relational Learning. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 978-987, Minneapolis, Minnesota, 2019. Association for Compu- tational Linguistics. doi: 10.18653/v1/N19-1103. (Cited on pp. 37 and 38.)
  53. Ian T. Jolliffe. Principal Component Analysis. Springer Series in Statistics. Springer- Verlag, New York, 2002. (Cited on p. 7.)
  54. Bhushan Kotnis and Vivi Nastase. Learning Knowledge Graph Embeddings with Type Regularizer. In Proc. Knowledge Capture Conference, K-CAP 2017, pages 19:1-19:4. ACM, 2017. ISBN 978-1-4503-5553-7. (Cited on p. 12.)
  55. Bhushan Kotnis and Vivi Nastase. Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs. In Workshop on Knowledge Base Construction, Reasoning and Mining (KBCOM), 2018. URL arxiv.org/abs/1708.06816. (Cited on p. 35.)
  56. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations, ICLR '20, 2020. (Cited on p. 9.)
  57. Ni Lao, Tom Mitchell, and William W. Cohen. Random Walk Inference and Learning in A Large Scale Knowledge Base. In Proc. 2011 Conference on Empirical Methods in Natural Language Processing, pages 529-539. Association for Computational Lin- guistics, 2011. URL www.aclweb.org/anthology/D11-1049. (Cited on pp. 36 and 45.)
  58. Ni Lao, Amarnag Subramanya, Fernando Pereira, and William W. Cohen. Reading The Web with Learned Syntactic-Semantic Inference Rules. In Proc. 2012 Joint Confer- ence on Empirical Methods in Natural Language Processing and Computational Natu- ral Language Learning, pages 1017-1026. Association for Computational Linguistics, 2012. URL www.aclweb.org/anthology/D12-1093. (Cited on p. 45.)
  59. Yann LeCun and Yoshua Bengio. Convolutional Networks for Images, Speech, and Time Series. The handbook of brain theory and neural networks, 3361(10):1995, 1995. (Cited on p. 40.)
  60. Omer Levy and Yoav Goldberg. Neural word embedding as implicit matrix factorization. In Advances in neural information processing systems, pages 2177-2185, 2014a. (Cited on p. 11.)
  61. Omer Levy and Yoav Goldberg. Linguistic Regularities in Sparse and Explicit Word Representations. In Proc. Eighteenth Conference on Computational Natural Language Learning, pages 171-180. Association for Computational Linguistics, 2014b. URL www.aclweb.org/anthology/W14-1618. (Cited on p. 21.)
  62. Pengshuai Li, Xinsong Zhang, Weijia Jia, and Hai Zhao. GAN Driven Semi-distant Supervision for Relation Extraction. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, Volume 1 (Long and Short Papers), pages 3026-3035, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. URL www.aclweb.org/anthology/ N19-1307. (Cited on p. 30.)
  63. Qi Li and Heng Ji. Incremental Joint Extraction of Entity Mentions and Relations. In Proc. 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 402-412, Baltimore, Maryland, 2014. Association for Compu- tational Linguistics. URL www.aclweb.org/anthology/P14-1038. (Cited on pp. 42, 43, and 44.)
  64. Yan Liang, Xin Liu, Jianwen Zhang, and Yangqiu Song. Relation Discovery with Out-of-Relation Knowledge Base as Supervision. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume 1 (Long and Short Papers), pages 3280-3290, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. URL www.aclweb.org/anthology/N19-1332. (Cited on p. 51.)
  65. Thomas Lin, Oren Etzioni, et al. Entity linking at web scale. In Proc. Joint Workshop on Automatic Knowledge Base Construction and Web-scale Knowledge Extraction, pages 84-88. Association for Computational Linguistics, 2012. (Cited on p. 48.)
  66. Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning Entity and Relation Embeddings for Knowledge Graph Completion. In Proc. Twenty-Ninth AAAI Conference on Artificial Intelligence, AAAI'15, pages 2181-2187. AAAI Press, 2015. ISBN 0-262-51129-0. URL dl.acm.org/citation.cfm?id=2886521.2886624. (Cited on p. 35.)
  67. Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. Neural Relation Extraction with Selective Attention over Instances. In Proc. 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2124-2133, Berlin, Germany, 2016. Association for Computational Linguistics. URL www.aclweb.org/anthology/P16-1200. (Cited on p. 31.)
  68. Tianyi Liu, Xinsong Zhang, Wanhao Zhou, and Weijia Jia. Neural Relation Extraction via Inner-Sentence Noise Reduction and Transfer Learning. In Proc. 2018 Conference on Empirical Methods in Natural Language Processing, pages 2195-2204, Brussels, Belgium, 2018. Association for Computational Linguistics. URL www.aclweb.org/ anthology/D18-1243. (Cited on p. 31.)
  69. Yang Liu, Furu Wei, Sujian Li, Heng Ji, Ming Zhou, and Houfeng Wang. A Dependency- Based Neural Network for Relation Classification. In Proc. 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 285-290, 2015. URL www.aclweb.org/anthology/P15-2047. (Cited on pp. 20, 22, and 40.)
  70. Yang Liu, Yifeng Zeng, Yingke Chen, Jing Tang, and Yinghui Pan. Self-Improving Gen- erative Adversarial Reinforcement Learning. In Proc. 18th International Conference on Autonomous Agents and MultiAgent Systems, pages 52-60, 2019a. (Cited on p. 33.)
  71. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly op- timized BERT pretraining approach. arXiv preprint arXiv:1907.11692, 2019b. (Cited on p. 9.)
  72. Colin Lockard, Prashant Shiralkar, and Xin Luna Dong. OpenCeres: When Open In- formation Extraction Meets the Semi-Structured Web. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3047- 3056, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. URL www.aclweb.org/anthology/N19-1309. (Cited on p. 48.)
  73. Bo Long, Zhongfei (Mark) Zhang, Xiaoyun Wu, and Philip S. Yu. Spectral Clustering for Multi-type Relational Data. In Proc. 23rd International Conference on Machine Learning (ICML '06), pages 585-592, 2006. doi: doi.org/10.1145/1143844.1143918. (Cited on p. 14.)
  74. Fan Luo, Ajay Nagesh, Rebecca Sharp, and Mihai Surdeanu. Semi-Supervised Teacher- Student Architecture for Relation Extraction. In Proc. Third Workshop on Structured Prediction for NLP, pages 29-37, Minneapolis, Minnesota, 2019. Association for Com- putational Linguistics. URL www.aclweb.org/anthology/W19-1505. (Cited on p. 34.)
  75. Diego Marcheggiani and Ivan Titov. Discrete-State Variational Autoencoders for Joint Discovery and Factorization of Relations. Transactions of the Association for Computational Linguistics, 4:231-244, 2016. doi: 10.1162/tacl_a_00095. URL www.aclweb.org/anthology/Q16-1017. (Cited on p. 51.)
  76. Warren S. McCulloch and Walter H. Pitts. A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5:115--133, 1943. (Cited on p. 1.)
  77. Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan Černockỳ, and Sanjeev Khudanpur. Recurrent neural network based language model. In Eleventh annual conference of the international speech communication association, 2010. (Cited on p. 18.)
  78. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient Estimation of Word Representations in Vector Space, 2013a. (Cited on p. 36.)
  79. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed Representations of Words and Phrases and their Compositionality. In C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pages 3111-3119, 2013b. (Cited on p. 8.)
  80. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic Regularities in Continuous Space Word Representations. In Proc. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746-751, Atlanta, Georgia, 2013c. (Cited on pp. 1, 8, 9, and 21.)
  81. Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. Distant supervision for relation extraction without labeled data. In Proc. Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Process- ing of the AFNLP: Volume 2-Volume 2, ACL '09, pages 1003-1011, 2009. ISBN 978-1-932432-46-6. URL dl.acm.org/citation.cfm?id=1690219.1690287. (Cited on p. 26.)
  82. Jeff Mitchell and Mirella Lapata. Composition in distributional models of semantics. Cognitive science, 34 8:1388 -1429, 2010. (Cited on p. 17.)
  83. Makoto Miwa and Mohit Bansal. End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures. In Proc. 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1105-1116, Berlin, Germany, 2016. Association for Computational Linguistics. URL www.aclweb.org/ anthology/P16-1105. (Cited on pp. 24 and 43.)
  84. Makoto Miwa and Yutaka Sasaki. Modeling Joint Entity and Relation Extraction with Table Representation. In Proc. 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1858-1869, Doha, Qatar, 2014. Association for Computational Linguistics. URL www.aclweb.org/anthology/D14-1200. (Cited on p. 43.)
  85. Arvind Neelakantan, Jeevan Shankar, Alexandre Passos, and Andrew McCallum. Effi- cient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space. In Proc. 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1059-1069, Doha, Qatar, 2014. Association for Computational Lin- guistics. URL www.aclweb.org/anthology/D14-1113. (Cited on p. 9.)
  86. Arvind Neelakantan, Benjamin Roth, and Andrew McCallum. Compositional Vector Space Models for Knowledge Base Completion. In Proc. 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 156-166. Association for Computational Linguistics, 2015. URL aclweb.org/anthology/P15-1016. (Cited on p. 37.)
  87. Thien Huu Nguyen and Ralph Grishman. Relation Extraction: Perspective from Con- volutional Neural Networks. In Proc. 1st Workshop on Vector Space Modeling for Natural Language Processing, pages 39-48, Denver, Colorado, 2015. Association for Computational Linguistics. URL www.aclweb.org/anthology/W15-1506. (Cited on p. 40.)
  88. Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. A Three-Way Model for Collective Learning on Multi-Relational Data. In Proceedings of ICML, 2011. (Cited on pp. 4, 10, 35, and 36.)
  89. Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. A Review of Relational Machine Learning for Knowledge Graphs. Proc. IEEE, 104(1):11-33, 2016a. ISSN 0018-9219. doi: 10.1109/JPROC.2015.2483592. (Cited on pp. 11, 12, and 34.)
  90. Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. Holographic Embeddings of Knowledge Graphs. In Proc. Thirtieth AAAI Conference on Artificial Intelligence, AAAI'16, pages 1955-1961. AAAI Press, 2016b. URL dl.acm.org/citation.cfm? id=3016100.3016172. (Cited on pp. 12 and 36.)
  91. Mathias Niepert. Discriminative Gaifman Models. In Proc. 30th International Con- ference on Neural Information Processing Systems, NIPS'16, pages 3413-3421, USA, 2016. Curran Associates Inc. ISBN 978-1-5108-3881-9. URL dl.acm.org/citation. cfm?id=3157382.3157479. (Cited on p. 39.)
  92. Madhav Nimishakavi, Uday Singh Saini, and Partha P. Talukdar. Relation Schema Induction using Tensor Factorization with Side Information. CoRR, abs/1605.04227, 2016. URL arxiv.org/abs/1605.04227. (Cited on pp. 47 and 48.)
  93. Alberto Paccanaro and Geoffrey E. Hinton. Learning Hierarchical Structures with Linear Relational Embedding. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, pages 857-864. Curran Associates, Inc., 2002. URL papers.nips.cc/paper/ 2068-learning-hierarchical-structures-with-linear-relational-embedding. pdf. (Cited on p. 13.)
  94. Nanyun Peng, Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih. Cross- Sentence N-ary Relation Extraction with Graph LSTMs. Transactions of the Asso- ciation for Computational Linguistics, 5:101-115, 2017. doi: 10.1162/tacl_a_00049. URL www.aclweb.org/anthology/Q17-1008. (Cited on pp. 25 and 50.)
  95. Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global Vectors for Word Representation. In Proc. 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532-1543, Doha, Qatar, 2014. Association for Computational Linguistics. URL www.aclweb.org/anthology/D14-1162. (Cited on p. 9.)
  96. Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. DeepWalk: Online Learning of Social Representations. In Proc. 20th ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, KDD '14, pages 701-710, 2014. ISBN 978-1-4503- 2956-9. URL doi.acm.org/10.1145/2623330.2623732. (Cited on pp. 14 and 36.)
  97. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Ken- ton Lee, and Luke Zettlemoyer. Deep Contextualized Word Representations. In Proc. 2018 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227-2237, New Orleans, Louisiana, 2018. Association for Computational Linguistics. URL www.aclweb.org/anthology/N18-1202. (Cited on p. 9.)
  98. Mohammad Taher Pilehvar and Nigel Collier. De-Conflated Semantic Representations. In Proc. 2016 Conference on Empirical Methods in Natural Language Processing, pages 1680-1690, Austin, Texas, 2016. Association for Computational Linguistics. doi: 10. 18653/v1/D16-1174. URL www.aclweb.org/anthology/D16-1174. (Cited on p. 9.)
  99. Pengda Qin, Weiran XU, and William Yang Wang. DSGAN: Generative Adversarial Training for Distant Supervision Relation Extraction. In Proc. 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 496- 505, Melbourne, Australia, 2018a. Association for Computational Linguistics. URL www.aclweb.org/anthology/P18-1046. (Cited on p. 30.)
  100. Pengda Qin, Weiran Xu, and William Yang Wang. Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning. In Proc. 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2137-2147, Melbourne, Australia, 2018b. Association for Computational Linguistics. URL www. aclweb.org/anthology/P18-1199. (Cited on p. 33.)
  101. Chris Quirk and Hoifung Poon. Distant Supervision for Relation Extraction beyond the Sentence Boundary. In Proc. 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 1171-1182, Valencia, Spain, 2017. Association for Computational Linguistics. URL www.aclweb. org/anthology/E17-1110. (Cited on p. 50.)
  102. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. arXiv e-prints, 2019. (Cited on p. 9.)
  103. Xiang Ren, Zeqiu Wu, Wenqi He, Meng Qu, Clare R Voss, Heng Ji, Tarek F Abdelza- her, and Jiawei Han. Cotype: Joint extraction of typed entities and relations with knowledge bases. In Proc. 26th International Conference on World Wide Web, pages 1015-1024. International World Wide Web Conferences Steering Committee, 2017. (Cited on pp. 12 and 42.)
  104. Sebastian Riedel, Limin Yao, and Andrew McCallum. Modeling Relations and Their Mentions Without Labeled Text. In Proc. 2010 European Conference on Machine Learning and Knowledge Discovery in Databases: Part III, ECML PKDD'10, pages 148-163, Berlin, Heidelberg, 2010. Springer-Verlag. ISBN 3-642-15938-9, 978-3-642- 15938-1. URL dl.acm.org/citation.cfm?id=1889788.1889799. (Cited on p. 26.)
  105. Sebastian Riedel, Limin Yao, Andrew McCallum, and M. Benjamin Marlin. Relation Ex- traction with Matrix Factorization and Universal Schemas. In Proc. 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 74-84. Association for Computational Linguistics, 2013. URL aclweb.org/anthology/N13-1008. (Cited on pp. 15 and 47.)
  106. Frank. Rosenblatt. The Perceptron: A Probabilistic Model for Information Storage and Organization in The Brain. Psychological Review, 65(6):386-408, 1958. (Cited on p. 1.)
  107. Gaetano Rossiello, Alfio Gliozzo, Robert Farrell, Nicolas Fauceglia, and Michael Glass. Learning Relational Representations by Analogy using Hierarchical Siamese Networks. In Proc. 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, Volume 1 (Long and Short Pa- pers), pages 3235-3245, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. URL www.aclweb.org/anthology/N19-1327. (Cited on p. 41.)
  108. Dan Roth and Wen-Tau Yih. Global Inference for Entity and Relation Identification via a Linear Programming Formulation. In Introduction to Statistical Relational Learning. MIT Press, 2007. (Cited on p. 42.)
  109. Chengsen Ru, Jintao Tang, Sasha Li, Songxian Xie, and Ting Wang. Using seman- tic similarity to reduce wrong labels in distant supervision for relation extraction. Information Processing and Management, 54(4):593-608, 2018. (Cited on p. 27.)
  110. David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning Rep- resentations by Back-propagating Errors. Nature, 323(6088):533-536, 1986. doi: 10.1038/323533a0. (Cited on pp. 1 and 12.)
  111. Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The Graph Neural Network Model. Trans. Neur. Netw., 20(1):61-80, 2009. ISSN 1045-9227. URL dx.doi.org/10.1109/TNN.2008.2005605. (Cited on pp. 24 and 38.)
  112. Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In European Semantic Web Conference, pages 593-607. Springer, 2018. (Cited on p. 39.)
  113. Jürgen Schmidhuber. Artificial curiosity based on discovering novel algorithmic predictability through coevolution. In Proc. 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406), volume 3, pages 1612-1618. IEEE, 1999. (Cited on p. 29.)
  114. Vered Shwartz and Ido Dagan. Paraphrase to Explicate: Revealing Implicit Noun- Compound Relations. In Proc. 56th Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), pages 1200-1211, Melbourne, Australia, 2018. Association for Computational Linguistics. URL www.aclweb.org/anthology/ P18-1111. (Cited on p. 41.)
  115. Vered Shwartz, Yoav Goldberg, and Ido Dagan. Improving hypernymy detection with an integrated path-based and distributional method. arXiv preprint arXiv:1603.06076, 2016. (Cited on p. 42.)
  116. Ajit Paul Singh and Geoffrey J. Gordon. Relational learning via collective matrix fac- torization. In Proc. 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2008. doi: 10.1145/1401890.1401969. (Cited on p. 14.)
  117. Sameer Singh, Sebastian Riedel, Brian Martin, Jiaping Zheng, and Andrew McCallum. Joint Inference of Entities, Relations, and Coreference. In Proc. 2013 Workshop on Automated Knowledge Base Construction (AKBC '13), pages 1-6, 2013. ISBN 978- 1-4503-2411-3. URL doi.acm.org/10.1145/2509558.2509559. (Cited on p. 42.)
  118. Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. Matching the Blanks: Distributional Similarity for Relation Learning. In Proc. 57th Annual Meeting of the Association for Computational Linguistics, pages 2895-2905, 2019. (Cited on p. 23.)
  119. Richard Socher, Eric H. Huang, Jeffrey Pennington, Andrew Y. Ng, and Christopher D. Manning. Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection. In Proc. 24th International Conference on Neural Information Processing Systems, NIPS'11, pages 801-809, USA, 2011a. Curran Associates Inc. ISBN 978-1- 61839-599-3. URL dl.acm.org/citation.cfm?id=2986459.2986549. (Cited on p. 26.)
  120. Richard Socher, Cliff Chiung-Yu Lin, Andrew Y. Ng, and Christopher D. Manning. Parsing Natural Scenes and Natural Language with Recursive Neural Networks. In Proc. 28th International Conference on International Conference on Machine Learn- ing, ICML'11, pages 129-136, USA, 2011b. Omnipress. ISBN 978-1-4503-0619-5. URL dl.acm.org/citation.cfm?id=3104482.3104499. (Cited on p. 18.)
  121. Richard Socher, Brody Huval, Christopher D. Manning, and Andrew Y. Ng. Seman- tic Compositionality Through Recursive Matrix-vector Spaces. In Proc. 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL '12, pages 1201-1211, Stroudsburg, PA, USA, 2012. Association for Computational Linguistics. URL dl.acm.org/citation. cfm?id=2390948.2391084. (Cited on pp. 21 and 22.)
  122. Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. Rea- soning With Neural Tensor Networks for Knowledge Base Completion. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Wein- berger, editors, Advances in Neural Information Processing Systems 26, pages 926-934. Curran Associates, Inc., 2013. URL papers.nips.cc/paper/ 5028-reasoning-with-neural-tensor-networks-for-knowledge-base-completion. pdf. (Cited on p. 35.)
  123. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A Simple Way to Prevent Neural Networks from Over- fitting. Journal of Machine Learning Research, 15(56):1929-1958, 2014. URL jmlr.org/papers/v15/srivastava14a.html. (Cited on p. 5.)
  124. M. Steyvers and T. Griffiths. Probabilistic topic models. In Latent Semantic Analysis: A Road to Meaning. Lawrence Erlbaum, 2006. (Cited on p. 7.)
  125. Sen Su, Ningning Jia, Xiang Cheng, Shuguang Zhu, and Ruiping Li. Exploring Encoder- Decoder Model for Distant SupervisedRelation Extraction. In Proc. Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18), pages 4389-4395, 2018. (Cited on p. 28.)
  126. Shaohua Sun, Ni Lao, Rahul Gupta, and Dave Orr. 50,000 Lessons on How to Read: a Relation Extraction Corpus, 2013. URL research.googleblog.com/2013/04/ 50000-lessons-on-how-to-read-relation.html. (Cited on p. 26.)
  127. Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D. Manning. Multi-instance Multi-label Learning for Relation Extraction. In Proc. 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL '12, pages 455-465, Stroudsburg, PA, USA, 2012. Association for Computational Linguistics. URL dl.acm.org/citation. cfm?id=2390948.2391003. (Cited on p. 28.)
  128. Ilya Sutskever and Geoffrey E. Hinton. Using matrices to model sym- bolic relationship. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bot- tou, editors, Advances in Neural Information Processing Systems 21, pages 1593-1600. Curran Associates, Inc., 2009. URL papers.nips.cc/paper/ 3482-using-matrices-to-model-symbolic-relationship.pdf. (Cited on p. 14.)
  129. Ryo Takahashi, Ran Tian, and Kentaro Inui. Interpretable and Compositional Relation Learning by Joint Training with an Autoencoder. In Proc. 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2148- 2159, Melbourne, Australia, 2018. Association for Computational Linguistics. URL www.aclweb.org/anthology/P18-1200. (Cited on p. 37.)
  130. Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. Representing Text for Joint Embedding of Text and Knowl- edge Bases. In Proc. 2015 Conference on Empirical Methods in Natural Language Processing, pages 1499-1509, Lisbon, Portugal, 2015. Association for Computational Linguistics. URL www.aclweb.org/anthology/D15-1174. (Cited on pp. 16 and 46.)
  131. Kristina Toutanova, Victoria Lin, Wen-tau Yih, Hoifung Poon, and Chris Quirk. Com- positional Learning of Embeddings for Relation Paths in Knowledge Base and Text. In Proc. 54th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 1434-1444. Association for Computational Linguistics, 2016. URL www.aclweb.org/anthology/P16-1136. (Cited on p. 46.)
  132. Théo Trouillon, Christopher R Dance, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Knowledge Graph Completion via Complex Tensor Factor- ization. arXiv preprint arXiv:1702.06879, 2017. (Cited on pp. 26 and 36.)
  133. Shikhar Vashishth, Rishabh Joshi, Sai Suman Prayaga, Chiranjib Bhattacharyya, and Partha Talukdar. RESIDE: Improving Distantly-Supervised Neural Relation Extrac- tion using Side Information. In Proc. 2018 Conference on Empirical Methods in Nat- ural Language Processing, pages 1257-1266, Brussels, Belgium, 2018. Association for Computational Linguistics. URL www.aclweb.org/anthology/D18-1157. (Cited on p. 29.)
  134. Patrick Verga, Arvind Neelakantan, and Andrew McCallum. Generalizing to Unseen Entities and Entity Pairs with Rowless Universal Schema. In Proc. 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 613-622, Valencia, Spain, 2017. Association for Computational Linguistics. URL www.aclweb.org/anthology/E17-1058. (Cited on p. 47.)
  135. Guanying Wang, Wen Zhang, Ruoxu Wang, Yalin Zhou, Xi Chen, Wei Zhang, Hai Zhu, and Huajun Chen. Label-Free Distant Supervision for Relation Extraction via Knowledge Graph Embedding. In Proc. 2018 Conference on Empirical Methods in Natural Language Processing, pages 2246-2255, Brussels, Belgium, 2018. Association for Computational Linguistics. URL www.aclweb.org/anthology/D18-1248. (Cited on p. 28.)
  136. Hai Wang and Hoifung Poon. Deep Probabilistic Logic: A Unifying Framework for Indi- rect Supervision. In Proc. 2018 Conference on Empirical Methods in Natural Language Processing, pages 1891-1902, Brussels, Belgium, 2018. Association for Computational Linguistics. URL www.aclweb.org/anthology/D18-1215. (Cited on p. 50.)
  137. Hong Wang, Wenhan Xiong, Mo Yu, Xiaoxiao Guo, Shiyu Chang, and William Yang Wang. Sentence Embedding Alignment for Lifelong Relation Extraction. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 796-806, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. URL www.aclweb.org/anthology/N19-1086. (Cited on pp. 51 and 52.)
  138. Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. Knowledge Graph Embedding: A Survey of Approaches and Applications. IEEE Transactions on Knowledge and Data Engineering, 29(12):2724-2743, 2017. ISSN 1041-4347. doi: 10.1109/TKDE. 2017.2754499. (Cited on pp. 12, 34, and 36.)
  139. Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embed- ding by translating on hyperplanes. In Twenty-Eighth AAAI conference on artificial intelligence, 2014. (Cited on p. 35.)
  140. Jason Weston, Antoine Bordes, Oksana Yakhnenko, and Nicolas Usunier. Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction. In Proc. 2013 Conference on Empirical Methods in Natural Language Processing, pages 1366-1371. Association for Computational Linguistics, 2013. URL www.aclweb.org/ anthology/D13-1136. (Cited on p. 48.)
  141. Han Xiao, Minlie Huang, and Xiaoyan Zhu. Transg: A generative model for knowledge graph embedding. In Proc. 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 2316-2325, 2016. (Cited on pp. 35 and 36.)
  142. Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. Representation learning of knowledge graphs with entity descriptions. In Proceedings of AAAI-2016, pages 2659-2665, 2016. (Cited on p. 49.)
  143. Kun Xu, Yansong Feng, Songfang Huang, and Dongyan Zhao. Semantic Relation Clas- sification via Convolutional Neural Networks with Simple Negative Sampling. CoRR, abs/1506.07650, 2015. URL arxiv.org/abs/1506.07650. (Cited on p. 43.)
  144. Yan Xu, Ran Jia, Lili Mou, Ge Li, Yunchuan Chen, Yangyang Lu, and Zhi Jin. Improved relation classification by deep recurrent neural networks with data augmentation. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1461-1470, Osaka, Japan, 2016. The COLING 2016 Organizing Committee. URL www.aclweb.org/anthology/C16-1138. (Cited on pp. 19, 20, and 40.)
  145. Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. In Proc. 2015 International Conference on Representation Learning, 2015. (Cited on pp. 35 and 36.)
  146. Kaijia Yang, Liang He, Xin-yu Dai, Shujian Huang, and Jiajun Chen. Exploiting Noisy Data in Distant Supervision Relation Classification. In Proc. 2019 Confer- ence of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3216- 3225, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. URL www.aclweb.org/anthology/N19-1325. (Cited on p. 33.)
  147. Zhi-Xiu Ye and Zhen-Hua Ling. Distant Supervision Relation Extraction with Intra- Bag and Inter-Bag Attentions. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, Volume 1 (Long and Short Papers), pages 2810-2819, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. URL www.aclweb.org/anthology/ N19-1288. (Cited on p. 31.)
  148. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao, et al. Relation Classifi- cation via Convolutional Deep Neural Network. In COLING, pages 2335-2344, 2014. (Cited on pp. 40 and 41.)
  149. Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks . In Proc. 2015 Conference on Empirical Methods in Natural Language Processing , pages 1753-1762. Association for Computational Linguistics, 2015. URL aclweb.org/anthology/D15-1203. (Cited on p. 28.)
  150. Xiangrong Zeng, Daojian Zeng, Shizhu He, Kang Liu, and Jun Zhao. Extracting Rela- tional Facts by an End-to-End Neural Model with Copy Mechanism. In Proc. 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 506-514, Melbourne, Australia, 2018. Association for Computational Linguistics. URL www.aclweb.org/anthology/P18-1047. (Cited on p. 44.)
  151. Dongxu Zhang, Subhabrata Mukherjee, Colin Lockard, Luna Dong, and Andrew Mc- Callum. OpenKI: Integrating Open Information Extraction and Knowledge Bases with Relation Inference. In Proc. 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technolo- gies, Volume 1 (Long and Short Papers), pages 762-772, Minneapolis, Minnesota, 2019a. Association for Computational Linguistics. doi: 10.18653/v1/N19-1083. URL www.aclweb.org/anthology/N19-1083. (Cited on p. 48.)
  152. Ningyu Zhang, Shumin Deng, Zhanlin Sun, Guanying Wang, Xi Chen, Wei Zhang, and Huajun Chen. Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks. In Proc. 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3016-3025, Minneapolis, Minnesota, 2019b. Association for Computational Linguistics. (Cited on pp. 39 and 49.)
  153. Yuhao Zhang, Peng Qi, and Christopher D. Manning. Graph Convolution over Pruned Dependency Trees Improves Relation Extraction. In Proc. 2018 Conference on Em- pirical Methods in Natural Language Processing, pages 2205-2215, Brussels, Belgium, 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1244. (Cited on p. 25.)
  154. Suncong Zheng, Jiaming Xu, Peng Zhou, Hongyun Bao, Zhenyu Qi, and Bo Xu. A neural network framework for relation extraction: Learning entity semantic and re- lation pattern. Knowledge-Based Systems, 114:12 -23, 2016. ISSN 0950-7051. doi: doi.org/10.1016/j.knosys.2016.09.019. URL www.sciencedirect.com/science/ article/pii/S0950705116303501. (Cited on p. 40.)
  155. Suncong Zheng, Feng Wang, Hongyun Bao, Yuexing Hao, Peng Zhou, and Bo Xu. Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme. In Proc. 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers), pages 1227-1236, Vancouver, Canada, 2017. Association for Computational Lin- guistics. doi: 10.18653/v1/P17-1113. URL www.aclweb.org/anthology/P17-1113. (Cited on pp. 43 and 44.)
  156. Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, and Maosong Sun. Graph Neural Networks: A Review of Methods and Applications. CoRR, abs/1812.08434, 2018. URL arxiv.org/abs/1812.08434. (Cited on pp. 24 and 38.)
  157. Yan Zhou, Murat Kantarcioglu, Bhavani Thuraisingham, and Bowei Xi. Adversarial Support Vector Machine Learning. In Proc. 18th ACM SIGKDD International Con- ference on Knowledge Discovery and Data Mining, KDD '12, pages 1059-1067, 2012. ISBN 978-1-4503-1462-6. URL doi.acm.org/10.1145/2339530.2339697. (Cited on p. 29.)

Related papers

Semantic Relations and Deep Learning
Stan Szpakowicz

ArXiv, 2020

The second edition of "Semantic Relations Between Nominals" (by Vivi Nastase, Stan Szpakowicz, Preslav Nakov and Diarmuid O Seaghdha) will be published by Morgan & Claypool. A new Chapter 5 of the book discusses relation classification/extraction in the deep-learning paradigm which arose after the first edition appeared. This is a preview of Chapter 5, made public by the kind permission of Morgan & Claypool.

View PDFchevron_right
Deep neural network-based relation extraction: an overview
Yusuf Zakari

Neural Computing and Applications, 2022

Knowledge is a formal way of understanding the world, providing a human-level cognition and intelligence for the next-generation artificial intelligence (AI). One of the representations of knowledge is semantic relations between entities. An effective way to automatically acquire this important knowledge, called Relation Extraction (RE), a sub-task of information extraction, plays a vital role in Natural Language Processing (NLP). Its purpose is to identify semantic relations between entities from natural language text. To date, there are several studies for RE in previous works, which have documented these techniques based on Deep Neural Networks (DNNs) become a prevailing technique in this research. Especially, the supervised and distant supervision methods based on DNNs are the most popular and reliable solutions for RE. This article 1) introduces some general concepts, and further 2) gives a comprehensive overview of DNNs in RE from two points of view: supervised RE, which attempts to improve the standard RE systems, and distant supervision RE, which adopts DNNs to design sentence encoder and de-noise method. We further 3) cover some novel methods and recent trends as well as discuss possible future research directions for this task.

View PDFchevron_right
Relation Extraction Using Deep Learning Methods-A Survey
Ajeesh Ramanujan

2019

Relation extraction has an important role in extracting structured information from unstructured raw text. This task is a crucial ingredient in numerous information extraction systems seeking to mine structured facts from text. Nowadays, neural networks play an important role in the task of relation extraction. The traditional non deep learning models require feature engineering. Deep Learning models such as Convolutional Neural Networks and Long Short Term Memory networks require less feature engineering than non-deep learning models. Relation Extraction has the potential of employing deep learning models with the creation of huge datasets using distant supervision. This paper surveys the current trend in Relation Extraction using Deep Learning models.

View PDFchevron_right
A Review of Relation Classification with Convolutional Neural Network
International Journal of Scientific Research in Science, Engineering and Technology IJSRSET

Relation classification is one of the important research issues in the field of Natural Language Processing (NLP). It is a crucial intermediate step in complex knowledge intensive applications like automatic knowledgebase construction, question answering, textual entailment, search engine etc. Recently neural network has given state of art results in various relation extraction tasks without depending much on manually engineered features. In this paper we present brief review on different model that has been proposed for relation classification and compare their results.

View PDFchevron_right
Neural relation extraction: a survey
Furkan Özbay

ArXiv, 2020

Neural relation extraction discovers semantic relations between entities from unstructured text using deep learning methods. In this study, we present a comprehensive review of methods on neural network based relation extraction. We discuss advantageous and incompetent sides of existing studies and investigate additional research directions and improvement ideas in this field.

View PDFchevron_right

Related topics

  • Computer Science
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved