Academia.eduAcademia.edu

Outline

Title
Abstract
FAQs
Introduction
Traditional Methods
Categories of Methods
Distant Supervision Re Data Sets
CNN-Based Methods
RNN or LSTM Based Methods
MIX-Structure Based Methods
Encoder-Based Methods
Representation-Based Methods
Knowledge-Based Methods
Plug-And-Play Based Methods
Discussion
DS Positive Dataset
Conclusions
References
All Topics
Psychology
Cognitive Science

Deep neural network-based relation extraction: an overview

Profile image of Yusuf ZakariYusuf Zakari

2022, Neural Computing and Applications

https://doi.org/10.1007/S00521-021-06667-3
visibility

…

description

32 pages

link

1 file

Abstract

Knowledge is a formal way of understanding the world, providing a human-level cognition and intelligence for the next-generation artificial intelligence (AI). One of the representations of knowledge is semantic relations between entities. An effective way to automatically acquire this important knowledge, called Relation Extraction (RE), a sub-task of information extraction, plays a vital role in Natural Language Processing (NLP). Its purpose is to identify semantic relations between entities from natural language text. To date, there are several studies for RE in previous works, which have documented these techniques based on Deep Neural Networks (DNNs) become a prevailing technique in this research. Especially, the supervised and distant supervision methods based on DNNs are the most popular and reliable solutions for RE. This article 1) introduces some general concepts, and further 2) gives a comprehensive overview of DNNs in RE from two points of view: supervised RE, which attempts to improve the standard RE systems, and distant supervision RE, which adopts DNNs to design sentence encoder and de-noise method. We further 3) cover some novel methods and recent trends as well as discuss possible future research directions for this task.

FAQs

sparkles

AI

What explains the importance of positional relations in DNN-based RE?add

The paper demonstrates that integrating position embeddings significantly improves sentence representation, enhancing information capture around entities. For instance, the position feature method achieves state-of-the-art performance in SemEval-2010-task8.

How have DNN-based methods evolved to address distant supervision challenges?add

Recent advancements in DNN-based RE leverage hybrid models like PCNN with multi-instance learning to mitigate noisy labels in distant supervision. These approaches have reported improvements in classification accuracy by refining sentence encoding and reducing data noise.

When were distant supervision methods first proposed in the context of RE?add

Distant supervision techniques were introduced in 2009 by Mintz et al. at the ACL conference, aiming to generate large-scale training datasets without manual annotation.

What are the main categories of DNN-based relation extraction methods?add

DNN-based relation extraction is categorized into supervised methods, distant supervision, and semi-supervised methods, each tailored to specific data availability and accuracy requirements in NLP tasks.

How do different neural network architectures affect relation extraction performance?add

The study indicates that varying architectures, such as CNNs or RNNs, provide distinct advantages; for instance, RNNs excel with long-distance dependencies, while CNNs are favored for capturing local structural features.

Related papers

Deep Neural Architectures for End-to-End Relation Extraction
Tung Khanh Tran

2020

View PDFchevron_right
A Review of Relation Classification with Convolutional Neural Network
International Journal of Scientific Research in Science, Engineering and Technology IJSRSET

Relation classification is one of the important research issues in the field of Natural Language Processing (NLP). It is a crucial intermediate step in complex knowledge intensive applications like automatic knowledgebase construction, question answering, textual entailment, search engine etc. Recently neural network has given state of art results in various relation extraction tasks without depending much on manually engineered features. In this paper we present brief review on different model that has been proposed for relation classification and compare their results.

View PDFchevron_right
An Enhanced Deep Neural Network-Based Architecture for Joint Extraction of Entity Mentions and Relations
mehdi fartash

INTERNATIONAL JOURNAL of FUZZY LOGIC and INTELLIGENT SYSTEMS

Named entity recognition and relation extraction are two principal tasks in most natural language processing systems. The majority of methods used in the field implement these two issues independently, thus leading to possible problems such as error propagation from one component (entity detection) to another (relation extraction). To solve such problems, we propose a new architecture for joint identification of entity mentions and their relation by employing a deep neural network framework. The model not only overcomes the error propagation challenge but also improves the detection results of both tasks owing to the cooperation with each other. Experiments on publicly available sources demonstrate that our joint model surpasses competitors in terms of accuracy. The results highlight the improvement achieved by the proposed deep neural network framework for the entity mention and relation classification tasks. Furthermore, we tested the effect of increasing the sentence length and demonstrated its negative influence on the performance.

View PDFchevron_right
MIT at SemEval-2017 Task 10: Relation Extraction with Convolutional Neural Networks
Peter Szolovits

Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)

Over 50 million scholarly articles have been published: they constitute a unique repository of knowledge. In particular, one may infer from them relations between scientific concepts, such as synonyms and hyponyms. Artificial neural networks have been recently explored for relation extraction. In this work, we continue this line of work and present a system based on a convolutional neural network to extract relations. Our model ranked first in the SemEval-2017 task 10 (ScienceIE) for relation extraction in scientific articles (subtask C).

View PDFchevron_right
Deep Learning for Semantic Relations
Stan Szpakowicz

2020

The second edition of "Semantic Relations Between Nominals" (by Vivi Nastase, Stan Szpakowicz, Preslav Nakov and Diarmuid \'O S\'eaghdha) will be published by Morgan & Claypool. A new Chapter 5 of the book discusses relation classification/extraction in the deep-learning paradigm which arose after the first edition appeared. This is a preview of Chapter 5, made public by the kind permission of Morgan & Claypool.

View PDFchevron_right
Improving Neural Relation Extraction with Implicit Mutual Relations
ullya rahmawati

2020 IEEE 36th International Conference on Data Engineering (ICDE)

Relation extraction (RE) aims at extracting the relation between two entities from the text corpora. It is a crucial task for Knowledge Graph (KG) construction. Most existing methods predict the relation between an entity pair by learning the relation from the training sentences, which contain the targeted entity pair. In contrast to existing distant supervision approaches that suffer from insufficient training corpora to extract relations, our proposal of mining implicit mutual relation from the massive unlabeled corpora transfers the semantic information of entity pairs into the RE model, which is more expressive and semantically plausible. After constructing an entity proximity graph based on the implicit mutual relations, we preserve the semantic relations of entity pairs via embedding each vertex of the graph into a low-dimensional space. As a result, we can easily and flexibly integrate the implicit mutual relations and other entity information, such as entity types, into the existing RE methods. Our experimental results on a New York Times and another Google Distant Supervision datasets suggest that our proposed neural RE framework provides a promising improvement for the RE task, and significantly outperforms the state-of-the-art methods. Moreover, the component for mining implicit mutual relations is so flexible that can help to improve the performance of both CNN-based and RNN-based RE models significant.

View PDFchevron_right
Simple Large-scale Relation Extraction from Unstructured Text
Arpit Mittal

ArXiv, 2018

Knowledge-based question answering relies on the availability of facts, the majority of which cannot be found in structured sources (e.g. Wikipedia info-boxes, Wikidata). One of the major components of extracting facts from unstructured text is Relation Extraction (RE). In this paper we propose a novel method for creating distant (weak) supervision labels for training a large-scale RE system. We also provide new evidence about the effectiveness of neural network approaches by decoupling the model architecture from the feature design of a state-of-the-art neural network system. Surprisingly, a much simpler classifier trained on similar features performs on par with the highly complex neural network system (at 75x reduction to the training time), suggesting that the features are a bigger contributor to the final performance.

View PDFchevron_right
State-ofthe-Art Models for Relation Extraction
Jens Albrecht

2021

The task of relation extraction aims at classifying the semantic relations between entities in a text. When coupled with named-entity recognition these can be used as the building blocks for an information extraction procedure that results in the construction of a Knowledge Graph. While many NLP libraries support named-entity recognition, there is no off-the-shelf solution for relation extraction. In this paper, we evaluate and compare several state-of-the-art approaches on a subset of the FewRel data set as well as a manually annotated corpus. The custom corpus contains six relations from the area of market research and is available for public use. Our approach provides guidance for the selection of models and training data for relation extraction in realworld projects.

View PDFchevron_right
Combining Long Short Term Memory and Convolutional Neural Network for Cross-Sentence n-ary Relation Extraction
Frans Coenen

ArXiv, 2019

We propose in this paper a combined model of Long Short Term Memory and Convolutional Neural Networks (LSTM-CNN) that exploits word embeddings and positional embeddings for cross-sentence n-ary relation extraction. The proposed model brings together the properties of both LSTMs and CNNs, to simultaneously exploit long-range sequential information and capture most informative features, essential for cross-sentence n-ary relation extraction. The LSTM-CNN model is evaluated on standard dataset on cross-sentence n-ary relation extraction, where it significantly outperforms baselines such as CNNs, LSTMs and also a combined CNN-LSTM model. The paper also shows that the LSTM-CNN model outperforms the current state-of-the-art methods on cross-sentence n-ary relation extraction.

View PDFchevron_right
A Review of Relation Extraction
强 王

Many applications in information extraction, natural language understanding, information retrieval require an understanding of the semantic relations between entities. We present a comprehensive review of various aspects of the entity relation extraction task. Some of the most important supervised and semi-supervised classification approaches to the relation extraction task are covered in sufficient detail along with critical analyses. We also discuss extensions to higher-order relations. Evaluation methodologies for both supervised and semi-supervised methods are described along with pointers to the commonly used performance evaluation datasets. Finally, we also give short descriptions of two important applications of relation extraction, namely question answering and biotext mining.

View PDFchevron_right

References (47)

  1. Kumar S (2017) A survey of deep learning methods for relation extraction. arXiv preprint arXiv:170503645
  2. Pawar S, Palshikar GK, Bhattacharyya P (2017) Relation extraction: A survey. arXiv preprint arXiv:171205191
  3. Hendrickx I, Kim SN, Kozareva Z, Nakov P, Ó Séaghdha D, Padó S, Pen- nacchiotti M, Romano L, Szpakowicz S (2009) Semeval-2010 task 8: Multi- ods in natural language processing (EMNLP), pp 1532-1543
  4. Zeng D, Liu K, Lai S, Zhou G, Zhao J, et al. (2014) Relation classification via convolutional deep neural network
  5. Nguyen TH, Grishman R (2015) Relation extraction: Perspective from convolutional neural networks. In: Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pp 39-48
  6. Santos CNd, Xiang B, Zhou B (2015) Classifying relations by ranking with convolutional neural networks. arXiv preprint arXiv:150406580
  7. Wang L, Cao Z, De Melo G, Liu Z (2016) Relation classification via multi- level attention cnns. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 1298-1307
  8. Zhang D, Wang D (2015) Relation classification via recurrent neural net- work. arXiv preprint arXiv:150801006
  9. Qin P, Xu W, Guo J (2017) Designing an adaptive attention mechanism for relation classification. In: 2017 International Joint Conference on Neural Networks (IJCNN), IEEE, pp 4356-4362
  10. Zhang C, Cui C, Gao S, Nie X, Xu W, Yang L, Xi X, Yin Y (2019) Multi-gram cnn-based self-attention model for relation classification. IEEE Access 7:5343-5357
  11. Ren F, Zhou D, Liu Z, Li Y, Zhao R, Liu Y, Liang X (2018) Neural relation classification with text descriptions. In: Proceedings of the 27th International Conference on Computational Linguistics, pp 1167-1177
  12. Zhang L, Xiang F (2018) Relation classification via bilstm-cnn. In: Inter- national Conference on Data Mining and Big Data, Springer, pp 373-382
  13. Mooney RJ, Bunescu RC (2006) Subsequence kernels for relation extrac- tion. In: Advances in neural information processing systems, pp 171-178
  14. Xu Y, Mou L, Li G, Chen Y, Peng H, Jin Z (2015) Classifying relations via long short term memory networks along shortest dependency paths. In: proceedings of the 2015 conference on empirical methods in natural language processing, pp 1785-1794
  15. Cai R, Zhang X, Wang H (2016) Bidirectional recurrent convolutional neu- ral network for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol 1, pp 756-765
  16. Guo X, Zhang H, Yang H, Xu L, Ye Z (2019) A single attention-based com- bination of cnn and rnn for relation classification. IEEE Access 7:12467- 12475
  17. Jin L, Song L, Zhang Y, Xu K, Ma Wy, Yu D (2020) Relation extraction exploiting full dependency forests. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 8034-8041
  18. Hearst MA (1992) Automatic acquisition of hyponyms from large text cor- pora. In: Proceedings of the 14th conference on Computational linguistics- Volume 2, Association for Computational Linguistics, pp 539-545
  19. Berland M, Charniak E (1999) Finding parts in very large corpora. In: Proceedings of the 37th annual meeting of the Association for Computa- 206-218
  20. Lin Y, Liu Z, Sun M (2017) Neural relation extraction with multi-lingual attention. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 34-43
  21. Lin Y, Shen S, Liu Z, Luan H, Sun M (2016) Neural relation extraction with selective attention over instances. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol 1, pp 2124-2133
  22. Banerjee S, Tsioutsiouliklis K (2018) Relation extraction using multi- encoder lstm network on a distant supervised dataset. In: 2018 IEEE 12th International Conference on Semantic Computing (ICSC), IEEE, pp 235- 238
  23. Du J, Han J, Way A, Wan D (2018) Multi-level structured self- attentions for distantly supervised relation extraction. arXiv preprint arXiv:180900699
  24. Ji G, Liu K, He S, Zhao J (2017) Distant supervision for relation extrac- tion with sentence-level attention and entity descriptions. In: Thirty-First AAAI Conference on Artificial Intelligence
  25. Wang G, Zhang W, Wang R, Zhou Y, Chen X, Zhang W, Zhu H, Chen H (2018) Label-free distant supervision for relation extraction via knowledge graph embedding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 2246-2255
  26. Vashishth S, Joshi R, Prayaga SS, Bhattacharyya C, Talukdar P (2018) Reside: Improving distantly-supervised neural relation extraction using side information. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 1257-1266
  27. Qin P, Xu W, Wang WY (2018) Robust distant supervision relation ex- traction via deep reinforcement learning. arXiv preprint arXiv:180509927
  28. Qin P, Xu W, Wang WY (2018) Dsgan: Generative adversarial training for distant supervision relation extraction. arXiv preprint arXiv:180509929
  29. Angeli G, Premkumar MJJ, Manning CD (2015) Leveraging linguistic structure for open domain information extraction. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), vol 1, pp 344-354
  30. Pavlick E, Rastogi P, Ganitkevitch J, Van Durme B, Callison-Burch C (2015) Ppdb 2.0: Better paraphrase ranking, fine-grained entailment re- lations, word embeddings, and style classification. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), vol 2, pp 425-430
  31. Zhang N, Deng S, Sun Z, Chen X, Zhang W, Chen H (2018) Attention- based capsule networks with dynamic routing for relation extraction. arXiv preprint arXiv:181211321
  32. Ye H, Chao W, Luo Z, Li Z (2016) Jointly extracting relations with class ties via effective deep ranking. arXiv preprint arXiv:161207602
  33. Miwa M, Bansal M (2016) End-to-end relation extraction using lstms on sequences and tree structures. arXiv preprint arXiv:160100770
  34. Li F, Zhang M, Fu G, Ji D (2017) A neural joint model for entity and relation extraction from biomedical text. BMC bioinformatics 18(1):198
  35. Zheng S, Wang F, Bao H, Hao Y, Zhou P, Xu B (2017) Joint extraction of entities and relations based on a novel tagging scheme. arXiv preprint arXiv:170605075
  36. Xiao Y, Tan C, Fan Z, Xu Q, Zhu W (2020) Joint entity and relation extraction with a hybrid transformer and reinforcement learning based model. In: AAAI, pp 9314-9321
  37. Bethard S, Carpuat M, Cer D, Jurgens D, Nakov P, Zesch T (2016) Proceedings of the 10th international workshop on semantic evaluation (semeval-2016). In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)
  38. Gábor K, Buscaldi D, Schumann AK, QasemiZadeh B, Zargayouna H, Charnois T (2018) Semeval-2018 task 7: Semantic relation extraction and classification in scientific papers. In: Proceedings of The 12th International Workshop on Semantic Evaluation, pp 679-688
  39. Zhang Y, Zhong V, Chen D, Angeli G, Manning CD (2017) Position- aware attention and supervised data improve slot filling. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Process- ing (EMNLP 2017), pp 35-45, URL https://nlp.stanford.edu/pubs/ zhang2017tacred.pdf
  40. Segura Bedmar I, Martinez P, Sánchez Cisneros D (2011) The 1st ddiextraction-2011 challenge task: Extraction of drug-drug interactions from biomedical texts
  41. Segura Bedmar I, Martínez P, Herrero Zazo M (2013) Semeval-2013 task 9: Extraction of drug-drug interactions from biomedical texts (ddiextraction 2013). Association for Computational Linguistics
  42. Di S, Shen Y, Chen L (2019) Relation extraction via domain-aware transfer learning. In: Proceedings of the 25th ACM SIGKDD International Con- ference on Knowledge Discovery & Data Mining, pp 1348-1357
  43. Sun C, Wu Y (2019) Distantly supervised entity relation extraction with adapted manual annotations. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 33, pp 7039-7046
  44. Zhang N, Deng S, Sun Z, Chen J, Zhang W, Chen H (2019) Transfer learn- ing for relation extraction via relation-gated adversarial learning. arXiv preprint arXiv:190808507
  45. Sahu SK, Christopoulou F, Miwa M, Ananiadou S (2019) Inter-sentence relation extraction with document-level graph convolutional neural net- work. arXiv preprint arXiv:190604684
  46. Guo Z, Zhang Y, Lu W (2019) Attention Guided Graph Convolutional Networks for Relation Extraction pp 241-251, DOI 10.18653/v1/p19-1024, 1906.07510
  47. Zhang Y, Qi P, Manning CD (2019) Graph Convolution over Pruned Dependency Trees Improves Relation Extraction (2005):2205-2215, DOI

Related papers

Relation Extraction Using Deep Learning Methods-A Survey
Ajeesh Ramanujan

2019

Relation extraction has an important role in extracting structured information from unstructured raw text. This task is a crucial ingredient in numerous information extraction systems seeking to mine structured facts from text. Nowadays, neural networks play an important role in the task of relation extraction. The traditional non deep learning models require feature engineering. Deep Learning models such as Convolutional Neural Networks and Long Short Term Memory networks require less feature engineering than non-deep learning models. Relation Extraction has the potential of employing deep learning models with the creation of huge datasets using distant supervision. This paper surveys the current trend in Relation Extraction using Deep Learning models.

View PDFchevron_right
Deep Neural Networks for Relation Extraction
Tapas Nayak

ArXiv, 2021

Relation extraction from text is an important task for automatic knowledge base population. In this thesis, we first propose a syntax-focused multi-factor attention network model for finding the relation between two entities. Next, we propose two joint entity and relation extraction frameworks based on encoder-decoder architecture. Finally, we propose a hierarchical entity graph convolutional network for relation extraction across documents.

View PDFchevron_right
Neural relation extraction: a survey
Furkan Özbay

ArXiv, 2020

Neural relation extraction discovers semantic relations between entities from unstructured text using deep learning methods. In this study, we present a comprehensive review of methods on neural network based relation extraction. We discuss advantageous and incompetent sides of existing studies and investigate additional research directions and improvement ideas in this field.

View PDFchevron_right
Neural relation extraction: a review
Furkan Özbay

Turkish J. Electr. Eng. Comput. Sci., 2021

Neural relation extraction discovers semantic relations between entities from unstructured text using deep learning methods. In this study, we make a clear categorization of the existing relation extraction methods in terms of data expressiveness and data supervision, and present a comprehensive and comparative review. We describe the evaluation methodologies and the datasets used for model assessment. We explicitly state the common challenges in relation extraction task and point out the potential of the pretrained models to solve them. Accordingly, we investigate additional research directions and improvement ideas in this field.

View PDFchevron_right
Comparative Analysis of Contextual Relation Extraction based on Deep Learning Models
Priyadharshini Ravichandran

An International Journal of Engineering Science, 2023

Contextual Relation Extraction (CRE) is mainly used for constructing a knowledge graph with a help of ontology. It performs various tasks such as semantic search, query answering, and textual entailment. Relation extraction identifies the entities from raw texts and the relations among them. An efficient and accurate CRE system is essential for creating domain knowledge in the biomedical industry. Existing Machine Learning and Natural Language Processing (NLP) techniques are not suitable to predict complex relations from sentences that consist of more than two relations and unspecified entities efficiently. In this work, deep learning techniques have been used to identify the appropriate semantic relation based on the context from multiple sentences. Even though various machine learning models have been used for relation extraction, they provide better results only for binary relations, i.e., relations occurred exactly between the two entities in a sentence. Machine learning models are not suited for complex sentences that consist of the words that have various meanings. To address these issues, hybrid deep learning models have been used to extract the relations from complex sentence effectively. This paper explores the analysis of various deep learning models that are used for relation extraction.

View PDFchevron_right

Related topics

  • Cognitive Science
  • Computer Science
  • Artificial Intelligence
  • computer science Artificial inte...
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved