Structure embedding for knowledge base completion and analytics

arXiv (Cornell University), 2022

Knowledge graph embedding (KGE) is a increasingly popular technique that aims to represent entities and relations of knowledge graphs into low-dimensional semantic spaces for a wide spectrum of applications such as link prediction, knowledge reasoning and knowledge completion. In this paper, we provide a systematic review of existing KGE techniques based on representation spaces. Particularly, we build a fine-grained classification to categorise the models based on three mathematical perspectives of the representation spaces: (1) Algebraic perspective, (2) Geometric perspective, and (3) Analytical perspective. We introduce the rigorous definitions of fundamental mathematical spaces before diving into KGE models and their mathematical properties. We further discuss different KGE methods over the three categories, as well as summarise how spatial advantages work over different embedding needs. By collating the experimental results from downstream tasks, we also explore the advantages of mathematical space in different scenarios and the reasons behind them. We further state some promising research directions from a representation space perspective, with which we hope to inspire researchers to design their KGE models as well as their related applications with more consideration of their mathematical space properties.

Twenty First Pacific Asia Conference on Information Systems (PACIS), 2017

Knowledge Graphs capture entities and their relationships. However, many knowledge graphs are afflicted by missing data. Recently, embedding methods have been used to alleviate this issue via knowledge graph completion. However, most existing methods only consider the relationship in triples, even though contextual relation types, consisting of the surrounding relation types of a triple, can substantially improve prediction accuracy. Therefore, we propose a contextual embedding method that learns the embeddings of entities and predicates while taking contextual relation types into account. The main benefits of our approach are: (1) improved scalability via a reduced number of epochs needed to achieve comparable or better results with the same memory complexity, (2) higher prediction accuracy (an average of 14%) compared to the related algorithms, and (3) high accuracy for both missing entity and predicate predictions. The source code and the YAGO43k dataset of this paper can be found from (https://github.ncsu.edu/cmoon2/kg).

Neural Computing and Applications, 2018

Large-scale knowledge graphs have currently reached impressive sizes; however, they are still far from complete. In addition, most existing methods for knowledge graph completion only consider the direct links between entities, ignoring the vital impact of the semantics of relation paths. In this paper, we study the problem of how to better embed entities and relations of knowledge graphs into different low-dimensional spaces by taking full advantage of the additional semantics of relation paths and propose a novel relation path embedding model named as RPE. Specifically, with the corresponding relation and path projections, RPE can simultaneously embed each entity into two types of latent spaces. Moreover, type constraints are extended from traditional relation-specific type constraints to the proposed path-specific type constraints and both of the two type constraints can be seamlessly incorporated into RPE. The proposed model is evaluated on the benchmark tasks of link prediction and triple classification. The results of experiments demonstrate our method outperforms all baselines on both tasks. They indicate that our model is capable of catching the semantics of relation paths, which is significant for knowledge representation learning.

Journal of Computer and Knowledge Engineering, 2022

Representation learning on a knowledge graph aims to capture patterns in the knowledge graph as lowdimensional dense distributed representation vectors in the continuous semantic space, which is a powerful technique for predicting missing links in knowledge bases. The problem of knowledge base completion can be viewed as predicting new triples based on the existing ones. One of the prominent approaches in knowledge base completion is the embedding model. Currently, the majority of existing knowledge graph embedding models cannot deal with unbalanced entities and relations. In this paper, a new embedding model is proposed, with a general solution instead of using the additional corpus. First, a triple-based neural network is presented to maximize the likelihood of the knowledge bases finding a low-dimensional embedding space. Second, two procedures to generate positive triples are proposed. They produce positive triples and add them to the training data. The policies can capture rare triples, and simultaneously remain efficient to compute. Experiments show that the embedded model proposed in this paper has superior performance.

Indian Journal of Science and Technology, 2021

Objective: The main objective of this research is constructing a knowledge graph using a superior model that leverage the knowledge through relational mapping properties and the matrix constructed dynamically based on the changes in the relational mapping. Methods: This research work proposes symbolic objects in the knowledge graph that are signified by two vectors, the first vector denotes the entity/relation and the vector denote the matrix which is constructed with the help of mapping properties. The proposed methodology uses only minimum parameters and avoids multiplication processes which make the proposed scheme effective in the large scale graphs. The matrix representation with the assistance of mapping properties reduces the computational complication and the time requires to attain the knowledge is also minimized. Findings: The empirical research on knowledge graph mining gives significant insights and it is applied on benchmark datasets namely FB15K-237 and WN18RR. The performance are evaluated in terms of precision and recall in percentage.

IEICE Transactions on Information and Systems, 2020

Knowledge graph embedding aims to embed entities and relations of multi-relational data in low dimensional vector spaces. Knowledge graphs are useful for numerous artificial intelligence (AI) applications. However, they (KGs) are far from completeness and hence KG embedding models have quickly gained massive attention. Nevertheless, the state-of-the-art KG embedding models ignore the category specific projection of entities and the impact of entity types in relational aspect. For example, the entity "Washington" could belong to the person or location category depending on its appearance in a specific relation. In a KG, an entity usually holds many type properties. It leads us to a very interesting question: are all the type properties of an entity are meaningful for a specific relation? In this paper, we propose a KG embedding model TPRC that leverages entity-type properties in the relational context. To show the effectiveness of our model, we apply our idea to the TransE, TransR and TransD. Our approach outperforms other state-of-the-art approaches as TransE, TransD, DistMult and ComplEx. Another, important observation is: introducing entity type properties in the relational context can improve the performances of the original translation distance based models.

2017

Knowledge Graphs capture entities and their relationships. However, many knowledge graphs are afflicted by missing data. Recently, embedding methods have been used to alleviate this issue via knowledge graph completion. However, most existing methods only consider the relationship in triples, even though contextual relation types, consisting of the surrounding relation types of a triple, can substantially improve prediction accuracy. Therefore, we propose a contextual embedding method that learns the embeddings of entities and predicates while taking contextual relation types into account. The main benefits of our approach are: (1) improved scalability via a reduced number of epochs needed to achieve comparable or better results with the same memory complexity, (2) higher prediction accuracy (an average of 14%) compared to the related algorithms, and (3) high accuracy for both missing entity and predicate predictions. The source code and the YAGO43k dataset of this paper can be found from ().

ArXiv, 2016

Nowadays, large-scale knowledge bases containing billions of facts have reached impressive sizes; however, they are still far from completion. In addition, most existing methods only consider the direct links between entities, ignoring the vital impact about the semantic of relation paths. In this paper, we study the problem of how to better embed entities and relations into different low dimensional spaces. A compositional learning model of relation paths embedding (RPE) is proposed to take full advantage of additional semantics expressed by relation paths. More specifically, using corresponding projection matrices, RPE can simultaneously embed entities into corresponding relation and path spaces. It is also suggested that type constraints could be extended from traditional relation-specific to the new proposed path-specific ones. Both of the two type constraints can be seamlessly incorporated into RPE and decrease the errors in prediction. Experiments are conducted on the benchmar...

2019

In this paper we study techniques to improve the performance of bilinear embedding methods for knowledge graph completion on large datasets, where at each epoch the model sees a very small percentage of the training data, and the number of generated negative examples for each positive example is limited to a small portion of the entire set of entities. We first present a heuristic method to infer the types and type constraints of entities and relations. We then use this method to construct both a joint learning model, and a straightforward method for increasing the quality of sampled negatives during training. We show that when these two techniques are combined, they give an improvement in performance of up to 5.6% for Hits@1. We find the improvement is especially significant when the batch size and the number of generated negative examples are low relative to the size of the dataset.

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Distance-based knowledge graph embeddings have shown substantial improvement on the knowledge graph link prediction task, from TransE to the latest state-of-the-art RotatE. However, complex relations such as N-to-1, 1to-N and N-toN still remain challenging to predict. In this work, we propose a novel distance-based approach for knowledge graph link prediction. First we extend the RotatE from 2D complex domain to high dimensional space with orthogonal transforms to model relations. The orthogonal transform embedding for relations keeps the capability for modeling symmetric/anti-symmetric, inverse and compositional relations while achieves better modeling capacity. Second, the graph context is integrated into distance scoring functions directly. Specifically, graph context is explicitly modeled via two directed context representations. Each node embedding in knowledge graph is augmented with two context representations, which are computed from the neighboring outgoing and incoming nodes/edges respectively. The proposed approach improves prediction accuracy on the difficult N-to-1, 1to-N and N-toN cases. Our experimental results show that it achieves state-of-the-art results on two common benchmarks FB15k-237 and WNRR-18, especially on FB15k-237 which has many high in-degree nodes.