The field matching problem: Algorithms and applications

IEEE Transactions on Knowledge and Data Engineering, 2017

Schema Matching (SM) and Record Matching (RM) are two necessary steps in integrating multiple relational tables of different schemas, where SM unifies the schemas and RM detects records referring to the same real-world entity. The two processes have been thoroughly studied separately, but few attention has been paid to the interaction of SM and RM. In this work, we find that, even alternating them in a simple manner, SM and RM can benefit from each other to reach a better integration performance (i.e., in terms of precision and recall). Therefore, combining SM and RM is a promising solution for improving data integration. To this end, we define novel matching rules for SM and RM, respectively, that is, every SM decision is made based on intermediate RM results, and vice versa, such that SM and RM can be performed alternately. The quality of integration is guaranteed by a Matching Likelihood Estimation model and the control of semantic drift, which prevent the effect of mismatch magnification. To reduce the computational cost, we design an index structure based on q-grams and a greedy search algorithm that can reduce around 90 percent overhead of the interaction. Extensive experiments on three data collections show that the combination and interaction between SM and RM significantly outperforms previous works that conduct SM and RM separately.

Management Science, 2007

B ecause of the heterogeneous nature of different data sources, data integration is often one of the most challenging tasks in managing modern information systems. While the existing literature has focused on problems such as schema integration and entity identification, it has largely overlooked a basic question: When an attribute value for a real-world entity is recorded differently in different databases, how should the "best" value be chosen from the set of possible values? This paper provides an answer to this question. We first show how a probability distribution over a set of possible values can be derived. We then demonstrate how these probabilities can be used to solve a given decision problem by minimizing the total cost of type I, type II, and misrepresentation errors. Finally, we propose a framework for integrating multiple data sources when a single "best" value has to be chosen and stored for every attribute of an entity.

ICIS 2004 Proceedings, 2004

Because of the heterogeneous nature of multiple data sources, data integration is often one of the most challenging tasks of today's information systems. While the existing literature has focused on problems such as schema integration and entity identification, our current study attempts to answer a basic question: When an attribute value for a real-world entity is recorded differently in two databases, how should the "best" value be chosen from the set of possible values? We first show how probabilities for attribute values can be derived, and then propose a framework for deciding the cost-minimizing value based on the total cost of type I, type II, and misrepresentation errors.

Matching operation takes as input ontologies, each consisting of a set of discrete entities (e.g., tables, XML elements, classes, properties) and determines as output the relationships (e.g., equivalence, subsumption) holding between these entities 5 Semantic Web Technololgy Show Case at ESTC'07, Vienna, Austria Example: two XML schemas Equivalence Generality Disjointness 6 Semantic Web Technololgy Show Case at ESTC'07, Vienna, Austria Example: two ontologies Equivalence Generality Disjointness .year = 7 Semantic Web Technololgy Show Case at ESTC'07, Vienna, Austria Statement of the problem Scope Reducing heterogeneity can be performed in two steps: Match, thereby determine the alignment Process the alignment (merge, transform, translate...) 8 Semantic Web Technololgy Show Case at ESTC'07, Vienna, Austria Statement of the problem Correspondence is a 5-tuple <id, e1, e2, R, n> id is a unique identifier of the given correspondence e1 and e2 are entities (XML elements, classes,...) R is a relation (equivalence, more general, disjointness,...) n is a confidence measure, typically in the [0,1] range Alignment (A) is a set of correspondences with some cardinality: 1-1, 1-n, ... some other properties (complete) 9 Semantic Web Technololgy Show Case at ESTC'07, Vienna, Austria Statement of the problem Alignment A p (weights, .. ) r (WordNet, …) Matching Alignment A'

2015

Matching and merging data from conflicting sources is the bread and butter of data integration, which drives search verticals, e-commerce comparison sites and cyber intelligence. Schema matching lifts data integration—traditionally focused on well-structured data—to highly heterogeneous sources. While schema matching has enjoyed significant success in matching data attributes, inconsistencies can exist at a deeper level, making full integration difficult or impossible. We propose a more fine-grained approach that focuses on correspondences between the values of attributes across data sources. Since the semantics of attribute values derive from their use and co-occurrence, we argue for the suitability of canonical correlation analysis (CCA) and its variants. We demonstrate the superior statistical and computational performance of multiple sparse CCA compared to a suite of baseline algorithms, on two datasets which we are releasing to stimulate further research. Our crowd-annotated da...

1997

Determining attribute correspondences is a difficult, time-consuming, knowledgeintensive part of database integration. We report on experiences with tools that identified candidate correspondences, as a step in a large scale effort to improve communication among Air Force systems. First, we describe a new method that was both simple and surprisingly successful: Data dictionary and catalog information were dumped to unformatted text; then off-the-shelf information retrieval software estimated string similarity, generated candidate matches, and provided the interface. The second method used a different set of clues, such as statistics on database populations, to compute separate similarity metrics (using neural network techniques). We report on substantial use of the first tool, and then report some limited initial experiments that examine the two techniques' accuracy, consistency and complementarity.

* The theory and the mathematical modelling presented in this report has not been published elsewhere. However, different applications of the theory are under review.

IEEE Expert / IEEE Intelligent Systems, 2003

I n f o r m a t i o n I n t e g r a t i o n o n t h e W e b (s, t, i, j), which is the edit distance between the first i letters in s and the first j letters in t. Let s i denote the ith letter of s, and, similarly, let t j be the jth letter of t. It's easy to see that we can define D(s, t, i, j) recursively, where D(s, t, 0, 0) = 0 and Identifying approximately duplicate database records that refer to the same entity is essential for information integration. The authors compare and describe methods for combining and learning similarity measures for name matching.

* The theory and the mathematical modelling presented in this report has not been published elsewhere. However, different applications of the theory are under review.

Lecture Notes in Computer Science, 2005

Schema and ontology matching is a critical problem in many application domains, such as semantic web, schema/ontology integration, data warehouses, e-commerce, etc. Many different matching solutions have been proposed so far. In this paper we present a new classification of schema-based matching techniques that builds on the top of state of the art in both schema and ontology matching. Some innovations are in introducing new criteria which are based on (i) general properties of matching techniques, (ii) interpretation of input information, and (iii) the kind of input information. In particular, we distinguish between approximate and exact techniques at schema-level; and syntactic, semantic, and external techniques at element-and structure-level. Based on the classification proposed we overview some of the recent schema/ontology matching systems pointing which part of the solution space they cover. The proposed classification provides a common conceptual basis, and, hence, can be used for comparing different existing schema/ontology matching techniques and systems as well as for designing new ones, taking advantages of state of the art solutions. For more information on the topic (e.g., tutorials, relevant events), please visit the Ontology Matching web-site at www.OntologyMatching.org