Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Methods Configurations and Results
Results
Multiple Data Sources
Related Work
Conclusion and Future Work
References
All Topics
Computer Science
Information Systems

Scaling entity resolution: A loosely schema-aware approach

Profile image of Sonia BERGAMASCHISonia BERGAMASCHI

2019, Information Systems

https://doi.org/10.1016/J.IS.2019.03.006
visibility

…

description

50 pages

link

1 file

Abstract

In big data sources, real-world entities are typically represented with a variety of schemata and formats (e.g., relational records, JSON objects, etc.). Different profiles (i.e., representations) of an entity often contain redundant and/or inconsistent information. Thus identifying which profiles refer to the same entity is a fundamental task (called Entity Resolution) to unleash the value of big data. The naïve all-pairs comparison solution is impractical on large data, hence blocking methods are employed to partition a profile collection into (possibly overlapping) blocks and limit the comparisons to profiles that appear in the same block together. Meta-blocking is the task of restructuring a block collection, removing superfluous comparisons. Existing meta-blocking approaches rely exclusively on schema-agnostic features, under the assumption that handling the schema variety of big data does not pay-off for such a task. In this paper, we demonstrate how "loose" schema information (i.e., statistics collected directly from the data) can be exploited to enhance the quality of the blocks in a holistic loosely schema-aware (meta-)blocking approach that can be used to speed up your favorite Entity Resolution algorithm. We call it Blast (Blocking with Loosely-Aware Schema Techniques). We show how Blast can automatically extract the loose schema information by adopting an LSH-based step for efficiently handling volume and schema heterogeneity of the data. Furthermore, we introduce a novel meta-blocking algorithm that can be employed to efficiently execute Blast on MapReduce-like systems (such as Apache Spark). Finally, we experimentally demonstrate, on real-world datasets, how Blast outperforms the state-of-the-art (meta-)blocking approaches.

Related papers

Noisy-aware Blocking over Heterogeneous Data
Kostas Stefanidis

2018

Entity resolution (ER) emerges as a fundamental step to integrate multiple knowledge bases or identify similarities between entities. Blocking is widely applied as an initial step of ER to avoid computing similarities between all pairs of entities. Heterogeneous and noisy data increase the difficulties faced by blocking, since these issues directly interfere the block generation. In this work, we propose a technique capable to tolerate noisy data to extract information regarding the data schema, and generate high-quality blocks. We apply Locality Sensitive Hashing (LSH) to hash the entities values, and enable the generation of high-quality blocks, even with the presence of noise in the values. In the experiments, we highlight that our approach has better effectiveness compared to the state-of-the-art technique, as well as less produced

View PDFchevron_right
Incremental Blocking for Entity Resolution over Web Streaming Data
Tiago Brasileiro

IEEE/WIC/ACM International Conference on Web Intelligence

The widespread use of information systems has become a valuable source of semi-structured data. In this context, Entity Resolution (ER) emerges as a fundamental task to integrate multiple knowledge bases or identify similarities between data items (i.e., entities). Since ER is an inherently quadratic task, blocking techniques are often used to improve efficiency. Beyond the challenges related to the data volume and heterogeneity, blocking techniques also face two other challenges: streaming data and incremental processing. To address these challenges, we propose PRIME, a novel incremental schema-agnostic blocking technique that utilizes parallelism to enhance blocking efficiency. The proposed technique deals with streaming and incremental data using a distributed computational infrastructure. To improve efficiency, the technique avoids unnecessary comparisons and applies a time window strategy to prevent excessive memory consumption.

View PDFchevron_right
Exploiting block co-occurrence to control block sizes for entity resolution
Demetrio Mestre

Knowledge and Information Systems, 2019

The problem of identifying duplicated entities in a dataset has gained increasing importance during the last decades. Due to the large size of the datasets, this problem can be very costly to be solved due to its intrinsic quadratic complexity. Both researchers and practitioners have developed a variety of techniques aiming to speed up a solution to this problem. One of these techniques is called blocking, an indexing technique that splits the dataset into a set of blocks, such that each block contains entities that share a common property evaluated by a blocking key function. In order to improve the efficacy of the blocking technique, multiple blocking keys may be used, and thus, a set of blocking results is generated. In this paper, we investigate how to control the size of the blocks generated by the use of multiple blocking keys and maintain reasonable quality results, which is measured by the quality of the produced blocks. By controlling the size of the blocks, we can reduce the overall cost of solving an entity resolution problem and facilitate the execution of a variety of tasks (e.g., real-time and privacy-preserving entity resolution). For doing so, we propose many heuristics which exploit the co-occurrence of entities among the generated blocks for pruning, splitting and merging blocks. The experimental results we carry out using four datasets confirm the adequacy of the proposed heuristics for generating block sizes within a predefined range threshold as well as maintaining reasonable blocking quality results.

View PDFchevron_right
A noise tolerant and schema-agnostic blocking technique for entity resolution
Kostas Stefanidis

Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, 2019

The increasing use of Web systems has become a valuable source of semi-structured data. In this context, the Entity Resolution (ER) task emerges as a fundamental step to integrate multiple knowledge bases or identify similarities between the data items (i.e., entities). Usually, blocking techniques are widely applied as an initial step of ER approaches in order to avoid computing similarities between all pairs of entities (quadratic cost). In practice, heterogeneous and noisy data increase the difficulties faced by blocking techniques, since these issues directly interfere the block generation. To address these challenges, we propose the NA-BLOCKER technique, which is capable of tolerating noisy data to extract information regarding the data schema and generate high-quality blocks. NA-BLOCKER applies Locality Sensitive Hashing (LSH) to hash the attribute values of entities and enable the generation of high-quality blocks, even with the presence of noise in the attribute values. In our experimental evaluation, we use five real-world datasets, and highlight that NA-BLOCKER presents better results regarding effectiveness compared to the state-of-the-art technique. In terms of efficiency, NA-BLOCKER produces, on average, 34% less comparisons. However, due to the cost introduced by LSH, it results in an increase of the execution time at around 30%, on average.

View PDFchevron_right
MFIBlocks: An effective blocking algorithm for entity resolution
Batya Kenig

Information Systems, 2013

Entity resolution is the process of discovering groups of tuples that correspond to the same real-world entity. Blocking algorithms separate tuples into blocks that are likely to contain matching pairs. Tuning is a major challenge in the blocking process and in particular, high expertise is needed in contemporary blocking algorithms to construct a blocking key, based on which tuples are assigned to blocks. In this work, we introduce a blocking approach that avoids selecting a blocking key altogether, relieving the user from this difficult task. The approach is based on maximal frequent itemsets selection, allowing early evaluation of block quality based on the overall commonality of its members. A unique feature of the proposed algorithm is the use of prior knowledge of the estimated size of duplicate sets in enhancing the blocking accuracy. We report on a thorough empirical analysis, using common benchmarks of both real-world and synthetic datasets to exhibit the effectiveness and efficiency of our approach.

View PDFchevron_right
Generalized Supervised Meta-blocking (technical report)
Sonia BERGAMASCHI

arXiv (Cornell University), 2022

Entity Resolution constitutes a core data integration task that relies on Blocking in order to tame its quadratic time complexity. Schemaagnostic blocking achieves very high recall, requires no domain knowledge and applies to data of any structuredness and schema heterogeneity. This comes at the cost of many irrelevant candidate pairs (i.e., comparisons), which can be significantly reduced through Meta-blocking techniques, i.e., techniques that leverage the co-occurrence patterns of entities inside the blocks: first, a weighting scheme assigns a score to every pair of candidate entities in proportion to the likelihood that they are matching and then, a pruning algorithm discards the pairs with the lowest scores. Supervised Meta-blocking goes beyond this approach by combining multiple scores per comparison into a feature vector that is fed to a binary classifier. By using probabilistic classifiers, Generalized Supervised Meta-blocking associates every pair of candidates with a score that can be used by any pruning algorithm. For higher effectiveness, new weighting schemes are examined as features. Through an extensive experimental analysis, we identify the best pruning algorithms, their optimal sets of features as well as the minimum possible size of the training set. The resulting approaches achieve excellent performance across several established benchmark datasets.

View PDFchevron_right
Entity resolution with iterative blocking
Georgia Koutrika

Proceedings of the 35th SIGMOD international conference on Management of data - SIGMOD '09, 2009

Entity Resolution (ER) is the problem of identifying which records in a database refer to the same real-world entity. An exhaustive ER process involves computing the similarities between pairs of records, which can be very expensive for large datasets. Various blocking techniques can be used to enhance the performance of ER by dividing the records into blocks in multiple ways and only comparing records within the same block. However, most blocking techniques process blocks separately and do not exploit the results of other ...

View PDFchevron_right
Web-Scale Blocking, Iterative and Progressive Entity Resolution
Vassilis Christophides

2017 IEEE 33rd International Conference on Data Engineering (ICDE), 2017

Entity resolution (ER) aims to identify descriptions of the same entity within or across knowledge bases. In this tutorial, we provide a comprehensive and cohesive overview of the key research results in the area of ER. We are interested in frameworks addressing the new challenges in ER posed by the Web of data in which real world entities are described by interlinked data rather than documents. Since such descriptions are usually partial, overlapping and sometimes evolving, ER emerges as a central problem both to increase dataset linking, but also to search the Web of data for entities and their relations. We focus on Web-scale blocking, iterative and progressive solutions for ER. Specifically, to reduce the required number of comparisons, blocking is performed to place similar descriptions into blocks and executes comparisons to identify matches only between descriptions within the same block. To minimize the number of missed matches, an iterative ER process can exploit any intermediate results of blocking and matching, discovering new candidate description pairs for resolution. Finally, we overview works on progressive ER, which attempt to discover as many matches as possible given limited computing budget, by estimating the matching likelihood of yet unresolved descriptions, based on the matches found so far.

View PDFchevron_right
A framework for entity resolution with efficient blocking
Liangcai Shu

2012

Abstract In applications of Web data integration, we frequently need to identify whether data objects in different data sources represent the same entity in the real world. This problem is known as entity resolution. In this paper, we propose a generic framework for entity resolution for relational data sets, called BARM, consisting of the Blocker, Attribute matchers and the Record Matcher. BARM is convenient for different blocking and matching algorithms to fit into it.

View PDFchevron_right
Benchmarking Blocking Algorithms for Web Entities
Vassilis Christophides

IEEE Transactions on Big Data, 2016

An increasing number of entities are described by interlinked data rather than documents on the Web. Entity Resolution (ER) aims to identify descriptions of the same real-world entity within one or across knowledge bases in the Web of data. To reduce the required number of pairwise comparisons among descriptions, ER methods typically perform a pre-processing step, called blocking, which places similar entity descriptions into blocks and thus only compare descriptions within the same block. We experimentally evaluate several blocking methods proposed for the Web of data using real datasets, whose characteristics significantly impact their effectiveness and efficiency. The proposed experimental evaluation framework allows us to better understand the characteristics of the missed matching entity descriptions and contrast them with ground truth obtained from different kinds of relatedness links.

View PDFchevron_right

References (61)

  1. P. Christen, A survey of indexing techniques for scalable record linkage and deduplication, IEEE transactions on knowledge and data engineering 24 (9) (2012) 1537-1555.
  2. X. L. Dong, D. Srivastava, Big data integration, Synthesis Lectures on Data Management 7 (1) (2015) 1-198.
  3. S. Bergamaschi, D. Beneventano, F. Mandreoli, R. Martoglia, F. Guerra, M. Orsini, L. Po, M. Vincini, G. Simonini, S. Zhu, et al., From data in- tegration to big data integration, in: A Comprehensive Guide Through the Italian Database Research Over the Last 25 Years, Springer, 2018, pp. 43-59.
  4. R. Baxter, P. Christen, T. Churches, et al., A comparison of fast blocking methods for record linkage, in: ACM SIGKDD, Vol. 3, Citeseer, 2003, pp. 25-27.
  5. P. Konda, S. Das, P. Suganthan GC, A. Doan, A. Ardalan, J. R. Bal- lard, H. Li, F. Panahi, H. Zhang, J. Naughton, et al., Magellan: Toward building entity matching management systems, Proceedings of the VLDB Endowment 9 (12) (2016) 1197-1208.
  6. J. Madhavan, S. R. Jeffery, S. Cohen, X. Dong, D. Ko, C. Yu, A. Halevy, Web-scale data integration: You can only afford to pay as you go, in: Proceedings of CIDR, 2007, pp. 342-350.
  7. G. Papadakis, E. Ioannou, T. Palpanas, C. Niederee, W. Nejdl, A blocking framework for entity resolution in highly heterogeneous information spaces, IEEE Transactions on Knowledge and Data Engineering 25 (12) (2013) 2665-2682.
  8. G. Papadakis, G. Koutrika, T. Palpanas, W. Nejdl, Meta-blocking: Taking entity resolution to the next level, IEEE Transactions on Knowledge and Data Engineering 26 (8) (2014) 1946-1960.
  9. Y. Ma, T. Tran, Typimatch: Type-specific unsupervised learning of keys and key values for heterogeneous web data integration, in: Proceedings of the sixth ACM international conference on Web search and data mining, ACM, 2013, pp. 325-334.
  10. C. E. Shannon, A mathematical theory of communication, SIGMOBILE Mob. Comput. Commun. Rev. 5 (1) (2001) 3-55. doi:10.1145/584091. 584093.
  11. G. Simonini, S. Bergamaschi, H. Jagadish, Blast: a loosely schema-aware meta-blocking approach for entity resolution, Proceedings of the VLDB Endowment 9 (12) (2016) 1173-1184.
  12. G. Papadakis, G. Papastefanatos, T. Palpanas, M. Koubarakis, Scaling en- tity resolution to large, heterogeneous data with enhanced meta-blocking., in: EDBT, 2016, pp. 221-232.
  13. V. Efthymiou, G. Papadakis, G. Papastefanatos, K. Stefanidis, T. Pal- panas, Parallel meta-blocking for scaling entity resolution over big hetero- geneous data, Information Systems 65 (2017) 137-157.
  14. P. Christen, Data Matching -Concepts and Techniques for Record Link- age, Entity Resolution, and Duplicate Detection, Data-Centric Systems and Applications, Springer, 2012. doi:10.1007/978-3-642-31164-2.
  15. V. Christophides, V. Efthymiou, K. Stefanidis, Entity resolution in the web of data, Synthesis Lectures on the Semantic Web 5 (3) (2015) 1-122.
  16. G. Simonini, G. Papadakis, T. Palpanas, S. Bergamaschi, Schema-agnostic progressive entity resolution, IEEE Trans. Knowl. Data Eng. (2018)doi: 10.1109/TKDE.2018.2852763.
  17. P. Shvaiko, J. Euzenat, Ontology matching: state of the art and future challenges, IEEE Transactions on knowledge and data engineering 25 (1) (2013) 158-176.
  18. T. Ranbaduge, D. Vatsalan, P. Christen, A scalable and efficient subgroup blocking scheme for multidatabase record linkage, in: Pacific-Asia Confer- ence on Knowledge Discovery and Data Mining, Springer, 2018, pp. 15-27.
  19. A. Z. Broder, On the resemblance and containment of documents, in: Com- pression and complexity of sequences 1997. proceedings, IEEE, 1997, pp. 21-29.
  20. J. Leskovec, A. Rajaraman, J. D. Ullman, Mining of massive datasets, Cambridge university press, 2014.
  21. T. M. Cover, J. A. Thomas, Elements of information theory, John Wiley & Sons, 2012.
  22. T. De Vries, H. Ke, S. Chawla, P. Christen, Robust record linkage block- ing using suffix arrays, in: Proceedings of the 18th ACM conference on Information and knowledge management, ACM, 2009, pp. 305-314.
  23. A. Agresti, M. Kateri, Categorical data analysis, in: International encyclo- pedia of statistical science, Springer, 2011, pp. 206-208.
  24. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauly, M. J. Franklin, S. Shenker, I. Stoica, Resilient distributed datasets: A fault- tolerant abstraction for in-memory cluster computing, in: Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12), USENIX, San Jose, CA, 2012, pp. 15-28.
  25. J. Dean, S. Ghemawat, Mapreduce: simplified data processing on large clusters, Communications of the ACM 51 (1) (2008) 107-113.
  26. URL https://spark.apache.org/docs/2.1.0/programming-guide. html#shuffle-operations
  27. A. S. Das, M. Datar, A. Garg, S. Rajaram, Google news personalization: scalable online collaborative filtering, in: Proceedings of the 16th interna- tional conference on World Wide Web, ACM, 2007, pp. 271-280.
  28. S. Blanas, J. M. Patel, V. Ercegovac, J. Rao, E. J. Shekita, Y. Tian, A com- parison of join algorithms for log processing in mapreduce, in: Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, ACM, 2010, pp. 975-986.
  29. H. Köpcke, A. Thor, E. Rahm, Evaluation of entity resolution approaches on real-world match problems, Proceedings of the VLDB Endowment 3 (1- 2) (2010) 484-493.
  30. S. Das, A. Doan, P. S. G. C., C. Gokhale, P. Konda, The mag- ellan data repository, https://sites.google.com/site/anhaidgroup/ projects/data.
  31. A. Harth, Billion triples challenge data set (2012).
  32. D. Hand, P. Christen, A note on using the f-measure for evaluating record linkage algorithms, Statistics and Computing 28 (3) (2018) 539-547.
  33. M. Ebraheem, S. Thirumuruganathan, S. Joty, M. Ouzzani, N. Tang, Dis- tributed representations of tuples for entity resolution, Proceedings of the VLDB Endowment 11 (11) (2018) 1454-1467.
  34. S. Mudgal, H. Li, T. Rekatsinas, A. Doan, Y. Park, G. Krishnan, R. Deep, E. Arcaute, V. Raghavendra, Deep learning for entity matching: A design space exploration, in: Proceedings of the 2018 International Conference on Management of Data, ACM, 2018, pp. 19-34.
  35. O. Benjelloun, H. Garcia-Molina, D. Menestrina, Q. Su, S. E. Whang, J. Widom, Swoosh: a generic approach to entity resolution, The VLDB Journal-The International Journal on Very Large Data Bases 18 (1) (2009) 255-276.
  36. H. Köpcke, E. Rahm, Frameworks for entity matching: A comparison, Data & Knowledge Engineering 69 (2) (2010) 197-210.
  37. F. Naumann, M. Herschel, An Introduction to Duplicate Detection, Synthe- sis Lectures on Data Management, Morgan & Claypool Publishers, 2010. doi:10.2200/S00262ED1V01Y201003DTM003.
  38. M. Stonebraker, D. Bruckner, I. F. Ilyas, G. Beskales, M. Cherniack, S. B. Zdonik, A. Pagan, S. Xu, Data curation at scale: The data tamer system, in: CIDR 2013, Sixth Biennial Conference on Innovative Data Systems Re- search, Asilomar, CA, USA, January 6-9, 2013, Online Proceedings, 2013.
  39. G. Papadakis, L. Tsekouras, E. Thanos, G. Giannakopoulos, T. Palpanas, M. Koubarakis, The return of jedai: End-to-end entity resolution for structured and semi-structured data, PVLDB 11 (12) (2018) 1950-1953. doi:10.14778/3229863.3236232.
  40. V. Efthymiou, G. Papadakis, K. Stefanidis, V. Christophides, Simplifying entity resolution on web data with schema-agnostic, non-iterative matching, in: 34th IEEE International Conference on Data Engineering, ICDE 2018, Paris, France, April 16-19, 2018, 2018, pp. 1296-1299. URL https://doi.org/10.1109/ICDE.2018.00134
  41. A. D. Sarma, A. Jain, A. Machanavajjhala, P. Bohannon, CBLOCK: an automatic blocking mechanism for large-scale de-duplication tasks, CoRR abs/1111.3689. arXiv:1111.3689. URL http://arxiv.org/abs/1111.3689
  42. U. Draisbach, F. Naumann, A generalization of blocking and windowing algorithms for duplicate detection, in: 2011 International Conference on Data and Knowledge Engineering, ICDKE 2011, Milano, Italy, September 6, 2011, 2011, pp. 18-24. doi:10.1109/ICDKE.2011.6053920.
  43. L. Gravano, P. G. Ipeirotis, H. V. Jagadish, N. Koudas, S. Muthukrishnan, D. Srivastava, Approximate string joins in a database (almost) for free, in: VLDB 2001, Proceedings of 27th International Conference on Very Large Data Bases, September 11-14, 2001, Roma, Italy, 2001, pp. 491-500.
  44. A. McCallum, K. Nigam, L. H. Ungar, Efficient clustering of high- dimensional data sets with application to reference matching, in: Proceed- ings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, Boston, MA, USA, August 20-23, 2000, 2000, pp. 169-178. doi:10.1145/347090.347123.
  45. G. Simonini, G. Papadakis, T. Palpanas, S. Bergamaschi, Schema-agnostic progressive entity resolution, in: 34th IEEE International Conference on Data Engineering, ICDE 2018, Paris, France, April 16-19, 2018, 2018, pp. 53-64. doi:10.1109/ICDE.2018.00015.
  46. S. E. Whang, D. Marmaros, H. Garcia-Molina, Pay-as-you-go entity res- olution, IEEE Trans. Knowl. Data Eng. 25 (5) (2013) 1111-1124. doi: 10.1109/TKDE.2012.43.
  47. T. Papenbrock, A. Heise, F. Naumann, Progressive duplicate detection, IEEE Trans. Knowl. Data Eng. 27 (5) (2015) 1316-1329. doi:10.1109/ TKDE.2014.2359666.
  48. D. Firmani, B. Saha, D. Srivastava, Online entity resolution using an oracle, PVLDB 9 (5) (2016) 384-395. doi:10.14778/2876473.2876474. URL http://www.vldb.org/pvldb/vol9/p384-firmani.pdf
  49. D. Firmani, S. Galhotra, B. Saha, D. Srivastava, Robust entity resolution using a crowdoracle, IEEE Data Eng. Bull. 41 (2) (2018) 91-103. URL http://sites.computer.org/debull/A18june/p91.pdf
  50. G. Papadakis, G. Papastefanatos, G. Koutrika, Supervised meta-blocking, PVLDB 7 (14) (2014) 1929-1940. doi:10.14778/2733085.2733098.
  51. G. dal Bianco, M. A. Gonçalves, D. Duarte, Bloss: Effective meta-blocking with almost no effort, Information Systems 75 (2018) 75-89.
  52. A. N. Ngomo, S. Auer, LIMES -A time-efficient approach for large-scale link discovery on the web of data, in: IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, July 16-22, 2011, 2011, pp. 2312-2317. doi:10.5591/ 978-1-57735-516-8/IJCAI11-385.
  53. P. Vandenbussche, B. Vatant, Linked open vocabularies, ERCIM News 2014 (96).
  54. S. Bergamaschi, D. Ferrari, F. Guerra, G. Simonini, Y. Velegrakis, Provid- ing insight into data source topics, J. Data Semantics 5 (4) (2016) 211-228. doi:10.1007/s13740-016-0063-6.
  55. L. Kolb, A. Thor, E. Rahm, Dedoop: Efficient deduplication with hadoop, PVLDB 5 (12) (2012) 1878-1881. doi:10.14778/2367502.2367527.
  56. S. Das, P. S. G. C., A. Doan, J. F. Naughton, G. Krishnan, R. Deep, E. Arcaute, V. Raghavendra, Y. Park, Falcon: Scaling up hands-off crowd- sourced entity matching to build cloud services, in: Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD Con- ference 2017, Chicago, IL, USA, May 14-19, 2017, 2017, pp. 1431-1446. doi:10.1145/3035918.3035960.
  57. Y. Altowim, S. Mehrotra, Parallel progressive approach to entity resolution using mapreduce, in: 33rd IEEE International Conference on Data Engi- neering, ICDE 2017, San Diego, CA, USA, April 19-22, 2017, 2017, pp. 909-920. doi:10.1109/ICDE.2017.139.
  58. T. B. Araújo, C. E. S. Pires, T. P. da Nóbrega, Spark-based streamlined metablocking, in: Computers and Communications (ISCC), 2017 IEEE Symposium on, IEEE, 2017, pp. 844-850.
  59. F. Benedetti, D. Beneventano, S. Bergamaschi, G. Simonini, Computing inter-document similarity with context semantic analysis, Inf. Syst. 80 (2019) 136-147. doi:10.1016/j.is.2018.02.009.
  60. S. Bergamaschi, L. Gagliardelli, G. Simonini, S. Zhu, Bigbench workload executed by using apache flink, Procedia Manufacturing 11 (2017) 695-702.
  61. F. Guerra, G. Simonini, M. Vincini, Supporting image search with tag clouds: A preliminary approach, Adv. in MM 2015 (2015) 439020:1- 439020:10. doi:10.1155/2015/439020.

Related papers

Parallel meta-blocking: Realizing scalable entity resolution over large, heterogeneous data
Kostas Stefanidis

2015 IEEE International Conference on Big Data (Big Data), 2015

Entity resolution constitutes a crucial task for many applications, but has an inherently quadratic complexity. Typically, it scales to large volumes of data through blocking: similar entities are clustered into blocks so that it suffices to perform comparisons only within each block. Meta-blocking further increases efficiency by cleaning the overlapping blocks from unnecessary comparisons. However, even Meta-blocking can be time-consuming: applying it to blocks with 7.4 million entities and 2.2•10 11 comparisons takes almost 8 days on a modern high-end server. In this paper, we parallelize Metablocking based on MapReduce. We propose a simple strategy that explicitly creates the core concept of Meta-blocking, the blocking graph. We then describe an advanced strategy that creates the blocking graph implicitly, reducing the overhead of data exchange. We also introduce a load balancing algorithm that distributes the computationally intensive workload evenly among the available compute nodes. Our experimental analysis verifies the superiority of our advanced strategy and demonstrates an almost linear speedup for all meta-blocking techniques with respect to the number of available nodes.

View PDFchevron_right
Schema-agnostic blocking for streaming data
Jyrki Nummenmaa

Proceedings of the 35th Annual ACM Symposium on Applied Computing, 2020

Currently, a wide number of information systems produce a large amount of data continuously. Since these sources may have overlapping knowledge, the Entity Resolution (ER) task emerges as a fundamental step to integrate multiple knowledge bases or identify similarities between entities. Considering the quadratic cost of the ER task, blocking techniques are often used to improve efficiency. Such techniques face two main challenges related to data volume (i.e., large data sources) and variety (i.e., heterogeneous data). Besides these challenges, blocking techniques also face two other ones: streaming data and incremental processing. To address these four challenges simultaneously, we propose PI-Block, a novel incremental schema-agnostic blocking technique that utilizes parallelism (through distributed computational infrastructure) to enhance blocking efficiency. In our experimental evaluation, we use four real-world data source pairs, and highlight that PI-Block achieves better results regarding efficiency and effectiveness compared to the state-of-the-art technique.

View PDFchevron_right
Parallel meta-blocking for scaling entity resolution over big heterogeneous data
Kostas Stefanidis

Information Systems, 2017

Entity resolution constitutes a crucial task for many applications, but has an inherently quadratic complexity. In order to enable entity resolution to scale to large volumes of data, blocking is typically employed: it clusters similar entities into (overlapping) blocks so that it suffices to perform comparisons only within each block. To further increase efficiency, Meta-blocking is being used to clean the overlapping blocks from unnecessary comparisons, increasing precision by orders of magnitude at a small cost in recall. Despite its high time efficiency though, using Meta-blocking in practice to solve entity resolution problem on very large datasets is still challenging: applying it to 7.4 million entities takes (almost) 8 full days on a modern high-end server. In this paper, we introduce scalable algorithms for Meta-blocking, exploiting the MapReduce framework. Specifically, we describe a strategy for parallel execution that explicitly targets the core concept of Meta-blocking, the blocking graph. Furthermore, we propose two more advanced strategies, aiming to reduce the overhead of data exchange. The comparison-based strategy creates the blocking graph implicitly, while the entity-based strategy is independent of the blocking graph, employing fewer MapReduce jobs with a more elaborate processing. We also introduce a load balancing algorithm that distributes the computationally intensive workload evenly among the available compute nodes. Our experimental analysis verifies the feasibility and superiority of our advanced strategies, and demonstrates their scalability to very large datasets.

View PDFchevron_right
Meta-Blocking: Taking Entity Resolution to the Next Level
Georgia Koutrika, George Papadakis

IEEE Transactions on Knowledge and Data Engineering, 2000

Entity Resolution is an inherently quadratic task that typically scales to large data collections through blocking. In the context of highly heterogeneous information spaces, blocking methods rely on redundancy in order to ensure high effectiveness at the cost of lower efficiency (i.e., more comparisons). This effect is partially ameliorated by coarse-grained block processing techniques that discard entire blocks either a-priori or during the resolution process. In this paper, we introduce meta-blocking as a generic procedure that intervenes between the creation and the processing of blocks, transforming an initial set of blocks into a new one with substantially fewer comparisons and equally high effectiveness. In essence, meta-blocking aims at extracting the most similar pairs of entities by leveraging the information that is encapsulated in the block-to-entity relationships. To this end, it first builds an abstract graph representation of the original set of blocks, with the nodes corresponding to entity profiles and the edges connecting the co-occurring ones. During the creation of this structure all redundant comparisons are discarded, while the superfluous ones can be removed by pruning of the edges with the lowest weight. We analytically examine both procedures, proposing a multitude of edge weighting schemes, graph pruning algorithms as well as pruning criteria. Our approaches are schema-agnostic, thus accommodating any type of blocks. We evaluate their performance through a thorough experimental study over three large-scale, real-world datasets, with the outcomes verifying significant efficiency enhancements at a negligible cost in effectiveness.

View PDFchevron_right
Incremental Entity Blocking over Heterogeneous Streaming Data
Tiago Brasileiro

Information

Web systems have become a valuable source of semi-structured and streaming data. In this sense, Entity Resolution (ER) has become a key solution for integrating multiple data sources or identifying similarities between data items, namely entities. To avoid the quadratic costs of the ER task and improve efficiency, blocking techniques are usually applied. Beyond the traditional challenges faced by ER and, consequently, by the blocking techniques, there are also challenges related to streaming data, incremental processing, and noisy data. To address them, we propose a schema-agnostic blocking technique capable of handling noisy and streaming data incrementally through a distributed computational infrastructure. To the best of our knowledge, there is a lack of blocking techniques that address these challenges simultaneously. This work proposes two strategies (attribute selection and top-n neighborhood entities) to minimize resource consumption and improve blocking efficiency. Moreover,...

View PDFchevron_right

Related topics

  • Information Systems
  • Computer Science
  • Entity Resolution
  • Star Schema

    • Cited by

    An Overview of End-to-End Entity Resolution for Big Data
    Vassilis Christophides

    ACM Computing Surveys

    One of the most critical tasks for improving data quality and increasing the reliability of data analytics is Entity Resolution (ER), which aims to identify different descriptions that refer to the same real-world entity. Despite several decades of research, ER remains a challenging problem. In this survey, we highlight the novel aspects of resolving Big Data entities when we should satisfy more than one of the Big Data characteristics simultaneously (i.e., Volume and Velocity with Variety). We present the basic concepts, processing steps, and execution strategies that have been proposed by database, semantic Web, and machine learning communities in order to cope with the loose structuredness , extreme diversity , high speed, and large scale of entity descriptions used by real-world applications. We provide an end-to-end view of ER workflows for Big Data, critically review the pros and cons of existing methods, and conclude with the main open research directions.

    View PDFchevron_right
    Generalized supervised meta-blocking
    Sonia BERGAMASCHI

    Proceedings of the VLDB Endowment

    Entity Resolution is a core data integration task that relies on Blocking to scale to large datasets. Schema-agnostic blocking achieves very high recall, requires no domain knowledge and applies to data of any structuredness and schema heterogeneity. This comes at the cost of many irrelevant candidate pairs (i.e., comparisons), which can be significantly reduced by Meta-blocking techniques that leverage the entity co-occurrence patterns inside blocks: first, pairs of candidate entities are weighted in proportion to their matching likelihood, and then, pruning discards the pairs with the lowest scores. Supervised Meta-blocking goes beyond this approach by combining multiple scores per comparison into a feature vector that is fed to a binary classifier. By using probabilistic classifiers, Generalized Supervised Meta-blocking associates every pair of candidates with a score that can be used by any pruning algorithm. For higher effectiveness, new weighting schemes are examined as featur...

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved